[postgis-devel] Postgis Raster JPEG 2000 (openjpeg)

[Pierre Racine]

My two cents:

- Answer Bborie's questions first: What are the current short-comings? Where are the performance limitations? What are the wish-list items for a replacement format?

- There would have been an advantage to store PostGIS tiles if no importation step would be needed. But you have to import (which actually means convert) rasters anyway. So there must be others REAL pros for using an existing raster format that might bring unnecessary complexity in the DB code. For example we don't support color tables, color scheme or embedded statistics for the sake of simplicity. You can get to that otherwise by using other columns in your table or extra tables. Many choices were made for PostGIS raster because we are in the context of a database, not a raster file. Most people understand raster as a big unique file in the filesystem. This has size limitations. I prefer to view it as a tiled coverage of gridded data. The best way to see a table of rasters in PostGIS is like a folder of tiles in the filesystem. But with a sophisticated indexing and the possibility to append metadata to each tile. I had long fights about this with people used to GDAL and I think, in the end, I proved to be right. All that to say the simpler the better, think database and I'm affraid JPEG 2000 would bring unnecessary complexity.

- PostGIS raster was designed mainly to support raster analysis. I contrast rasters with images this way: a rasters is a generalization of an images in that it only carries data, no interpretation of it. Images are more specific in that they carry visual (color) information and hence an interpretation of the data they contains. This is why images support color scheme (color table, RGB, RGBA, greyscale) and other features and not rasters (according to my distinction). I thing images brings in a level of complexity that is not necessary for raster analysis. 

- As Bborie said we already support out-db JPEG 2000 through GDAL. We could maybe say that JPEG 2000 is the preferred out-db format but I think GeoTIFF match better the PostGIS raster internal format.

- Multi-resolution in the same tile was planned at the beginning. It works for file system images but not for tiles coverages where each tile would embed its own lower resolutions. You have to merge tiles together when creating lower resolutions. In the end there is not a one to one relationship between higher and lower resolution tiles. There must be less tiles in the lower resolution version. If you don't merge lower resolution tiles you become limited in the number of lower resolutions you can provide. I can build an example to illustrate this if it's not clear...

-Does JPEG 2000 supports different pixel types for different bands? This is one of the very important particularities of PostGIS raster.

I still don't see one good argument in favor of replacing the actual format or adding JPEG 2000 as an internal format that would bring killer features.

Pierre

> -----Original Message-----
> From: postgis-devel-bounces at lists.osgeo.org [mailto:postgis-devel-
> bounces at lists.osgeo.org] On Behalf Of Nathaniel Clay
> Sent: Thursday, February 20, 2014 8:46 AM
> To: Rémi Cura
> Cc: PostGIS Development Discussion
> Subject: Re: [postgis-devel] Postgis Raster JPEG 2000 (openjpeg)
> 
> Remi,
> 
> 
> JPEG 2000 offers several formats, one being JP2C or code stream. This
> format breaks down the file (intended for streaming across the wire)
> internally into tiles with markers then inside the tile markers packets
> containing the data for the specified tile. My initial thought is to store each
> of the tile packet streams in a row that is indexed in a table with in row
> separation of the packets for each of the resolutions (this appears to be
> able to be done with out an decode or encode for reassembly). Thus Union
> and simple Clip (with out trimming) operations would be copy and modify
> the header information operation back into a valid JP2C code stream to be
> sent directly to the client or consuming function/application for decode.
> JPEG 2000 is a large specification with many subparts, I am in the process of
> preparing two emails one to the list answering Bories' questions (more
> general and format neutral) , and another with more specifics of my idea for
> JPEG 2000, internal database format storage.  I am taking the time to
> confirm the specifics as related to the JPEG 2000 specification and
> postgresql.
> 
> 
> Thanks,
> 
> 
> Nathaniel Hunter Clay