[postgis-devel] Postgis Raster JPEG 2000 (openjpeg)

Paul Ramsey pramsey at cleverelephant.ca
Tue Feb 18 09:59:57 PST 2014


Note that PgSQL will be applying libz compression to over-sized tuples automatically in the TOAST machinery, so applying it yourself won’t radically change anything, unless it’s being done carefully within a blocking scheme, perhaps.

Also note that no matter what you do, in-database data is going to be cut up into page-size hunks and dropped into tables, it’s never stored directly on the file system. That’s true for blobs, and true for toasted tuples too.

P.

-- 
Paul Ramsey
http://cleverelephant.ca
http://postgis.net

On February 18, 2014 at 9:56:02 AM, Nathaniel Clay (clay.nathaniel at gmail.com) wrote:




On Tue, Feb 18, 2014 at 11:24 AM, Bborie Park <dustymugs at gmail.com> wrote:
How does OpenJPEG 2000 compare to other formats, such as GeoTIFF, HDF5 or NetCDF?
 All of the mentioned, formats use libz compression, a loss less compression scheme. With the exception that GTiff offers plain jpeg compression along with a fax format??
I think it may be a good idea to also ofter libz (deflate) and or LZMA compression.

Before I move on to the next questions a little background on my thought process.
I see three different use types:
1. Active Server

Before deciding on a replacement format, we should identify if a replacement is required? What are the current short-comings? Where are the performance limitations? What are the wish-list items for a replacement format?

I have spent some time at poking at what I considered short-comings, where I think performance is suffering, and what features are missing. I'm not ready to publicly share my list though.

I wasn't involved with the original PostGIS raster binary format but from my understanding while working it with, it appears as though the goal was to keep things simple as all input and output goes through PostgreSQL, which can (should?) be considered a black-box.

-bborie


On Mon, Feb 17, 2014 at 6:32 PM, Nathaniel Clay <clay.nathaniel at gmail.com> wrote:
Hi all,


This is just some food for thought and discussion. I've take a cursory look at JPEG 2000 openjpeg. I think it may hold promise for a possible future format or co-format for Postgis Raster in database storage.
Supports both Loss less and Lossful Compression approaching the performance of ERDAS ( uncompressed Gtif 200M, openjpeg JP2  25M Lossfull and 80M Lossless, ERDAS JP2 15M)
I don't have a ERADAS encoder license, but this image was of the same original
Openjpeg lossfull compression in my opinion visually identical to the original with default settings
Supports N band imagery
Multi-Resolution Packing
eg more than one raster resolution with in the "code stream"
can be ordered by user specification, default smallest to largest
This would allow us to get rid of the overview tables or make them views.
http://ericportis.com/posts/2014/resolution-progressive-jpeg2000-performance/
Extensible header
SRID, IPX,IPY, etc
Area of interest specification 
Allows you to specify a region with in the same raster to have a higher bitrate, etc
Error Detection and Correction
Openjpeg project seems to be alive last commit was a few days a go.
BSD License


Possible Concerns identified so far:

Writes require a recode of the entire tile, however the tile size is user specifiable
takes a long time to encode image, ~10 sec on my system with the test image ( 3067x3777 8bit 4bands) 
I dont have anything to compare it to, so I dont know if this is a good or bad
I havent tested on small image size and other possible scenarios. 
Memory Usage seemed high during decode and encode, ~2G, I believe this is due to the decoder and encoder preallocate memory for all of the "Pyramids" and bands at once. (but have not investigated thoroughly) ( Image Res 3067x 3777  8bit  4 bands)
Its included decoder application has weird issues with tif, other formats are fine. ( Dont think it has anything to do with the openjpeg lib)
May be indective of other problems
If I were to start on this project it would be some time in the future and I would most likely require help and lots of input, but I thought I it would be a good idea to bring, it up as a discussion point.

Any thoughts,



Nathaniel Hunter Clay


_______________________________________________
postgis-devel mailing list
postgis-devel at lists.osgeo.org
http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel


_______________________________________________
postgis-devel mailing list
postgis-devel at lists.osgeo.org
http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel

_______________________________________________  
postgis-devel mailing list  
postgis-devel at lists.osgeo.org  
http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20140218/f6851297/attachment.html>


More information about the postgis-devel mailing list