[gdal-dev] Arc File Geodatabase

[Even Rouault]

Hi Richard,

> I saw a posting from July
> <https://lists.osgeo.org/pipermail/gdal-dev/2016-July/044761.html> about
> work towards deciphering the Arc File Geodatabase Raster format.
> 
> Inspired, I did some work today and successfully extracted a raster from
> a geodatabase.
> 
> Using Even Rouault's /dump_gdbtable/ as a starting point, I decompressed
> the raster blobs using zlib, as suggested in the posting.
> 
> For my raster, this gave 128x128 big-endian 32-bit floating-point values
> with a trailer of about 512 bytes.
> 
> Assembling these 128x128 tiles using the col_nbr and row_nbr fields
> resulted in the raster coming out.
> 
> I've uploaded my fork, along with appropriate input data and a command
> to run it all (see README.md) to https://github.com/r-barnes/dump_gdbtable.
> 
> What remains is to link metadata (block size, band_width, band_height)
> to the raster so the output can be sized appropriately, the data type
> deduced automatically, and projections saved. This is probably just a
> matter of linking the appropriate rows from some of the lower-numbered
> gdbtables with the files containing the raster data. It might also be
> nice to determine whether zlib is always being used, but I assume an
> error could be thrown if an unexpected compression format is encountered.

One can expect other compression formats according to :
http://desktop.arcgis.com/en/arcmap/10.3/manage-data/raster-and-images/raster-compression.htm
None of them seem to be particularly difficult given what GDAL supports ("raw" RLE and "raw" LZW
would probably require a bit more work as there is no direct drivers that support them, although they
are codecs of some other GDAL drivers like TIFF, and so could be perhaps wrapped on the fly in a in-memory TIFF)

What they call LZ77 must be the ZLib one (according to
https://en.wikipedia.org/wiki/Zlib ZLib's DEFLATE is a modified LZ77)

> 
> I'm interested in seeing GDAL include the ability to translate Arc File
> Geodatabase Rasters into other (less proprietary) formats.
> What is a good way to move forward with that?

"A bit" of coding !
ogr/ogrsf_frmts/openfilegdb/filegdbtable.cpp should probably be extended to offer
the low level access to the rasters.
And ogropenfilegdbdatasource.cpp to implement the GDAL Raster API using the above.
Regarding efficiency of random reading of tiles, I'm wondering if they have some form of 2D (row, column)
indexing or perhaps 3D (row, column, pyramid_level). For the vector part, I only addressed the case
of 1D indexes.
As a fallback sequential reading through the records could still be used, perhaps with some tricks to read only the start
of the records to get the (row, column, pyramid_level) values, if located at the beginning of it,
and not the raster data if not needed.

Even

> 
> Best regards,
> Richard Barnes

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com