[gdal-dev] Arc File Geodatabase

[Richard Barnes]

I'm not sure that I have time to do a full integration with GDAL. But I 
did continue my previous line of work far enough that I now have a 
decent program for extracting the raster data into GeoTIFFs. It's 
available at: https://github.com/r-barnes/ArcRasterRescue

It seems each raster is associated with five gdbtables. From these the 
program makes a best effort to obtain a CRS WKT and geotransform. It 
also determines the data type and performs endian conversion. 
Decompression is still limited to LZ77 and uncompressed formats.

Best regards,
Richard

On 08/21/2016 05:17 AM, James Ramm wrote:
> Yes, there is some form 2d index - each raster has a corresponding 
> .col_index.atx, .row_index.atx, _.blk_key_index.atx and 
> .band_index.atx files. I've not looke at how they work together so far.
>
>
>
> On Sunday, 21 August 2016, Even Rouault <even.rouault at spatialys.com 
> <mailto:even.rouault at spatialys.com>> wrote:
>
>     Hi Richard,
>
>     > I saw a posting from July
>     >
>     <https://lists.osgeo.org/pipermail/gdal-dev/2016-July/044761.html
>     <https://lists.osgeo.org/pipermail/gdal-dev/2016-July/044761.html>>
>     about
>     > work towards deciphering the Arc File Geodatabase Raster format.
>     >
>     > Inspired, I did some work today and successfully extracted a
>     raster from
>     > a geodatabase.
>     >
>     > Using Even Rouault's /dump_gdbtable/ as a starting point, I
>     decompressed
>     > the raster blobs using zlib, as suggested in the posting.
>     >
>     > For my raster, this gave 128x128 big-endian 32-bit
>     floating-point values
>     > with a trailer of about 512 bytes.
>     >
>     > Assembling these 128x128 tiles using the col_nbr and row_nbr fields
>     > resulted in the raster coming out.
>     >
>     > I've uploaded my fork, along with appropriate input data and a
>     command
>     > to run it all (see README.md) to
>     https://github.com/r-barnes/dump_gdbtable
>     <https://github.com/r-barnes/dump_gdbtable>.
>     >
>     > What remains is to link metadata (block size, band_width,
>     band_height)
>     > to the raster so the output can be sized appropriately, the data
>     type
>     > deduced automatically, and projections saved. This is probably
>     just a
>     > matter of linking the appropriate rows from some of the
>     lower-numbered
>     > gdbtables with the files containing the raster data. It might
>     also be
>     > nice to determine whether zlib is always being used, but I assume an
>     > error could be thrown if an unexpected compression format is
>     encountered.
>
>     One can expect other compression formats according to :
>     http://desktop.arcgis.com/en/arcmap/10.3/manage-data/raster-and-images/raster-compression.htm
>     <http://desktop.arcgis.com/en/arcmap/10.3/manage-data/raster-and-images/raster-compression.htm>
>     None of them seem to be particularly difficult given what GDAL
>     supports ("raw" RLE and "raw" LZW
>     would probably require a bit more work as there is no direct
>     drivers that support them, although they
>     are codecs of some other GDAL drivers like TIFF, and so could be
>     perhaps wrapped on the fly in a in-memory TIFF)
>
>     What they call LZ77 must be the ZLib one (according to
>     https://en.wikipedia.org/wiki/Zlib
>     <https://en.wikipedia.org/wiki/Zlib> ZLib's DEFLATE is a modified
>     LZ77)
>
>     >
>     > I'm interested in seeing GDAL include the ability to translate
>     Arc File
>     > Geodatabase Rasters into other (less proprietary) formats.
>     > What is a good way to move forward with that?
>
>     "A bit" of coding !
>     ogr/ogrsf_frmts/openfilegdb/filegdbtable.cpp should probably be
>     extended to offer
>     the low level access to the rasters.
>     And ogropenfilegdbdatasource.cpp to implement the GDAL Raster API
>     using the above.
>     Regarding efficiency of random reading of tiles, I'm wondering if
>     they have some form of 2D (row, column)
>     indexing or perhaps 3D (row, column, pyramid_level). For the
>     vector part, I only addressed the case
>     of 1D indexes.
>     As a fallback sequential reading through the records could still
>     be used, perhaps with some tricks to read only the start
>     of the records to get the (row, column, pyramid_level) values, if
>     located at the beginning of it,
>     and not the raster data if not needed.
>
>     Even
>
>     >
>     > Best regards,
>     > Richard Barnes
>
>     --
>     Spatialys - Geospatial professional services
>     http://www.spatialys.com
>     _______________________________________________
>     gdal-dev mailing list
>     gdal-dev at lists.osgeo.org <javascript:;>
>     http://lists.osgeo.org/mailman/listinfo/gdal-dev
>     <http://lists.osgeo.org/mailman/listinfo/gdal-dev>
>

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20160829/20d11389/attachment.html>