[gdal-dev] OpenFileGDB 10 MB field description limitation

[Even Rouault]

Le vendredi 17 octobre 2014 20:31:00, Matthias Uden a écrit :
> Hi list,
> 
> I'm running GDAL 1.11.1 on an Ubuntu 14.04.1 64-bit system.
> 
> ogr2ogr fails while trying to convert a Feature Class from a rather huge
> FileGDB to a PostGIS table. Unfortunately, I have currently no means to
> access the original GDB file directly.
> It gives this error:
> 
> ERROR 1: Error occured in filegdbtable.cpp at line 659
> 
> Which breaks down to this line of code:
> 
> returnErrorIf
> <http://fossies.org/dox/gdal-1.11.1/filegdbtable__priv_8h.html#a2386c8f3b27
> 01f62e44b603c048cab4b>(nFieldDescLength
> 
>  > 10 * 1024 * 1024 || nFieldDescLength < 10 );
> 
> By putting in a debug output I figured that the "nFieldDescLength" of my
> feature class is /way/ larger than the maximum allowed value of 10 MB.
> In fact it's something around 1.6 GiB.... :-o   (what kind of field
> description is *this*...?)

Matthias,

This is the size of the area in the table that describe the fields (their name, 
type, etc..). 10 MB is already a huge value (you can likely describe 10000 
attributes or more). If you get 1.6 GB for that field, then something is wrong 
before.

Access to the file for inspection would be needed.

> 
> Does anyone of you have an idea on what exactly the "nFieldDescLength"
> is and - more importantly - why it's limited to 10MB and if further
> increasing this value would do any harm to the overall process?

That could lead to 1.6 GB being allocated in RAM and being read from the file 
(if it's bigger than 1.6 GB), so a bad idea. The sanity check must remain to 
avoid this.

> Is a
> field description of 1.6 gigs even realistic at all or is it more likely
> that I have a corrupt input file in some way?

Perhaps not corrupt, but it potentially exhibits a hole in the reverse 
engineering of the format.

> Can't I opt to just skip
> the lousy field description import and still import what I'm interested
> in - the /data/ itself?
> 
> Apparently there was already a bug ticket for this problem, which led to
> increasing the max. value from 1 to 10 MB [1]

That was just a blind guess, and I'm not convinced it actually helped for the 
ticket mentionned.

> Otherwise I'll have to try if the proprietary ESRI driver can handle 
> this... (not tested yet).

That could be a good hint if the file is corrupt or not. Note: if the GDB is a 
GDB 9.X, then the FGDB SDK driver will not be able to open it.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com