[gdal-dev] RFC 31 - OGR 64bit Support

[Even Rouault]

Frank,

If I agree we can upgrade the C++ API of OGRLayer::GetFeature() and 
DeleteFeature() to use GIntBig instead of long, I'm wondering whether the ABI 
backward incompatible change of OGR_L_GetFeature() and OGR_L_DeleteFeature() 
is something acceptable with respect to our (implicit) policy and past 
practices as far as the C ABI is concerned ? I don't remind of such a 
precedent in the 1.X series (at least since X >= 4). Couldn't this be "solved" 
by introducing OGR_L_GetFeature64() and OGR_L_SetFeature64() and let 
OGR_L_GetFeature() and OGR_L_SetFeature() do the cast from long to GIntBig ? I 
acknowledge this isn't pretty...

Or, if we don't want the OGR_L_GetFeature64/SetFeature64 symbols to be too 
apparent, we could also use a little trick like the following one :

* in ogr_api.h :

/* Do not use OGR_L_GetFeature64() directly. Use OGR_L_GetFeature instead */
/* OGR_L_GetFeature64() will be renamed OGR_L_GetFeature() in GDAL 2.0 */
OGRFeatureH CPL_DLL OGR_L_GetFeature64( OGRLayerH, GIntBig );
#define OGR_L_GetFeature OGR_L_GetFeature64

That way, application code recompiled against the GDAL 1.8.0 header will use 
the 64bit version implicitely.

* and in ogrlayer.cpp :

/* Hack to preserve GDAL < 1.8.0 ABI : to be removed for GDAL 2.0 */
#undef OGR_L_GetFeature
OGRFeatureH CPL_DLL OGR_L_GetFeature( OGRLayerH hLayer, long nFeatureId );
OGRFeatureH OGR_L_GetFeature( OGRLayerH hLayer, long nFeatureId )
{
	return (OGRFeatureH) ((OGRLayer *)hLayer)->GetFeature( 
(GIntBig)nFeatureId );
}

OGRFeatureH OGR_L_GetFeature64( OGRLayerH hLayer, GIntBig nFeatureId )
{
	return (OGRFeatureH) ((OGRLayer *)hLayer)->GetFeature( nFeatureId );
}

that way the old symbol and the new symbol are available...


Concerning the impact of introducing OFTInteger64 for existing applications, 
it is not obvious to predict it. From a quick "grep OFTInteger" in 2 popular 
applications :
* MapServer would need a minor update in mapogr.cpp. If code left as it is 
now, OFTInteger64 fields would be advertized as 'Character' fields in the GML 
schema instead of Integer with appropriate width (the only case OFTInteger in 
mapogr.cpp). Also, I'm not sure if the OGR field type is tested indirectly 
afterwards in the vector rendering/expression evaluation or if just a atof() 
is done on the field value. And in the new mapogroutput.c, according to the 
width of a item->type == "Integer", we could appropriately create a 
OFTInteger64. 
* The OGR provider in QGIS if unchanged would also present OFTInteger64 fields 
as QVariant::String fields as a default (but the values couldn't be 
edited/updated). I see that other QGIS providers use QVariant::LongLong, so 
OFTInteger64 could potentially be properly supported. Other places such as 
src/core/qgsvectorfilewriter.cpp currently turns QVariant::LongLong into 
OFTString when creating the OGR fields and could also be updated.

So if unchanged, it *looks* like those applications should still go on working 
(at least not crashing), but with perhaps a bit degraded functionnality w.r.t 
to the current situation (where the big integers are returned as double for 
shapefiles). For MapServer case, this is mitigated since OGR isn't actually 
used for access to shapefiles and postgis ...

Best regards,

Even

> Folks,
> 
> In the interest of moving towards a GDAL/OGR 1.8 release I have done some
> work on RFC 31 - 64bit integers for OGR fields and FIDs.  I've updated the
> RFC itself somewhat, and done a preliminary implementation of the RFC on
> my own system with a reasonable level of success.
> 
> I'm asking for additional comment on the RFC at:
> 
>    http://trac.osgeo.org/gdal/wiki/rfc31_ogr_64
> 
> The implications and complications of this update are greater than I had
> first anticipated and even with what I've done so far I'm sure lots of
> additional glitches will come out of the wood work.  Unfortunately, this
> is not funded work for me so it is hard to apply the level of effort that
> I would like to it.  In short, I'm a bit nervous about implementing this
> RFC though now that I've sunk 2-3 days into the prototype work I'm hesitant
> to throw it away either. :-)
> 
> Thoughts welcome.
> 
> Best regards,