[gdal-dev] Fwd: GPKG Driver

[Even Rouault]

Paul,

> <even.rouault at mines-paris.org> wrote:
> > Now my turn for questions :
> > - From your questions, I suppose you only consider the vector side of
> > GPKG ?
> 
> Yes, that's my initial target.  Any comments/ideas on how to share
> functionality between GDAL and OGR drivers, since presumably
> eventually the gpkg support will end up with one of each.

We have a few precedents of formats having a GDAL and OGR side : PCIDSK, SDTS, 
PDF, ...
As OGR can be disabled at build time ( OGR_ENABLED=NO , not sure who relies on 
that apart from ESRI maybe), but not the other way round, the OGR driver 
includes header files from the related GDAL driver. Usually not the GDAL driver 
code itself, but some code that forms an embedded library that deals with the 
format internals.
Here you could have a frmts/gpkg directory with a few common shared methods to 
deal with spatial_ref_sys, gpkg_contents, etc ?

To give you the full picture with the risk of making things confusing, I 
should mention that in the case of SQLite, the GDAL Rasterlite and MBTiles 
drivers actually depend on OGR (they are not compiled if OGR_ENABLED = NO). 
They do not actually depend on sqlite3.h, but they use the OGR API to trigger 
indirectly the OGR SQLite driver and let him process the SQL requests they 
need. So dynamic dependency rather than a static one.

> 
> > - Do you intend in writing everything from scratch, or did you consider
> > using https://bitbucket.org/luciad/libgpkg ?  Which is, AFAIK, the
> > equivalent of libspatialite for GPKG. Contains the SQL functions
> > mandated by the spec, the triggers etc.
> 
> I think most of the storage/retrieval stuff is going to be from
> scratch.

Yes, reading or writing GPKG geometry blobs can easily be done from scratch 
(as opposed to using ST_AsBinary() / ST_GeomFromWKB() ). 

> The more operational stuff, like rtree index bindings and so
> on, could/should lean on the luciad code (though I don't actually see
> those mentioned). 

https://bitbucket.org/luciad/libgpkg/overview mentions a CreateSpatialIndex()

> I'm loath to add a build dependency, but perhaps
> copying the tree into the driver would be acceptable? (Except it's a
> cmake project, and gdal is not, hm)
> 
> What was your thought?

That was an open question. There are clearly pros and cons in using or not 
using it. My experience with spatialite integration is that I initially f**ked 
up write support when I tried to do it myself without using spatialite support 
functions, since I omitted creating triggers that were essential for database 
integrity. The main issue of course was that there was/is no spatialite spec 
to follow. With GPKG, all the triggers are mentionned in the spec, so that 
should be doable at hand.

About adding a build dependency, yes it is annoying. But if we copy the 
libgpkg tree in GDAL, we should still make sure that it can also be built 
against external libgpkg ( a bit like we have libtiff in frmts/gtiff/libtiff but 
we can build against esternal libtiff ), which is something demanded by 
packagers of Linux distro (to ease updates, particularly security related). I 
see that the libgpkg also ships with an internal sqlite source tree. So 
dependency inside a dependency ... ?
SQLite is not a build requirement of GDAL, but it is still extensively used in 
quite a few drivers (MBTiles, Rasterlite, OGR SQLite, an advanced module in 
the GML, OGR VFK, OGR OSM).
At first, I though that it would be odd to internalize libgpkg and not SQLite, 
but in practice SQLite is a dependency available to all packagers (on Linux of 
course, as well as on OSGeo4W), so there's not a really interest in 
internalizing it.
On the contrary libgpkg will likely stay for some time a niche library, not 
easily found already packaged.
A drawback of internal copies of libraries is the risk of ABI incompatibility 
when a code links against GDAL and the external version of the internal 
library. Something that has given us a few headaches in the past, with GDAL 
internal libtiff 4.X vs external libtiff 3.X, particularly on Linux where symbol 
visibility is quite hard to control. I ended up generating all necessary 
"#define gdal_TIFFxxxxx TIFFxxxxx" for the internal copy. Not elegant, but this 
works. For libgpkg this is not something we will have to worry about for some 
time, unless someone decides to write a standalone QGIS data provider that 
doesn't use OGR.

Regarding cmake, I guess that adding the names of the .c files in a GNUmakefile 
/ makefile.vc should do it. One worry I have is the minimum MSVC version to 
build it. In libgpkg history I see a mention of Visual Studio Express 2012. 
Don't know if it works with older versions but I can see some use of C99 
(int64_t) extensions that will likely not please older compiler versions.

Even

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html