[gdal-dev] Discoverability of the new proj.db and GDAL_DATA/PROJ_LIB?

[Even Rouault]

On mardi 8 janvier 2019 15:16:47 CET Sean Gillies wrote:
> Hi,
> 
> I have a couple questions about discoverability of the new proj.db file and
> how I should prepare my Rasterio project for the changes. Background: I am
> publishing rasterio wheels on the Python package index that include GDAL
> and PROJ shared libraries and the data files so that users can `pip install
> rasterio` and get something with batteries included. Rasterio has an Env
> class that is to be used as a context manager and it ensures that PROJ_LIB
> (the env variable) and GDAL_DATA (the config option, not env variable) are
> set to the proper paths, paths within the installed wheel adjacent to
> rasterio's Python modules.
> 
> It seems like I'll be including proj.db in future wheels, which shouldn't
> be a problem.

proj.db will be searched in PROJ_LIB if defined, or otherwise in the
${prefix}/share/proj directory if you ./configure --with-prefix=${prefix}
Is the issue that binaries of wheels aren't installed in fix locations ?

PROJ 6 new API has a proj_context_set_database_path() function but I've not 
used/exposed through GDAL for now. It is something per-context (and thus 
potentially per thread since in my new GDAL branch PROJ contexts are per-
thread variables, which are not exposed in GDAL API), and only for proj.db 
itself, not other PROJ resources (such as grid files) normally accessible 
through PROJ_LIB / ${prefix}/share/proj

There's also a related ticket in PROJ tracker:
https://github.com/OSGeo/proj.4/issues/1150
We'd probably need to expose pj_set_searchpath() in the public PROJ API.

> 
> When proj.db replaces the .csv files in GDAL and PROJ, will the GDAL_DATA
> configuration option become obsolete?

GDAL_DATA will still be used by some drivers to retrieve resource files, but 
no longer for CRS related operations.

> 
> I've just thought of a question about data file caching that I want to tack
> on. I don't think it's widely understood by users, but GDAL caches data
> read from the .csv files. For example, you can set GDAL_DATA to a custom
> location (like rasterio does), call a function that requires .csv data,
> then return and call that function again with GDAL_DATA unset and get the
> right answer. 

That's an implementation detail.

> This is a good thing, because otherwise we'd be reading from
> disk every time. My question is whether or not this will change when we
> move over to the new PROJ. Will data be cached or will we need the path to
> proj.db to be configured all the time?

There will be some caching, but on a least-recently-used way, not full 
caching, and not on everything, so access to proj.db might occur at any time.

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com