[PROJ] specifying the search paths

[Roger Bivand]

On Wed, 1 Jul 2020, Even Rouault wrote:

> On mercredi 1 juillet 2020 17:33:05 CEST Roger Bivand wrote:
>> Even,
>>
>> On Wed, 1 Jul 2020, Even Rouault wrote:
>>> Roger,
>>>
>>>> I have a question about the use of
>>>> proj_context_set_search_paths(PJ_DEFAULT_CTX, ...). On initiating PRØJ, I
>>>> see from proj_info().searchpath that (now 7.1.0) I have three paths, the
>>>> last two identical /usr/local/share/proj, and the first
>>>> $HOME/.local/share/proj - which contains a cache.db.
>>>>
>>>> For testing on other platforms than my own, I need to change the first
>>>> path to a temporary directory with write access, so that I do not write
>>>> in
>>>> the user space of the test platform. So I use
>>>> proj_context_set_search_paths(PJ_DEFAULT_CTX, ...), providing the correct
>>>> length of the vector of paths, and the paths themselves. The
>>>> user-writeable path is first, as at startup. It, however, does not
>>>> contain
>>>> a cache.db file, even of zero length. After finding a coordinate
>>>> operation
>>>> (using a grid cached in $HOME/.local/share/proj), the temporary directory
>>>> remains empty, $HOME/.local/share/proj is used despite not being on the
>>>> active path list, and the operation completes successfully.
>>>>
>>>> This is, however, not what I need - I need the download to occur to the
>>>> temporary directory, so that I can demonstrate that cache.db gets
>>>> populated.
>>>>
>>>> May libproj be caching paths so that even if the path to the
>>>> use-writeable
>>>> directory has been set, it still uses the one determined on load, even if
>>>> it had been renamed and is not on the this of search paths? I find the
>>>> default use-writeable directory being re-created even when it is not on
>>>> the declared list of search paths. I'm only using PJ_DEFAULT_CTX - is
>>>> that
>>>> the problem?
>>>
>>> The location of the user writable directory for cache.db is not controlled
>>> through proj_context_set_search_paths(). You can get the location with
>>> proj_context_get_user_writable_directory()
>>
>> Thanks for a rapid and full reply. My context is that in order to include
>> runnable code examples in R packages, they need to meet a requirement not
>> to cause data to be written outside the R temporary directory for the
>> running instance irrespective of platform. So I need to control the
>> address of the user-writeable directory completely in order to show users
>> what is going on, and to permit the package to meet test requirements. I
>> can try pj_context_set_user_writable_directory() understanding that it is
>> not part of the public API, and will report back.
>
> I'd say R shouldn't worry about what PROJ does under the hood. This is just caching. An
> internal detail that shouldn't have observable effect from pepole using the PROJ API. The
> purpose of this per-account directory was that the cache could be shared by many PROJ
> using software: GDAL, QGIS, etc.
>
> If you consider using pj_context_set_user_writable_directory() for more than testing, I'd
> suggest you to propose a pull request to move it to the public API as
> proj_context_set_user_writable_directory().

Trying to be non-invasive, I'm trying to use the available environment 
variables. When rgdal is loaded/attached to the running R environment, I 
used to copy out proj_info().searchpath; now, I'm setting 
PROJ_SKIP_READ_USER_WRITABLE_DIRECTORY and can (apparently) skip the user 
writable directory. From src/filemanager.cpp, line 1197 in master, at the 
top of proj_context_get_user_writable_directory(), I see the environment 
variable PROJ_USER_WRITABLE_DIRECTORY. So I try to use it:

> library(rgdal)
Loading required package: sp
rgdal: version: 1.5-13, (SVN revision 1024)
Geospatial Data Abstraction Library extensions to R successfully loaded
Loaded GDAL runtime: GDAL 3.1.1, released 2020/06/22
Path to GDAL shared files: /usr/local/share/gdal
GDAL binary built with GEOS: TRUE
Loaded PROJ runtime: Rel. 7.1.0, August 1st, 2020, [PJ_VERSION: 710]
Path to PROJ shared files: /usr/local/share/proj:/usr/local/share/proj
PROJ CDN enabled:FALSE
Linking to sp version:1.4-2
To mute warnings of possible GDAL/OSR exportToProj4() degradation,
use options("rgdal_show_exportToProj4_warnings"="none") before loading 
rgdal.
> get_proj_search_paths()
[1] "/usr/local/share/proj" "/usr/local/share/proj"
> td <- tempfile()
> dir.create(td)
> Sys.getenv("PROJ_USER_WRITABLE_DIRECTORY")
[1] ""
> Sys.setenv("PROJ_USER_WRITABLE_DIRECTORY"=td)
> Sys.getenv("PROJ_USER_WRITABLE_DIRECTORY")
[1] "/tmp/RtmpogCuDh/fileac60e50159ef4"
> .Call("get_proj_user_writable_dir", PACKAGE="rgdal")
[1] "/home/rsb/.local/share/proj"

But it seems that my get_proj_user_writable_dir(), calling:

proj_context_get_user_writable_directory(PJ_DEFAULT_CTX, FALSE)

reaches line 1194 in the function and finds that 
ctx->user_writable_directory.empty() is not TRUE, so does not read the 
environment variable.

What am I missing? Does ctx->user_writable_directory get populated anyway 
by the on-load call to proj_info().searchpath? Or is it rather the use of 
PJ_DEFAULT_CTX (seen elsewhere, for example the PROJ_NETWORK environment 
variable seems to be ignored in PJ_DEFAULT_CTX)? Using environment 
variables - and PROJ_SKIP_READ_USER_WRITABLE_DIRECTORY seems to work - is 
less invasive than widening the proj.h API.

The target is to give control over the directory used by CDN, partly for 
documenting what is going on (listing cache.db/chunks before and after), 
partly for testing on CRAN platforms which really dislike packages being 
teested writing outside the R session-specific temporary directory), and 
partly for larger labs wanting to be sure that every user is using the 
same grids (in this case, simply disabling CDN and downloading the 
required grids to a directory on the search path is preferable).

Roger

>
> Even
>
>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en