[PROJ] specifying the search paths

[Roger Bivand]

Short version: environment variable PROJ_USER_WRITABLE_DIRECTORY will only 
override proj_context_get_user_writable_directory() if set before PROJ is 
invoked, example in:

https://rgdal.r-forge.r-project.org/articles/CRS_projections_transformations.html#transformation-grid-files-1

More at foot.

On Thu, 2 Jul 2020, Even Rouault wrote:

> On jeudi 2 juillet 2020 10:46:30 CEST Roger Bivand wrote:
>> On Wed, 1 Jul 2020, Even Rouault wrote:
>>> On mercredi 1 juillet 2020 17:33:05 CEST Roger Bivand wrote:
>>>> Even,
>>>>
>>>> On Wed, 1 Jul 2020, Even Rouault wrote:
>>>>> Roger,
>>>>>
>>>>>> I have a question about the use of
>>>>>> proj_context_set_search_paths(PJ_DEFAULT_CTX, ...). On initiating PRØJ,
>>>>>> I
>>>>>> see from proj_info().searchpath that (now 7.1.0) I have three paths,
>>>>>> the
>>>>>> last two identical /usr/local/share/proj, and the first
>>>>>> $HOME/.local/share/proj - which contains a cache.db.
>>>>>>
>>>>>> For testing on other platforms than my own, I need to change the first
>>>>>> path to a temporary directory with write access, so that I do not write
>>>>>> in
>>>>>> the user space of the test platform. So I use
>>>>>> proj_context_set_search_paths(PJ_DEFAULT_CTX, ...), providing the
>>>>>> correct
>>>>>> length of the vector of paths, and the paths themselves. The
>>>>>> user-writeable path is first, as at startup. It, however, does not
>>>>>> contain
>>>>>> a cache.db file, even of zero length. After finding a coordinate
>>>>>> operation
>>>>>> (using a grid cached in $HOME/.local/share/proj), the temporary
>>>>>> directory
>>>>>> remains empty, $HOME/.local/share/proj is used despite not being on the
>>>>>> active path list, and the operation completes successfully.
>>>>>>
>>>>>> This is, however, not what I need - I need the download to occur to the
>>>>>> temporary directory, so that I can demonstrate that cache.db gets
>>>>>> populated.
>>>>>>
>>>>>> May libproj be caching paths so that even if the path to the
>>>>>> use-writeable
>>>>>> directory has been set, it still uses the one determined on load, even
>>>>>> if
>>>>>> it had been renamed and is not on the this of search paths? I find the
>>>>>> default use-writeable directory being re-created even when it is not on
>>>>>> the declared list of search paths. I'm only using PJ_DEFAULT_CTX - is
>>>>>> that
>>>>>> the problem?
>>>>>
>>>>> The location of the user writable directory for cache.db is not
>>>>> controlled
>>>>> through proj_context_set_search_paths(). You can get the location with
>>>>> proj_context_get_user_writable_directory()
>>>>
>>>> Thanks for a rapid and full reply. My context is that in order to include
>>>> runnable code examples in R packages, they need to meet a requirement not
>>>> to cause data to be written outside the R temporary directory for the
>>>> running instance irrespective of platform. So I need to control the
>>>> address of the user-writeable directory completely in order to show users
>>>> what is going on, and to permit the package to meet test requirements. I
>>>> can try pj_context_set_user_writable_directory() understanding that it is
>>>> not part of the public API, and will report back.
>>>
>>> I'd say R shouldn't worry about what PROJ does under the hood. This is
>>> just caching. An internal detail that shouldn't have observable effect
>>> from pepole using the PROJ API. The purpose of this per-account directory
>>> was that the cache could be shared by many PROJ using software: GDAL,
>>> QGIS, etc.
>>>
>>> If you consider using pj_context_set_user_writable_directory() for more
>>> than testing, I'd suggest you to propose a pull request to move it to the
>>> public API as proj_context_set_user_writable_directory().
>>
>> Trying to be non-invasive, I'm trying to use the available environment
>> variables. When rgdal is loaded/attached to the running R environment, I
>> used to copy out proj_info().searchpath; now, I'm setting
>> PROJ_SKIP_READ_USER_WRITABLE_DIRECTORY and can (apparently) skip the user
>> writable directory. From src/filemanager.cpp, line 1197 in master, at the
>> top of proj_context_get_user_writable_directory(), I see the environment
>>
>> variable PROJ_USER_WRITABLE_DIRECTORY. So I try to use it:
>>> library(rgdal)
>>
>> Loading required package: sp
>> rgdal: version: 1.5-13, (SVN revision 1024)
>> Geospatial Data Abstraction Library extensions to R successfully loaded
>> Loaded GDAL runtime: GDAL 3.1.1, released 2020/06/22
>> Path to GDAL shared files: /usr/local/share/gdal
>> GDAL binary built with GEOS: TRUE
>> Loaded PROJ runtime: Rel. 7.1.0, August 1st, 2020, [PJ_VERSION: 710]
>> Path to PROJ shared files: /usr/local/share/proj:/usr/local/share/proj
>> PROJ CDN enabled:FALSE
>> Linking to sp version:1.4-2
>> To mute warnings of possible GDAL/OSR exportToProj4() degradation,
>> use options("rgdal_show_exportToProj4_warnings"="none") before loading
>> rgdal.
>>
>>> get_proj_search_paths()
>>
>> [1] "/usr/local/share/proj" "/usr/local/share/proj"
>>
>>> td <- tempfile()
>>> dir.create(td)
>>> Sys.getenv("PROJ_USER_WRITABLE_DIRECTORY")
>>
>> [1] ""
>>
>>> Sys.setenv("PROJ_USER_WRITABLE_DIRECTORY"=td)
>>> Sys.getenv("PROJ_USER_WRITABLE_DIRECTORY")
>>
>> [1] "/tmp/RtmpogCuDh/fileac60e50159ef4"
>>
>>> .Call("get_proj_user_writable_dir", PACKAGE="rgdal")
>>
>> [1] "/home/rsb/.local/share/proj"
>>
>> But it seems that my get_proj_user_writable_dir(), calling:
>>
>> proj_context_get_user_writable_directory(PJ_DEFAULT_CTX, FALSE)
>>
>> reaches line 1194 in the function and finds that

(the posted version got abbreviated here, I've added back the remainder 
for clarity).

>>
>> ctx->user_writable_directory.empty() is not TRUE, so does not read the
>>
>> environment variable.
>>
>> What am I missing? Does ctx->user_writable_directory get populated 
>> anyway by the on-load call to proj_info().searchpath?
>
> Yes. You should set the env var before any other call to PROJ to be on 
> the safe side.

That was it! Thanks, example now online at:

https://rgdal.r-forge.r-project.org/articles/CRS_projections_transformations.html#transformation-grid-files-1

The document gets checked, run, and rebuilt on local test/check platforms, 
so will vary from mine because most are at PROJ 6, but are looking at 7 
(hence the issue I raised for Brian Ripley and you resolved some days 
ago).

Thanks again,

Roger

>

-- 
Roger Bivand
Department of Economics, Norwegian School of Economics,
Helleveien 30, N-5045 Bergen, Norway.
voice: +47 55 95 93 55; e-mail: Roger.Bivand at nhh.no
https://orcid.org/0000-0003-2392-6140
https://scholar.google.no/citations?user=AWeghB0AAAAJ&hl=en