[gdal-dev] VSI_CACHE, open files and memory growth

[Even Rouault]

Peter,

> I'm using MapServer-7.0.6 built with GDAL-2.2.2.  Is VSI_CACHE_SIZE the
> cache size per process? per file handle?

Per-file handle :
"""
The default size of caching for each file is 25 MB, and
can be controlled with the VSI_CACHE_SIZE configuration option (value in bytes).
Content in that cache is discarded when the file handle is closed.
"""
( http://gdal.org/gdal_virtual_file_systems.html#gdal_virtual_file_systems_vsicache )

So with CLOSE_CONNECTION=DEFER, what you observed is logical. 
CLOSE_CONNECTION=DEFER could probably be improved with a least recently used cache to maintain
the maximum number of simultaneously opened datasets to a reasonable amount.

Alternatively with GDAL trunk, for /vsicurl/ and related file systems, you can also
tune a global cache
"""
In addition, a global least-recently-used cache of 16 MB shared among all
downloaded content is enabled by default, and content in it may be reused
after a file handle has been closed and reopen. Starting with GDAL 2.3, the
size of this global LRU cache can be modified by setting the configuration
option CPL_VSIL_CURL_CACHE_SIZE (in bytes).
"""
( http://gdal.org/gdal_virtual_file_systems.html#gdal_virtual_file_systems_vsicurl )

Even


-- 
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20171028/860ef25b/attachment-0001.html>