[gdal-dev] VSI_CACHE, open files and memory growth

[Peter Schmitt]

On Fri, Oct 27, 2017 at 4:14 PM, Even Rouault <even.rouault at spatialys.com>
wrote:

> Peter,
>
>
>
> > I'm using MapServer-7.0.6 built with GDAL-2.2.2. Is VSI_CACHE_SIZE the
>
> > cache size per process? per file handle?
>
>
>
> Per-file handle :
>
> """
>
> The default size of caching for each file is 25 MB, and
>
> can be controlled with the VSI_CACHE_SIZE configuration option (value in
> bytes).
>
> Content in that cache is discarded when the file handle is closed.
>
> """
>
> ( http://gdal.org/gdal_virtual_file_systems.html#gdal_
> virtual_file_systems_vsicache )
>
>
>
> So with CLOSE_CONNECTION=DEFER, what you observed is logical.
>
> CLOSE_CONNECTION=DEFER could probably be improved with a least recently
> used cache to maintain
>
> the maximum number of simultaneously opened datasets to a reasonable
> amount.
>
>
>

Ah, thanks Even!  I should have read the docs more carefully!


> Alternatively with GDAL trunk, for /vsicurl/ and related file systems, you
> can also
>
> tune a global cache
>
> """
>
> In addition, a global least-recently-used cache of 16 MB shared among all
>
> downloaded content is enabled by default, and content in it may be reused
>
> after a file handle has been closed and reopen. Starting with GDAL 2.3, the
>
> size of this global LRU cache can be modified by setting the configuration
>
> option CPL_VSIL_CURL_CACHE_SIZE (in bytes).
>
> """
>
> ( http://gdal.org/gdal_virtual_file_systems.html#gdal_
> virtual_file_systems_vsicurl )
>
>
>
Nice, this will be helpful!

Thanks again,
Pete
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20171027/77c8290c/attachment.html>