[gdal-dev] Changing default value for GDAL_CACHEMAX ? (was Re: Slow warping)

[Even Rouault]

Le mardi 01 décembre 2015 15:54:12, Ari Jolma a écrit :
> 28.11.2015, 17:13, Even Rouault kirjoitti:
> > Le vendredi 27 novembre 2015 13:43:34, Ari Jolma a écrit :
> >> 27.11.2015, 14:10, Even Rouault kirjoitti:
> >>> Try setting GDAL_CACHEMAX to 150 (6048 * 4032 * 3 = 73 MB. and a x 2
> >>> security margin)
> >> 
> >> This reduces the time into 4 s.
> >> 
> >> Quite a bit reduction from several hours.
> >> 
> >> Maybe the default is too low? 150 MB does not seem much.
> > 
> > Setting the default is not obvious because it depends on use cases and
> > hardware configurations. What if people run 100 gdal_translate in
> > parallel ? (not very common use cases admidetely, and people can still
> > override the default)
> > 
> > Perhaps rather than a fixed value, a percentage of the total RAM would be
> > more appropriate (*). Let's say 5% ?
> > 2 GB (which is the max for a 32 bit process) -> 100 MB
> > 4 GB -> 200 MB
> > 16 GB -> 800 MB.
> > 
> > We could also accept a GDAL_CACHEMAX=X% syntax
> 
> I that possible? The amount of memory available to a program seems a
> fuzzy topic.

/** Return the total physical RAM, usable by a process, in bytes.
 *
 * This is the same as CPLGetPhysicalRAM() except it will limit to 2 GB
 * for 32 bit processes.
 *
 * Note: This memory may already be partly used by other processes.
 *
 * @return the total physical RAM, usable by a process, in bytes (or 0 in case 
of failure).
 * @since GDAL 2.0
 */
GIntBig  CPLGetUsablePhysicalRAM(void)


> 
> Ari
> 
> > Even
> > 
> > (*) the ECW SDK does that for its own purposes, with a 25% default AFAIR

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com