[gdal-dev] Allocate large memory space with CPLMalloc(). Bug or not?

[Kyle Shannon]

On Thu, Oct 1, 2015 at 1:18 PM, fazotron <alexzhdanov1003 at gmail.com> wrote:
>
> Hello!
>
> Take a look on the following code:
>
> *...
> size_t w = 10000, h = 5000, d = 150;
> size_t sz = w * h * d * sizeof(float); // ~27.94 Gb
>
> float *volume = (float*)CPLMalloc(sz);
> ...*
>
> In this case GDAL throws error in runtime: "ERROR 1: CPLMalloc(-664771072):
> Silly size requested."
> But I think that it's not silly, because such amount of free RAM is
> available on my system. For example I can successfully allocate it calling
> standart malloc().
>
> I have opened CPLMalloc() source code and seen the following lines:
>
> *void *CPLMalloc( size_t nSize )
> ...
>     if( long(nSize) < 0 )
>     {
>         CPLError( CE_Failure, CPLE_AppDefined,
>                   "CPLMalloc(%ld): Silly size requested.\n",
>                   (long) nSize );
>         return NULL;
>     }
> ...*
>
> In my case long( size_t nSize ) returns negative value and it leads to
> error.
> As far as I understand converting one integer type to another smaller
> integer type (e.g. long long/size_t to long) is implementation-defined, so
> it's not safe to use CPLMalloc() with 64-bit integers.
>

According to the documentation, CPLMalloc is a convenience for
'smaller' allocations where the caller is 'unwilling to test for out
of memory conditions':

/**
 * Safe version of malloc().
 *
 * This function is like the C library malloc(), but raises a CE_Fatal
 * error with CPLError() if it fails to allocate the desired memory.  It
 * should be used for small memory allocations that are unlikely to fail
 * and for which the application is unwilling to test for out of memory
 * conditions.  It uses VSIMalloc() to get the memory, so any hooking of
 * VSIMalloc() will apply to CPLMalloc() as well.  CPLFree() or VSIFree()
 * can be used free memory allocated by CPLMalloc().
 *
 * @param nSize size (in bytes) of memory block to allocate.
 * @return pointer to newly allocated memory, only NULL if nSize is zero.
 */


Use the VSIMalloc*() family, it is just a wrapper for malloc, but
checks for overflow in VSIMalloc2(), VSIMalloc3().

>
>
> --
> View this message in context: http://osgeo-org.1560.x6.nabble.com/Allocate-large-memory-space-with-CPLMalloc-Bug-or-not-tp5227443.html
> Sent from the GDAL - Dev mailing list archive at Nabble.com.
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev




-- 
Kyle