[gdal-dev] GDAL memory management

[Jesús]

Thanks for your answer, Even.

I would want that, yes, but I was prepared for the answer being no :(.

I also though computational complexity would be the problem, but I checked
with VTune and it points the other way (malloc, free, realloc). Moreover,
performance is worse each time I execute the same test in the same run, so
I think that would match some memory fragmentation or something like that.

Thanks again,

Jesús Cano

El jue., 30 abr. 2020 a las 11:24, Even Rouault (<even.rouault at spatialys.com>)
escribió:

> Jesús,
>
>
>
> >
>
> > I have been working with C/C++ and GDAL with GEOS for some time now and I
>
> > have a question about GDAL memory management.
>
> >
>
> > I wonder if there are any memory management options or tips to control
>
> > allocations and deallocations in the library since we are noticing
>
> > performance problems as our data increases in complexity, mainly OGR
>
> > operations (intersection, simplify).
>
> >
>
> > For example, I would expect something to do the same as:
>
> > - std::allocator: http://www.cplusplus.com/reference/memory/allocator/
>
> > - CGAL allocator:
>
> > https://doc.cgal.org/latest/Manual/devman_memory_management.html
>
> > - SQLite memory allocation routines:
>
> > https://www.sqlite.org/c3ref/mem_methods.html
>
> >
>
> > I have been looking for it, but GDAL is so big that maybe I missed
>
> > something.
>
>
>
> Interaction between GDAL and GEOS is in ogr/ogrgeometry.cpp and
> ogr/ogrgeometryfactory.cpp
>
>
>
> Basically GDAL will export geometries to WKB and GEOS will instanciate its
> geometry object from it. The temporary WKB buffer created by GDAL is using
> CPLMalloc() that is just a wrapper over malloc().
>
> GEOS will likely use standard C++ memory operators (new, vector) etc.
>
> And ultimately GDAL will reconstruct a OGR Geometry from a GEOS WKB
> buffer, using a mix of C++ new and CPLMalloc
>
>
>
> From the links you refering, it seems you'd want a way to use your own
> memory management routines. There's no such possibility in GDAL.
> CPLMalloc()/CPLRealloc()/CPLFree() could possibly be redirected to some
> callbacks. On the C++ side, not sure what would be involved.
>
>
>
> Are you really sure that the bottleneck you see are in memory
> allocations/deallocations themselves ? I'd expect more computational
> complexity to be the main limit.
>
>
>
> Even
>
>
>
> --
>
> Spatialys - Geospatial professional services
>
> http://www.spatialys.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20200430/22cea0f8/attachment.html>