[postgis-devel] [raster] Memory management and IO concerns

[Bryce L Nordgren]

On Thu, Jun 23, 2011 at 10:35 PM, Paul Ramsey <pramsey at opengeo.org> wrote:

> Once you DETOAST_DATUM, it's loaded in. However you do have the option
> of just doing DETOAST_DATUM_SLICE. Obviously, this begins to
> complicate things :) however where performance is paramount, it's
> worth doing. The index code detoasts just the first slice of a
> gserialized that contains the bounding box, for example.
>
>
So looking a bit closer at the current code, if I understand it correctly:

1] DETOASTing an argument to get a rt_pgraster loads the header information
+ all data in all bands.
2] deserializing into an rt_raster copies the header information, and
optionally copies the metadata for each band into a new rt_band. However,
the actual band metadata is never copied out of the detoasted buffer; the
band's "data" pointer is just set such that it points into deToast.
3] Getting a handle to the driver with GDALDatasetH also doesn't copy data.

Clearly, the data are saved because they are returned, but if the data
weren't returned would the DETOASTed buffer be commited back to the backend?
(e.g., pass by reference?)

Ok, so my memory concerns are somewhat allieviated. I agree that fiddling
with rt_raster_deserialize (e.g., to "load" only specific bands) can't
improve things, as it's really only setting up access to something already
loaded.

Is there a way to encapsulate "DETOAST" logic such that only the raster
header, a specific band header, and the corresponding band data are loaded?
I'd hate to have to write that logic in more than one function. And the
correlary, can you DETOAST different slices as they're needed or do you have
to load everything upfront? (e.g., load image blocks?)

Bryce
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-devel/attachments/20110623/ee0080e8/attachment.html>