[gdal-dev] Can GDAL open binary buffer contents as input for MEM format?

[Frank Warmerdam]

Roger André wrote:
> Hi Frank,
> 
> Ok, that makes sense.  So for this to work, I will have to know 
> (somehow) what type of image mapserver is sending, and then instantiate 
> a MEM dataset with the correct parameters.  Problem with that is that I 
> can't think of a good way to read what those parameters are out of the 
> binary string.  It's no problem when I know exactly what data is coming 
> in, as is the case for my very specific need, but it means that swapping 
> in GDAL for PIL as a general solution might not be possible.

Roger,

I was confused about what you were using before, and now I realize you
are calling imageObj.getBytes() in mapscript (as you did explain!).

Reading the docs for this function:
     /*
     -------------------------------------------------------------------------
     getBytes returns a gdBuffer structure (defined in mapscript.i) which must
     be typemapped to an object appropriate to the target language.  This
     typemap must also gdFree the data member of the gdBuffer.  See the type-
     maps in java/javamodule.i and python/pymodule.i for examples.

     contributed by Jerry Pisk, jerry.pisk at gmail.com
     -------------------------------------------------------------------------
     */

it seems it may be difficult to do alot with the result.  Hmm, in fact
looking at the code:

     gdBuffer getBytes()
     {
         gdBuffer buffer;

         buffer.owns_data = MS_TRUE;

         buffer.data = msSaveImageBuffer(self, &buffer.size, self->format);

         if( buffer.data == NULL || buffer.size == 0 )
         {
             buffer.data = NULL;
             msSetError(MS_MISCERR, "Failed to get image buffer", "getBytes");
             return buffer;
         }

         return buffer;
     }

it isn't doing what is suggested by the comments.  It is actually encoding
it into whatever the format selected was (perhaps jpeg or png) and returning
the raw bytes of that.

My approach before depended on the image being a raw image (uncompressed,
without any format specific headers).  What you need is a way of opening
a supported file format, but from a memory buffer.

This sequence from autotest/gcore/tiff_read.py gives a clue how this might
be done in the GDAL Python bindings:

     content = open('data/byte.tif', mode='rb').read()

     # Create in-memory file
     gdal.FileFromMemBuffer('/vsimem/tiffinmem', content)

     ds = gdal.Open('/vsimem/tiffinmem', gdal.GA_Update)

You won't need to worry about copying palettes or anything.

> One last question. How does a utility like gdal_translate figure out 
> what the input data type is?

gdal.Open() reads a small chunk from the beginning of the file, and
then, in sequence, passes this to the Open method of each driver along
with the filename and some other info.  Each Open() has to decide if the
referenced file is in it's format by whatever mechanism is appropriate.
Usually this is done by looking for "magic" bytes for that format at
the beginning of the file.

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent