[gdal-dev] Can GDAL open binary buffer contents as input for MEM format?

[Roger André]

Hi Frank,

Ok, I agree with you that the MapServer doc says that the getBytes()
function "Returns the image contents as a binary buffer".  And I see in
tiff_read.py that tiff_vsimem() function is supposed to "# Test reading a
tiff from a memory buffer (#2931)", but there must be a difference between
the binary buffer that MS returns, and the memory buffer that tiff_vsimem is
reading as input.

I implemented your recommendation in the code below, and I am getting an
error when I try to open the buffer as it is done in the tiff_read code,
"TypeError: fileio() argument 1 must be encoded string without NULL bytes,
not str"

#! /usr/bin/python

from mapscript import mapObj
from osgeo import gdal

def getImageData():
  mapobject = mapObj('basemap.map')
  mapimage = mapobject.draw()
  image_data = mapimage.getBytes()
  return image_data

def cropImage(image_data):
  open(image_data, mode='rb').read()

def main():
  image_data = getImageData()
  cropImage(image_data)


if __name__ == "__main__":
    main()

I'm trying to figure out how to manipulate the getBytes() buffer in such a
way that "open" will work on it, but so far am not having any luck.

Roger
--



On Sat, Jan 16, 2010 at 5:57 PM, Frank Warmerdam <warmerdam at pobox.com>wrote:

> Roger André wrote:
>
>> Hi Frank,
>>
>> Ok, that makes sense.  So for this to work, I will have to know (somehow)
>> what type of image mapserver is sending, and then instantiate a MEM dataset
>> with the correct parameters.  Problem with that is that I can't think of a
>> good way to read what those parameters are out of the binary string.  It's
>> no problem when I know exactly what data is coming in, as is the case for my
>> very specific need, but it means that swapping in GDAL for PIL as a general
>> solution might not be possible.
>>
>
> Roger,
>
> I was confused about what you were using before, and now I realize you
> are calling imageObj.getBytes() in mapscript (as you did explain!).
>
> Reading the docs for this function:
>    /*
>
>  -------------------------------------------------------------------------
>    getBytes returns a gdBuffer structure (defined in mapscript.i) which
> must
>    be typemapped to an object appropriate to the target language.  This
>    typemap must also gdFree the data member of the gdBuffer.  See the type-
>    maps in java/javamodule.i and python/pymodule.i for examples.
>
>    contributed by Jerry Pisk, jerry.pisk at gmail.com
>
>  -------------------------------------------------------------------------
>    */
>
> it seems it may be difficult to do alot with the result.  Hmm, in fact
> looking at the code:
>
>    gdBuffer getBytes()
>    {
>        gdBuffer buffer;
>
>        buffer.owns_data = MS_TRUE;
>
>        buffer.data = msSaveImageBuffer(self, &buffer.size, self->format);
>
>        if( buffer.data == NULL || buffer.size == 0 )
>        {
>            buffer.data = NULL;
>            msSetError(MS_MISCERR, "Failed to get image buffer",
> "getBytes");
>            return buffer;
>        }
>
>        return buffer;
>    }
>
> it isn't doing what is suggested by the comments.  It is actually encoding
> it into whatever the format selected was (perhaps jpeg or png) and
> returning
> the raw bytes of that.
>
> My approach before depended on the image being a raw image (uncompressed,
> without any format specific headers).  What you need is a way of opening
> a supported file format, but from a memory buffer.
>
> This sequence from autotest/gcore/tiff_read.py gives a clue how this might
> be done in the GDAL Python bindings:
>
>    content = open('data/byte.tif', mode='rb').read()
>
>    # Create in-memory file
>    gdal.FileFromMemBuffer('/vsimem/tiffinmem', content)
>
>    ds = gdal.Open('/vsimem/tiffinmem', gdal.GA_Update)
>
> You won't need to worry about copying palettes or anything.
>
>
>  One last question. How does a utility like gdal_translate figure out what
>> the input data type is?
>>
>
> gdal.Open() reads a small chunk from the beginning of the file, and
> then, in sequence, passes this to the Open method of each driver along
> with the filename and some other info.  Each Open() has to decide if the
> referenced file is in it's format by whatever mechanism is appropriate.
> Usually this is done by looking for "magic" bytes for that format at
> the beginning of the file.
>
>
> Best regards,
> --
>
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam<http://pobox.com/%7Ewarmerdam>
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20100118/53a00d97/attachment.html