[gdal-dev] Writing data to "CreateCopy" files

[Frank Warmerdam]

Derek,

GDAL support multi-threading only in fairly restricted situations.
You particularly must not
be making calls on a single GDALDataset from mutliple threads at the
same time.  In fact,
due to a subtle issue with the way write caches are flushed you
shouldn't really do
multithreaded access to GDAL at all when writing to more than one file.

So I don't see how your suggested approach could work.



On Thu, Jul 21, 2011 at 3:27 PM, Cole, Derek <dcole at integrity-apps.com> wrote:
> I think I may have tried to go down this road before, but perhaps someone can elighten me.
>
> I am using CreateCopy on this file so that I get a copy of the file on disk with all the same metadata and projection info, etc.
>
> Is it possible for me to spawn CreateCopy in another thread which would presumably write the headers and such immediately. Then I could use my host thread or whatever to start writing my own data into the newly created file? It seems like it shows up on disk as the right size immediately when CreateCopy is called, so I assume all the delay is CreateCopy moving the raster data over (which I dont care about at this point, because I am overwriting it).
>
> Am I maybe just better off creating a new file from the get-go with the required size parameters? If I was going to do that what is the best route to go to ensure that I copy all of the important metadata and projection, coordinates, etc?

Yes, I think this is the best approach.  How careful you need to be
capturing everything depends a bit on how varied you believe your
datasets will be.  At a minimum you should be transferring the dataset
level coordinate system, geotransform and metadata.  For bands you
need to transfer the nodata value and color table .

Beyond that you start getting into rarer details.  GCPs, band level
metadata, geolocation metadata, RPC metadata, band level categories,
band level scaling info, etc.  Looking through the GDAL Data model
document will give you a sense of the things that are possible.  You
can look at gdal/apps/gdal_translate.c for an example (read from
VRTCreate() on.

Best regards,


>
> Thanks
>
> Derek
> ________________________________________
> From: fwarmerdam at gmail.com [fwarmerdam at gmail.com] on behalf of Frank Warmerdam [warmerdam at pobox.com]
> Sent: Friday, July 15, 2011 10:22 PM
> To: Cole, Derek
> Cc: gdal-dev at lists.osgeo.org
> Subject: Re: [gdal-dev] Writing data to "CreateCopy" files
>
> On Fri, Jul 15, 2011 at 3:57 PM, Cole, Derek <dcole at integrity-apps.com> wrote:
>> I have a couple of questions regarding this.
>>
>> Is the dataset that is returned from CreateCopy able to be used immediately?
>> I would like to use it to get the first band from it, and use RasterIO on
>> that band.
>
> Derek,
>
> Yes, you should be able to read the resulting file
> with the returned GDALDataset object immediately.
> Whether you can write to the resulting file will depend
> on the format.  Some formats are readonly (with
> CreateCopy being the only way to accomplishing
> writing).
>
>
>> Secondly - the data I am trying to write with RasterIO is unsigned char, but
>> when I was reading the data, I had to read it with a GDT_Float32, which was
>> allocated like so:
>>
>>     float * floatData = (float *)
>> CPLMalloc(sizeof(float)*bufSizeX*bufSizeY);
>>
>> and that seemed to be working well. Will my copy have to be written via a
>> GDT_Float32 data buffer like the one I read it into initially, or can I just
>> change the RasterIO parameter on the write to be GDT_Byte?
>
>
> You can read into a float buffer and write back out using
> an unsigned char buffer as long as you use the appropriate
> data type (GDT_Byte or GDT_Float32) in the RasterIO call.
> RasterIO wlil take care of on the fly conversion of the data.
>
> Be aware that writing a float buffer to a Byte file will not
> cause the file on disk to be upgraded to store floating
> point values.  Instead RasterIO will convert the floating
> point values to unsigned char.  When down converting
> types out of range values are clamped to the max and min
> of the target type and otherwise a floor() convertion is
> applied when going to integer types.
>
> Best regards,
> --
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Software Developer
>



-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Software Developer