[gdal-dev] Creating modified copies of a file

[Cole, Derek]

Well, I am about to give up on this I think and go a different route for creating a NITF from this data.

I tried a small test file, and it takes about 45s to CopyCrate() an 80mb NITF. 

here is the output I got after turning on CPL_DEBUG

GDAL: GDALOpen(/home/dcole/eraser/Images/dan_data.ntf, this=0x1ad7ec70) succeeds as NITF.
GDAL: QuietDelete(test1.ntf) invoking Delete()
GDAL: GDALOpen(test1.ntf, this=0x1ad81180) succeeds as NITF.
GDAL: GDALDefaultOverviews::OverviewScan()
GDAL: GDALClose(test1.ntf, this=0x1ad81180)
Warning 6: NITF only supports WGS84 geographic and UTM projections.

0GDAL: Potential thrashing on band 1 of /home/dcole/eraser/Images/dan_data.ntf.
...10...20...30...40...50...60...70...80...90...100 - done.
ERROR 6: Apparently no space reserved for IGEOLO info in NITF file.
NITFWriteIGEOGLO() fails.
ERROR 6: NITF only supports WGS84 geographic and UTM projections.

GDAL: GDALDefaultOverviews::OverviewScan()
ERROR 6: NITF only supports WGS84 geographic and UTM projections.


So to try to remedy this I changed the cachemax (programmatically) anywhere from 40MB to 3GB trying to see if any setting had any effect. It was about the same no matter what. 

I tried doing a gdal_translate on this file, which only took about a second. Looking through the source for that, it seems like I am doing approximately the same thing in relation to the CreateCopy(), so I am not sure what the problem is. However, my viewer which uses GDAL to read the original NITF file I am reading in will NOT read the translated NITF file that is generated. 

Any idea why the gdal_translate might be running faster than my CreateCopy, even though the parameters are just about identical?

Derek
________________________________________
From: Even Rouault [even.rouault at mines-paris.org]
Sent: Wednesday, July 06, 2011 5:32 PM
To: gdal-dev at lists.osgeo.org
Cc: Cole, Derek
Subject: Re: [gdal-dev] Creating modified copies of a file

Le mercredi 06 juillet 2011 23:18:21, Cole, Derek a écrit :
> I have noticed that when attempting to do this, I am able to do a gdalinfo
> on the source and destination files (even if the destination file is not
> complete, it seems the header gets created, and gdalinfo recognizes teh
> file).
>
> Is it possible that this is taking so long because my destination file has
> no geocoords or projection information? It seems like createcopy is not
> replicating all of the variables in the original and I am having to
> manually set some things, such as Block size, ABPP, etc etc.

This is expected. When doing CreateCopy() the target driver has no idea what
the source driver is, so he will not recognize ABPP. As far as block sizes are
concerned, it is up to the user to decide if he wants to keep or change the
block dimension of the source dataset.

I'm a bit surprised that CreateCopy() is *that* much slower than cp, but it is
hard to know why without a hard work of profiling. It might be related to pixel
or band interleaving, etc... There might cache trashing. You could perhaps run
with CPL_DEBUG=ON and see if interesting info comes up.

> And the
> coordinates and projection information is blank in the new file. Is there
> perhaps some calculations being done to correct that issue, and thats why
> the copy is taking forever?

No, not related at all.

> I thought based on the documentation
> CreateCopy() would clone the file's raster, as well as size, type,
> projection, geotransform, etc, but all of this is not showing up in the
> partially created file at least.

You cannot reliably trust gdalinfo on a file still being written. Depending on
the driver some information might just be written at the end of the process. A
quick look at the CreateCopy() implementation of the NITF driver suggests that
this is indeed the case since the geotransform is set after the imagery.