[gdal-dev] Re: Strange things with gdalwarp ...

[Jukka Rahkonen]

Hermann Peifer <peifer <at> gmx.eu> writes:

> 
> Jukka wrote

> > 
> > Did you try to use gdalwarp also for the second step with -srcnodata value
[value...]?  I believe you did set
> some NODATA value in step 1 for the intermediate images.
> > 
> > -Jukka Rahkonen- 
> 
> I did. However, the main problem with gdalwarp is its performance. So my
options seem to be:
> 
> 1.
> gdalwarping 1500 ASTER tiles in 1 run into a 4GB 100m raster output.tif
> = 5-10 days processing (my best estimate)
> 
> 2.
> gdalwarping 1500 ASTER tiles 1-by-1 as an intermediate step, then
> gdal_merge or gdal_translate to mosaic warped tiles into output.tif
> = smaller or larger NODATA areas in output.tif, along tile borders

Hi, 

I decided to do a little test in order to see how slow gdalwarp in in this kind
of usage with my own imagery. My test plan was like this:

1. Original images: 199 JPEG2000 images, with 3 bands and 10000 by 10000 pixels
each. All together make 28 gigabytes on disk in lossless JPEG2000 format, or 60
GB uncompressed.

2. My first try with gdalbuildvtr
gdalbuildvrt -input_file_list gdalwarptest.txt gdalwarptest.vrt
0...10...20...30...40...50...60...70...80...90...100 - done.
Nice experience, I like this little tool.

3. A try to make a tiff file from this virtual raster with gdalwarp
gdalwarp -co BIGTIFF=YES -srcnodata 0 gdalwarptest.vrt gdalwarptest.tif
Creating output file that is 182026P x 191204L.


Gdalwarp has now been running for 27 hours. Process is taking 70 megabytes of
memory and 5-15 percent of processor time. As a result I have now output tiff
file that is very, very slowly growing in size. Now, after 27-hour run the
filesize is 13 gigabytes. Judged by the filesize my new tiff file might be ready
after 4-5 more days.  However, I am not sure if this estimate is reliable. The
gdalwarp status indicator is still showing just zero for me:
Processing input file gdalwarptest.vrt.
0

Conclusion: Too slow to be useful with my settings.
Question 1: Should it work this way?
Question 2: If yes, is it worth trying to find the limits when it goes too slow
and perhaps file a ticket?

-Jukka Rahkonen-