[gdal-dev] gdalwarp with packbits compression: output too large

[Rutger]

Dear list,

Im using the gdalwarp command-line utility on Windows and im running into
some strange behavior. When reprojecting a large global grid to a subset,
the output filesize is way too large. Its a lot larger than
xsize*ysize*datatype, and when pulling the file through gdal_translate
shrinks the filesize to normal proportions. This occurs only when i run
gdalwarp with "COMPRESSION=PACKBITS", with no compression the results from
gdalwarp seem fine. 

The GDAL version im using is: GDAL 1.9.1, released 2012/05/16

I can reproduce the problem with the following example data:
The GMTED elevation dataset from the USGS, which can be found at:
http://topotools.cr.usgs.gov/GMTED_viewer/gmted2010_global_grids.php

The "Mean Statistic, 30 arc-seconds" is about 270MB. The effect is much more
prominent with the 7.5 arc second file, which is about 3.1 GB! The data is
delivered as an Arc/Info Binary Grid. 

I have tested it with several projections and different regions, for example
the United States, reprojecting with gdalwarp like this:

gdalwarp -ot Float32 -co "COMPRESS=PACKBITS" -r bilinear -t_srs "+proj=lcc
+lat_1=20 +lat_2=60 +lat_0=40 +lon_0=-96 +x_0=0 +y_0=0 +ellps=GRS80
+datum=NAD83 +units=m +no_defs" -ts 10400 5800 -te -2600000 -1600000 2600000
1300000 hdr.adf GMTED.tif

After gdalwarp the output is 401 MB, for a float32, i would expect an upper
limit of about 10400*5800*4 = 230MB. When running gdal_translate with no
compression and no other settings whatsoever the file size reduces to 235MB,
which seems ok. With packbits compression it reduces further to 154MB. 

So for the 30 arc second dataset, the initial gdalwarp result is about 2.5
times to large. When using the higher resolution 7.5 arc second input (but
otherwise the same procedure), the file sizes are:
after gdalwarp with packbits: 626MB
after gdal_translate: 235MB
after gdal_translate with packbits: 154MB

Making the gdalwarp result 4 times 'too large'. The effect seems larger with
the higher resolution input data, even with exactly the same output
settings. I have also tried to first convert the Arc/Info binary grid to a
Geotiff and then starting the entire procedure, this produces the same
results. 

Has anyone else ever noticed this behavior? And is there something i can do
to prevent it?

Thanks in advance.


Regards,
Rutger


 




--
View this message in context: http://osgeo-org.1560.n6.nabble.com/gdalwarp-with-packbits-compression-output-too-large-tp5001731.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.