[gdal-dev] gdalwarp with packbits compression: output too large

[jr.morreale at enoreth.net]

Le 2012-09-13 16:28, Rutger a écrit :
> Dear list,
>
> Im using the gdalwarp command-line utility on Windows and im running 
> into
> some strange behavior. When reprojecting a large global grid to a 
> subset,
> the output filesize is way too large. Its a lot larger than
> xsize*ysize*datatype, and when pulling the file through 
> gdal_translate
> shrinks the filesize to normal proportions. This occurs only when i 
> run
> gdalwarp with "COMPRESSION=PACKBITS", with no compression the results 
> from
> gdalwarp seem fine.
>
> The GDAL version im using is: GDAL 1.9.1, released 2012/05/16
>
> I can reproduce the problem with the following example data:
> The GMTED elevation dataset from the USGS, which can be found at:
> http://topotools.cr.usgs.gov/GMTED_viewer/gmted2010_global_grids.php
>
> The "Mean Statistic, 30 arc-seconds" is about 270MB. The effect is 
> much more
> prominent with the 7.5 arc second file, which is about 3.1 GB! The 
> data is
> delivered as an Arc/Info Binary Grid.
>
> I have tested it with several projections and different regions, for 
> example
> the United States, reprojecting with gdalwarp like this:
>
> gdalwarp -ot Float32 -co "COMPRESS=PACKBITS" -r bilinear -t_srs 
> "+proj=lcc
> +lat_1=20 +lat_2=60 +lat_0=40 +lon_0=-96 +x_0=0 +y_0=0 +ellps=GRS80
> +datum=NAD83 +units=m +no_defs" -ts 10400 5800 -te -2600000 -1600000 
> 2600000
> 1300000 hdr.adf GMTED.tif
>
> After gdalwarp the output is 401 MB, for a float32, i would expect an 
> upper
> limit of about 10400*5800*4 = 230MB. When running gdal_translate with 
> no
> compression and no other settings whatsoever the file size reduces to 
> 235MB,
> which seems ok. With packbits compression it reduces further to 
> 154MB.
>
> So for the 30 arc second dataset, the initial gdalwarp result is 
> about 2.5
> times to large. When using the higher resolution 7.5 arc second input 
> (but
> otherwise the same procedure), the file sizes are:
> after gdalwarp with packbits: 626MB
> after gdal_translate: 235MB
> after gdal_translate with packbits: 154MB
>
> Making the gdalwarp result 4 times 'too large'. The effect seems 
> larger with
> the higher resolution input data, even with exactly the same output
> settings. I have also tried to first convert the Arc/Info binary grid 
> to a
> Geotiff and then starting the entire procedure, this produces the 
> same
> results.
>
> Has anyone else ever noticed this behavior? And is there something i 
> can do
> to prevent it?
>
> Thanks in advance.
>
>
> Regards,
> Rutger
>

Add -wo OPTIMIZE_SIZE=TRUE

the process will be longer but the file will be similar to the output 
of gdaltranslate