[gdal-dev] gdal_merge GeoTIFF output twice correct size

[Nathan Vander Wilt]

On Mar 25, 2010, at 3:49 PM, Frank Warmerdam wrote:
> Nathan Vander Wilt wrote:
>> I used gdal_merge to combine four tiles into a perfectly square image.
>> However, the resulting file was double the size on disk than I would expect
>> based on its dimensions and bit depth. I opened it in a hex editor, and
>> there is a huge pattern of data that repeats 0x00003075 followed by a ton of
>> zeroes.
>> Why is gdal_merge writing all this extra data? Does it have something to do
>> with the "sparse" blocks discussed in this thread? http://lists.osgeo.org/pipermail/gdal-dev/2008-October/018750.html
>> I found how to perhaps compress these blocks with format options
>> (http://gdal.org/frmt_gtiff.html) and could try that, but that may lead to
>> compatibility issues. Why are these blocks in the file at all? Again, the
>> merged image is square and is completely covered by the source data.
> 
> NateVW,
> 
> There are a variety of reasons the file could be larger than expected.
> 
> 1) Is the file compressed?  If so, gdal_merge's rewriting of blocks
> can result in them being moved to the end of the file if the block is
> larger on the second write.
> 
> 2) Is the file tiled?  Depending on tile and image size partially used
> tiles on the right and bottom of an image can result in files being
> significantly larger than expected.
> 
> A "tiffdump" report on the output file might help deduce what is going on.



To test something else, I used gdal_retile.py -ps <half_original_width> <half_original_height> to chop a single source GeoTIFF into 4 pieces. I then simply used gdal_merge.py to recombine those four files into a new GeoTIFF. The result looks exactly the same in an image viewer, but the file itself is twice the size it should be. The extra zero cruft doesn't seem to be helping anyone but Seagate.

Here is the information tiffdump reports for the remerged file:

path/to/remerged.tif:
Magic: 0x4949 <little-endian> Version: 0x2a
Directory 0: offset 8 (0x8) next 0 (0)
ImageWidth (256) SHORT (3) 1<10000>
ImageLength (257) SHORT (3) 1<10000>
BitsPerSample (258) SHORT (3) 3<8 8 8>
Compression (259) SHORT (3) 1<1>
Photometric (262) SHORT (3) 1<2>
StripOffsets (273) LONG (4) 10000<281147892 281177892 281207892 281237892 281267892 281297892 281327892 281357892 281387892 281417892 281447892 281477892 281507892 281537892 281567892 281597892 281627892 281657892 281687892 281717892 281747892 281777892 281807892 281837892 ...>
SamplesPerPixel (277) SHORT (3) 1<3>
RowsPerStrip (278) SHORT (3) 1<1>
StripByteCounts (279) LONG (4) 10000<30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 ...>
PlanarConfig (284) SHORT (3) 1<1>
SampleFormat (339) SHORT (3) 3<1 1 1>
33550 (0x830e) DOUBLE (12) 3<0.15 0.15 0>
33922 (0x8482) DOUBLE (12) 6<0 0 0 553500 4.83e+06 0>
34735 (0x87af) SHORT (3) 32<1 1 0 7 1024 0 1 1 1025 0 1 1 1026 34737 22 0 2049 34737 7 22 2054 0 1 9102 ...>
34737 (0x87b1) ASCII (2) 30<WGS 84 / UTM zone 11N|WG ...>

The original file has the following info:

path/to/original.tif:
Magic: 0x4949 <little-endian> Version: 0x2a
Directory 0: offset 300000008 (0x11e1a308) next 0 (0)
ImageWidth (256) SHORT (3) 1<10000>
ImageLength (257) SHORT (3) 1<10000>
BitsPerSample (258) SHORT (3) 3<8 8 8>
Compression (259) SHORT (3) 1<1>
Photometric (262) SHORT (3) 1<2>
StripOffsets (273) LONG (4) 10000<8 30008 60008 90008 120008 150008 180008 210008 240008 270008 300008 330008 360008 390008 420008 450008 480008 510008 540008 570008 600008 630008 660008 690008 ...>
Orientation (274) SHORT (3) 1<1>
SamplesPerPixel (277) SHORT (3) 1<3>
RowsPerStrip (278) SHORT (3) 1<1>
StripByteCounts (279) LONG (4) 10000<30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 30000 ...>
PlanarConfig (284) SHORT (3) 1<1>
33550 (0x830e) DOUBLE (12) 3<0.15 0.15 1>
33922 (0x8482) DOUBLE (12) 6<0 0 0 553500 4.83e+06 0>
34735 (0x87af) SHORT (3) 28<1 1 0 6 1024 0 1 1 1025 0 1 1 2048 0 1 4326 2054 0 1 9102 3072 0 1 32611 ...>

thanks,
-natevw
http://calftrail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20100325/69eaf04c/attachment.html