[gdal-dev] Re: [FWTools] BigTiff problem in experimental FWTools3

[Petteri Packalen]

Chaitanya, Frank,

I verified that only one libgdal exists in a system. I have also added 
'-co' switch to gdal_fillnodata.py in order to force BigTiff creation 
(BIGTIFF=YES). But the problem persists.

However, I think I have traced the problem down to:

   http://svn.osgeo.org/gdal/trunk/gdal/alg/rasterfill.cpp

In lines 458 and 474 temporary GTiff files are created. Also creation 
option COMPRESS=LZW is used (not easy to guess whether the file size will 
be over 2^32?). In my case the file osValTmpFile increases until its size 
goes over 2^32 when the error is emitted by TIFF library. Thus this 
temporary file was not created as a BigTiff file.

Can you verify that I'm on the right track?

-- 
Best regards,
Petteri Packalen

**********************************************************************
Petteri Packalen
Senior Researcher, Forest Mensuration Science
University of Eastern Finland
Faculty of Science and Forestry,
School of Forest Sciences
P.O. Box 111
80101 Joensuu
E-mail: Petteri.Packalen at uef.fi
Phone: +358 13 251 4093
Room: Borealis 376
**********************************************************************


On Tue, 17 May 2011, Frank Warmerdam wrote:

> On 11-05-17 06:10 AM, Petteri Packalen wrote:
>> Hello,
>>
>> I'm facing a problem related to BigTiff support in experimental FWTools3 on
>> Linux. BigTiff support works fine with all compiled programs (e.g.
>> gdal_translate, gdalwarp) but not with the python script gdal_fillnodata.py.
>> For instance, executing:
>>
>> gdal_fillnodata.py -md 10 unfilled.tif filled.tif
>>
>> is terminated by the message:
>>
>> ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded
>> ERROR 1: An error occured while writing a dirty block
>> ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded
>> ...
>>
>> I have tested with versions 3.0.2 and 3.0.6 under 32 and 64 bit Ubuntus.
>>
>> Any hint what is going wrong?
>
> Petteri,
>
> Sorry for the delay in looking into this issue.
>
> I have tested with FWTools 3.0.6 and I do not encounter any problems with
> the python scripts and support for big tiff files.  In particular I
> created a 4.8GB TIFF file with gdal_translate and then I translated it into
> a new file with gdalimport.py without any problem.
>
> I looked into libtiff at the message you are seeing and I believe it only
> occurs in libtiff4, so you definately have the BigTIFF enabled TIFF library
> being used at runtime.  But should the file be created as a regular TIFF
> instead of a BigTIFF and yet it somehow runs over the 4GB mark you will get
> this message.
>
> I suspect that this is what is happening, though it is not immediately clear
> to me how that would happen in the above situation.  Unfortunately
> gdal_fillnodata.py does not provide -co options to pass extra creation options
> to the create function or else you could ensure that the output file is created
> as a bigtiff (using -co BIGTIFF=YES).
>
> What size is your input file (unfilled.tif)?
>
> I wonder if you could provide steps by which I could reproduce the problem
> without having to slug multi GB files around?
>
> Best regards,
> -- 
> ---------------------------------------+--------------------------------------
> I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
> light and sound - activate the windows | http://pobox.com/~warmerdam
> and watch the world go round - Rush    | Geospatial Programmer for Rent
>
> _______________________________________________
> FWTools mailing list
> FWTools at lists.maptools.org
> http://lists.maptools.org/mailman/listinfo/fwtools
> http://fwtools.maptools.org/
>