[gdal-dev] Re: [FWTools] BigTiff problem in experimental FWTools3

[Even Rouault]

Selon Petteri Packalen <packalen at cs.joensuu.fi>:

Yes, that's a very likely reason. When COMPRESS != NONE is specified, the driver
has no way to know if the file will be a regular tiff or a bigtiff, so it
chooses regular tiff. For your case, if you are in position to build GDAL by
yourself, you can try changing the following line in rasterfill.cpp

static const char *apszOptions[] = { "COMPRESS=LZW", NULL };

into

static const char *apszOptions[] = { "COMPRESS=LZW", "BIGTIFF=YES", NULL };



> Chaitanya, Frank,
>
> I verified that only one libgdal exists in a system. I have also added
> '-co' switch to gdal_fillnodata.py in order to force BigTiff creation
> (BIGTIFF=YES). But the problem persists.
>
> However, I think I have traced the problem down to:
>
>    http://svn.osgeo.org/gdal/trunk/gdal/alg/rasterfill.cpp
>
> In lines 458 and 474 temporary GTiff files are created. Also creation
> option COMPRESS=LZW is used (not easy to guess whether the file size will
> be over 2^32?). In my case the file osValTmpFile increases until its size
> goes over 2^32 when the error is emitted by TIFF library. Thus this
> temporary file was not created as a BigTiff file.
>
> Can you verify that I'm on the right track?
>
> --
> Best regards,
> Petteri Packalen
>
> **********************************************************************
> Petteri Packalen
> Senior Researcher, Forest Mensuration Science
> University of Eastern Finland
> Faculty of Science and Forestry,
> School of Forest Sciences
> P.O. Box 111
> 80101 Joensuu
> E-mail: Petteri.Packalen at uef.fi
> Phone: +358 13 251 4093
> Room: Borealis 376
> **********************************************************************
>
>
> On Tue, 17 May 2011, Frank Warmerdam wrote:
>
> > On 11-05-17 06:10 AM, Petteri Packalen wrote:
> >> Hello,
> >>
> >> I'm facing a problem related to BigTiff support in experimental FWTools3
> on
> >> Linux. BigTiff support works fine with all compiled programs (e.g.
> >> gdal_translate, gdalwarp) but not with the python script
> gdal_fillnodata.py.
> >> For instance, executing:
> >>
> >> gdal_fillnodata.py -md 10 unfilled.tif filled.tif
> >>
> >> is terminated by the message:
> >>
> >> ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded
> >> ERROR 1: An error occured while writing a dirty block
> >> ERROR 1: TIFFAppendToStrip:Maximum TIFF file size exceeded
> >> ...
> >>
> >> I have tested with versions 3.0.2 and 3.0.6 under 32 and 64 bit Ubuntus.
> >>
> >> Any hint what is going wrong?
> >
> > Petteri,
> >
> > Sorry for the delay in looking into this issue.
> >
> > I have tested with FWTools 3.0.6 and I do not encounter any problems with
> > the python scripts and support for big tiff files.  In particular I
> > created a 4.8GB TIFF file with gdal_translate and then I translated it into
> > a new file with gdalimport.py without any problem.
> >
> > I looked into libtiff at the message you are seeing and I believe it only
> > occurs in libtiff4, so you definately have the BigTIFF enabled TIFF library
> > being used at runtime.  But should the file be created as a regular TIFF
> > instead of a BigTIFF and yet it somehow runs over the 4GB mark you will get
> > this message.
> >
> > I suspect that this is what is happening, though it is not immediately
> clear
> > to me how that would happen in the above situation.  Unfortunately
> > gdal_fillnodata.py does not provide -co options to pass extra creation
> options
> > to the create function or else you could ensure that the output file is
> created
> > as a bigtiff (using -co BIGTIFF=YES).
> >
> > What size is your input file (unfilled.tif)?
> >
> > I wonder if you could provide steps by which I could reproduce the problem
> > without having to slug multi GB files around?
> >
> > Best regards,
> > --
> >
> ---------------------------------------+--------------------------------------
> > I set the clouds in motion - turn up   | Frank Warmerdam,
> warmerdam at pobox.com
> > light and sound - activate the windows | http://pobox.com/~warmerdam
> > and watch the world go round - Rush    | Geospatial Programmer for Rent
> >
> > _______________________________________________
> > FWTools mailing list
> > FWTools at lists.maptools.org
> > http://lists.maptools.org/mailman/listinfo/fwtools
> > http://fwtools.maptools.org/
> >
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>