[Gdal-dev] gdal_translate error with big file

[William Kyngesburye]

I ran into this problem with reading large TIFFs generated in  
Photoshop.  Photoshop writes a TIFF with a single strip.  Yet the  
strip size within libtiff uses a *signed* 32 bit integer, tsize_t, I  
think because it needs to return some error condition.  So I can't  
read >2GB TIFFs created in Photoshop.

A lot of other functions within libtiff use signed 32 bit integers,  
so it can still run into >2GB problems, even if the "largefile"  
support is built into it.  It would need a major rewrite to fix this.


Although, the error I see from Shoaib's output tells me it's having  
trouble READing past 2GB from the asc source.  ~30% of 6GB ~= 2GB :  
Cant seek to offset -1 in input file to read.  gdalinfo will 'read'  
it since it's just looking at the header, nowhere near past 2GB.

On Nov 28, 2005, at 11:02 PM, Aaron Racicot wrote:

> Not 100% sure, but I believe the max file size for a Tiff/GeoTiff  
> is 4GB due
> to 32 bit offsets.  Your error would seem to confirm that.
>
> Aaron
>
> -----Original Message-----
> From: gdal-dev-bounces at lists.maptools.org
> [mailto:gdal-dev-bounces at lists.maptools.org] On Behalf Of Shoaib  
> Burq (VPAC)
> Sent: Monday, November 28, 2005 8:30 PM
> To: gdal-dev at lists.maptools.org
> Subject: [Gdal-dev] gdal_translate error with big file
>
> Hi
>
> Trying a coversion from ArcInfo ascii to GeoTiff with internal tiling.
> This is to understand internal tiling better.
>
> The file size is 6.1GB
>
> Currently running FWTools on winXP:
> $ gdalinfo --version
> GDAL 1.3.1.0, FWTools 1.0.0a6, released 2005/11/03
>
> Upon running the gdal_translate to convert the ascii grids to  
> GeoTiff we
> encountered the following error:
>
> $ gdal_translate -co "TFW=YES" -co "TILED=YES" -of GTiff  
> SeafloorElevation
> .asc SeafloorElevation_tiled.tif
> Input file size is 34799, 29478
> 0...10...20...30.ERROR 3: Cant seek to offset -1 in input file to read
> data.
> ERROR 1: IReadBlock failed at X offset 0, Y offset 9789
> ERROR 1: GetBlockRef failed at X block offset 0, Y block offset 9789
>
> Works fine using ArcGISs ArcToolBox ASCIItoGRID utility.
>
> Gdalinfo is happy to read the ascii file too:
> C:\data>gdalinfo SeafloorElevation.asc
> Driver: AAIGrid/Arc/Info ASCII Grid
> Size is 34799, 29478
> Coordinate System is `'
> Origin = (246331.250000,5749620.750000)
> Pixel Size = (0.50000000,-0.50000000)
> Corner Coordinates:
> Upper Left  (  246331.250, 5749620.750)
> Lower Left  (  246331.250, 5734881.750)
> Upper Right (  263730.750, 5749620.750)
> Lower Right (  263730.750, 5734881.750)
> Center      (  255031.000, 5742251.250)
> Band 1 Block=34799x1 Type=Int16, ColorInterp=Undefined
>    NoData Value=-9999
>
> Is this a "big file" support issue? if so how do I get around it?
>
> Thanks for the help
> chrs
> Shoaib Burq

-----
William Kyngesburye <kyngchaos at kyngchaos.com>
http://www.kyngchaos.com/

First Pogril: Why is life like sticking your head in a bucket filled  
with hyena offal?
Second Pogril: I don't know.  Why IS life like sticking your head in  
a bucket filled with hyena offal?
First Pogril: I don't know either.  Wretched, isn't it?

-HitchHiker's Guide to the Galaxy