[gdal-dev] Bigtiff question

[Even Rouault]

Le Thursday 05 March 2009 20:45:49 Lucena, Ivan, vous avez écrit :
> Even,
>
> >  for the very poor performance when dealing with pixel interleaved GTiffs
> > with a large number of bands, I think you've hit ticket #2838 that has
> > been fixed 3 weeks ago in trunk and branches/1.6 (*). The performance
> > issue was about *reading* in such files, but sometimes when you write and
> > the block cache size is not big enough, you also end up reading back
> > partially written scanlines. Hopefully this would correct the problem you
> > observe.
>
> Ah!
>
> >  But as Frank suggested, I would not recommand using pixel interleaved
> > GTiffs with a large number of bands anyway. This is fine for RGB or RGBA
> > datasets, but for more bands, it could cause cache trashing problems as
> > we prefill the blocks for all bands in pixel-interleaved mode (#2838 was
> > about the fact this technic wasn't correctly implemented)
>
> What raster format would you suggest then?
>
> I mean, based on those requirements:
>
> - Multiband;
> - Large files;
> - Good performance *reading* the data in pixel space. Not the "band as
> usual" ;)

Have you tried a tiled (128x128) band-interleaved GeoTIFF instead ? And read 
block per block. In any event, you should be careful than the block cache 
size is at least blockXSize * blockYSize * sizeof(datatype) * nbands.

>
> >  #!/usr/bin/python
> >  import gdal
> >
> >  driver_tif = gdal.GetDriverByName("GTIFF")
> >  output_dst = driver_tif.Create( 'test.tif', 20, 20, 320, gdal.GDT_Byte,
> >  [ 'TILED=NO', 'INTERLEAVE=PIXEL' ])
> >  for i in range(320):
> >      output_band = output_dst.GetRasterBand( i + 1 )
> >      output_band.Fill(i + 1)
> >      output_band.FlushCache()
> >  output_dst = None
>
> My real data is 5000x2500x312 Int16, can you try with:
>
> --
>   output_dst = driver_tif.Create( 'test.tif', 5000, 2500, 312,
> gdal.GDT_Int16, [ 'TILED=NO', 'INTERLEAVE=PIXEL' ])
> --

This is too large for my modest PC. I've tried up to 10 bands 5000x2500 and it 
worked fine.

>
> My best regards,
>
> Ivan