[Gdal-dev] raster format and indexation for not so static repository

[Ed McNierney]

Marco -

I'm not sure just what is taking too much time to process - the
conversion to TIFF, or the classification and display in MapServer?
Compressed formats are probably not a good idea, since they are likely
to take more time to generate and more time to decode.  Compression
software is designed to save space rather than save time.

Your overall approach sounds fine.  If your conversion takes too long,
tell us what tools you are using for the conversion.  It is possible
that you simply have too much conversion to do for the equipment you
have, and you'll need more hardware to do it.  But it is also possible
that there are faster ways of doing the conversion.  But if you're going
to convert 100,000 files in less than 24 hours, you will need to convert
about 1 file per second, and that's pretty fast.  Each file should be
about 1.5 megabytes, so they're small files, but that's still a lot of
conversion.

     - Ed

Ed McNierney
President and Chief Mapmaker
TopoZone.com / Maps a la carte, Inc.
73 Princeton Street, Suite 305
North Chelmsford, MA  01863
ed at topozone.com
(978) 251-4242  

-----Original Message-----
From: gdal-dev-bounces at lists.maptools.org
[mailto:gdal-dev-bounces at lists.maptools.org] On Behalf Of Marco Vieira
Sent: Friday, August 05, 2005 2:24 PM
To: Gdal-dev at lists.maptools.org
Subject: [Gdal-dev] raster format and indexation for not so static
repository

Hi list.
 Hi have 150Gb of raster data distributed in hundred thousand files in a
non standart 16bit raw format that I can convert to a Arc/Info ASCII
Grid
- AAIGrid format to display with mapserver in many scales. This data
need to be updated almost everyday (converted and presented).
 I planned to convert to a non compressed format (tif) in three diferent
resolution (a fake "pyramids") to classify pixel values dinamically with
mapserver and planned to use simple tile indexation like gdaltindex +
shptree but it is not showing me a good idea because needes to much time
to process and to much volume to handle.
 Anyone has some suggestion about raster format and indexation to
optimize handling and fast presentation? ECW?


Marco Vieira.
Fone:    + 55 21 9499-6800
marco_vieira at uk2.net





_______________________________________________
Gdal-dev mailing list
Gdal-dev at lists.maptools.org
http://lists.maptools.org/mailman/listinfo/gdal-dev