[Mapserver-users] Large Raster files

[Jan Hartmann]

Martin Weinelt wrote:

> 
> I am not Ed  ;-) but handling DEM-hillshades in a current project which can 
> also get quite big. What I am doing reads almost exactly like the procedure 
> decribed by Jan. I stay away from any on-the-fly projection and do as much 
> 'preprocessing' as can be done. I also change RGB to indexed colors, which 
> gives almost the same impression IMHO.

Yes, both preprocessing tasks (projecting and converting RGB to indexed 
colors) are almost mandatory for production servers

> 
> Everything is done by GDAL and the utilities. I decided to have tiles about 
> the size of the map, without knowing exactly why this would be better then 
> 2000px (Jan). 

Nothing holy about the number. It's just that multiplying and dividing 
by 2 is so easy with a computer :-). I chose 2000, because a single 
request should be as much confined to one tile, but it should not have 
to resample the image too much. Most map images are about 500 by 500 
pixels, so there is a good chance that it is completely within one tile.

> I also think that compressing the rasters is even contraproductive, because 
> 'in memory' there is no compression and you add an extra job (decoding) to 
> the application.

That's what I thought too, but serving compressed TIF files is not much 
faster than uncompressed ones, at least in my experience. Some of my 
rasters have a very high rate of compression (5 to 20 times), so I would 
run into disk problems with this procedure. Reason is (I guess) that 
regions can be extracted from compressed tifs without decompressing the 
whole file, so the actual time lost to decompressing the region is 
marginal.

> 
> I keeping the shapefiles in different resolutions aswell, making my map file a 
> pain (number of MAXSCALE-MINSCALE-layers for the same data). I will look 
> deeper into the new DATAPATTERN and variable substitution mechanisms soon.
> 

Additionally, I have been thinking about generalizing shapefiles to 
different levels with PostGIS. IMO generalizing is one of the 
fundamental problems of WebMapping.

Jan