Serving raster images via MapServer?

[Paul Ramsey]

Wow, Armin's solution (using PostGIS as the tileindex) is well nigh  
perfect.  Have an image metadata table with nice information like  
capture date, sensor, whatnot, and the path reference to the raw  
data...  then using the arbitrary SQL capabilities of Mapserver/ 
PostGIS set up to do the "latest-and-greatest" query.  For a PoC you  
could just do a latest-limit-1 query.  For a real one, you'll  
probably have to do some pl/pgsql in order to reduce the return set  
to just the minimal latest overlapping set.  Could be v. v. slick.

P.

On Mar 30, 2006, at 7:47 AM, Thom DeCarlo wrote:

> Hey Ed,
> (So, you and Norm are over here, too. Small world!)
>
> This sounds good. I've been reading up on the gdaltindex and I think I
> understand how to get it set up. But, what is this shptree thing you
> mentioned?
> Also, two other issues. First, the client is going to be continually
> requesting map updates and the server will be getting frequent  
> additions to
> its repository. If the tile index is held in a shapefile will the  
> MapServer
> re-read that file when a new client request comes in or is it only  
> read when
> Mapserver starts?
> Second, since I've already got a PostgreSQL server managing the  
> vector data,
> can the tile index be held in it, too? I'm going to have a lot of
> overlapping imagery and it seems (to the novice that I am) that SQL  
> queries
> could find the best/newest data to serve out to the client. Or, is  
> this just
> unnecessary overhead?
>
> Thanks!
> Thom
>
> --
> Thom DeCarlo
> --------------------------------------------------------
>       Any sufficiently advanced technology
>     is indistinguishable from a rigged demo.
>
>
>> -----Original Message-----
>> From: Ed McNierney
>> Sent: Wednesday, March 29, 2006 3:36 PM
>> To: MAPSERVER-USERS at LISTS.UMN.EDU
>> Subject: Re: [UMN_MAPSERVER-USERS] Serving raster images via  
>> MapServer?
>>
>> Thom -
>>
>> There are a few general principles for working with large raster data
>> sets in MapServer.  Some of these are generalizations that apply  
>> to many
>> similar applications and aren't necessarily MapServer-specific.
>>
>> If your imagery is updated with any frequency, you'll want to  
>> simplify
>> things and avoid lots of complex fiddling with the images.  For most
>> users, it's good to keep the data on disks organized in a manner  
>> that's
>> consistent with the way you receive the data and updates.
>>
>> For performance reasons, GeoTIFF is almost always the preferred  
>> format.
>> Compression generally costs you in performance, and particularly  
>> costs
>> you in scalability; some decompressors run quickly but require  
>> lots of
>> RAM.  That's fine for a small number of users, but if you scale up  
>> you
>> may suffer as you can run out of physical RAM quickly.  Disk space is
>> cheap - I never compress imagery.
>>
>> If you can structure your GeoTIFFs to be internally tiled (using
>> gdal_translate) with overviews (via gdaladdo) you can get GeoTIFFs  
>> that
>> can be read quickly at a variety of scale levels.
>>
>> Organize your GeoTIFFs using TILEINDEX shapefiles (via gdaltindex)  
>> and
>> index those shapefiles with shptree.
>>
>> Reprojection of raster data is faster than many folks think, but it's
>> much slower than reprojecting vector data.  Where possible, store  
>> your
>> imagery in the same projection as you serve it.
>>
>> Keeping things simple is good - it's really not that complicated.   
>> I'm
>> currently providing WMS service (via MapServer) to just over 40
>> terabytes of imagery in around 500,000 image files, using roughly the
>> sort of system described above.
>>
>> 	- Ed
>>
>> Ed McNierney
>> President and Chief Mapmaker
>> TopoZone.com / Maps a la carte, Inc.
>> 73 Princeton Street, Suite 305
>> North Chelmsford, MA  01863
>> ed at topozone.com
>> (978) 251-4242
>>
>> -----Original Message-----
>> From: Thom DeCarlo
>> Sent: Wednesday, March 29, 2006 3:06 PM
>> To: MAPSERVER-USERS at LISTS.UMN.EDU
>> Subject: [UMN_MAPSERVER-USERS] Serving raster images via MapServer?
>>
>> Hi,
>> I've got a fairly large archive of raster images that we would  
>> like to
>> deliver to remote clients through WMS. I've still got much to learn
>> about the technology. I thought I should ask the first obvious  
>> question
>> here.
>>
>> Is there an accepted "best way" to organize and deliver imagery using
>> MapServer? I have a PostgreSQL/PostGIS server set up to handle our
>> vector data, but there doesn't seem to be support for raster data in
>> that database.
>>
>> Thanks,
>> Thom
>> --
>> Thom DeCarlo
>> --------------------------------------
>>      That Which Does Not Kill Me
>>      Can Still Hurt Really Bad.