Cluster/Supercomputer/HPC variants of Mapserver

[Biz King]

Whilst wholeheartedly agreeing that the code requires substantial  
optimisation, I think I have arrived at a reasonably logical process  
that allows me to create a limited parallelization of the serial  
process of drawing maps.
I intend to segment (tile?) the extents of the parameters being  
rendered by different machines into a series of proportional grids,  
send these parallel requests to cluster/Xgrid members, get the  
results back (probably as a stream) and then join the stream images  
back together into one coherent image and save that to the central  
file system which the web servers access.

We're actually having to produce live feedback of item locations  
(they can move, but can also remain static!) AND item activity (We're  
subject to NDA, and cannot give enough detail to explain clearly!  
Which is irritating.) which we then have to place into a graphical  
format (map with colour-coded dots indicating location, activity and  
status). Just to make things interesting, the extents of the data  
being displayed changes over time, so a series of pre-defined layer  
images just won't work as there are no 'standard' coordinate extents  
that we can work to.

The client wants as near real-time feedback as is possible, which  
compounds things even more!

I'll report back with more information as soon as I can.

cheers
Biz
On 19 Jan 2006, at 16:47, David Bitner wrote:

> Without getting into any clustering, there is probably a lot of
> optimization that you could do to your datasets.  There are a number
> of posts in the archives for this list and documentation on
> mapserver.gis.umn.edu on doing things like creating overviews at
> different resolutions and tiling for rasters that could likely help
> speed up the process.
>
>
> On 1/19/06, Biz King <biz.king at mac.com> wrote:
>> Hi All.
>>
>> Is anyone aware of anywhere (or better still, has experience of)
>> running Mapserver via an MPI/Grid interface or as a cluster?
>>
>> We're trying to develop a high-performance mapserver that can cope
>> with the load we're going to be throwing at it!  Currently it takes
>> 298 seconds (on a Mac OSX Server, 3.5 Gb Ram, dual 2Ghz processors)
>> to do what we need done on under 60 seconds!  There's not much we can
>> do to cut down the load as we're creating a whole series of nodes on
>> a layer via a database and we're then creating the imagery based on
>> these items and outputting them to graphics formats in varying sizes.
>>
>> The results get fed to users on demand without the delays associated
>> with 'on the fly' image creation.
>>
>> Any help will be welcomed!
>>
>> cheers
>>
>> Biz
>>