[OSGeo-Discuss] RE: OGC WPS and Amazon SQS

[Randy George]

I noticed OGC finalized the WPS spec:
http://www.opengeospatial.org/pressroom/pressreleases/843

 

Does anyone know of projects working on WPS implementations?

 

The goal of WPS is apparently to provide a consistent framework for
interchangeable service process algorithms that can potentially be chained
together into answers to higher level questions than the typical 'what',
'when', and 'where.' Dealing with 'why', 'how much', and 'what if' modeling
usually requires a process pipeline for convolutions, boolean band
operations, and summary pixel calculations, all of which are cpu cycle
intense, especially for large imagery sets. In fact cpu usage issues would
make the usual service approach prohibitive.  Even the little I have worked
on JAI pipelines shows me the futility of a one cpu to many service requests
approach for WPS.

 

However, looking at the AWS Simple Queue Service, SQS
http://www.amazon.com/Simple-Queue-Service-home-page/b/ref=sc_fe_l_2?ie=UTF8
<http://www.amazon.com/Simple-Queue-Service-home-page/b/ref=sc_fe_l_2?ie=UTF
8&node=13584001&no=3435361&me=A36L942TSJ2AJA>
&node=13584001&no=3435361&me=A36L942TSJ2AJA, some interesting possibilities
come to mind.

Locking message queues with AMI instance pools is essentially a poor man's
supercomputer. It would be interesting to look at harnessing the utility
computing concept with instance pools available for each stage in a process
pipeline connected using the asynchronous SQS service. This is a more or
less controlled 'distributed computing model' applied to WPS. 

Ref here for some examples of existing distributed computing projects:
http://distributedcomputing.info/projects.html 

 

Here are a couple possible approaches to a WPS service model that might
overcome the cpu bottle neck:

1)      Sequential SQS pipeline with dedicated instance for each process
node - this would work best for operations amenable to a streaming pipeline
- Boolean band operations or pixel summary operations for instance 

 

2)      Distributed computing model with a chunk server feeding a pipeline
and an array pool of instances processing the chunks coming down the SQS
queue - this would be better suited to tiled operations 

 

WPS is great when someone else provides the service. I imagine it would be
very interesting to the academic scientific world and government groups
tasked with providing access to all the myriad imagery coming off space
sensor platforms. 

 

Just thinking out loud.   More thoughts here:
http://www.cadmaps.com/gisblog/?p=28

 

randy

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/discuss/attachments/20080227/2b655474/attachment-0002.html>