[Mapserver-users] Sensor Web Enablement (SWE), YSI and MapServer
Yves Moisan
ymoisan at groupesm.com
Tue Jul 20 10:48:38 EDT 2004
This is a multi-part message in MIME format.
------=_NextPart_000_01C2_01C46E47.1D053370
Content-Type: text/plain;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
Hi All,
One of the objectives I had attending the recent OGIS conference, =
besides getting to know you all, was to understand how to =
store/retrieve/process in situ sensor data, in my particular case water =
quality data coming from YSI 6600 sondes, and deliver it in a map over =
the web. I wanted to be able to draw the envelope of the "data service =
tier" vs any client app, because I want to build my own client without =
bundling the data tier into it. Data must remain accessible to other =
parties and of course MapServer/Server is where the data tier ends. At =
the time, I figured MapServer could spew out maps with little dots =
showing sensor locations, people click on the dots, get the data in =
tabular format. Standard stuff. I hadn't thought of sensor data =
*standard storage* per se, though.
I am currently developing a small Python utility to gobble up YSI 6600 =
sonde data. For now, I want to be able to simulate sonde data using a =
"data generator" that would write such binary files for the sake of =
having a system pseudo-working. We are not getting a sonde any time =
soos, but I want to get going. When I get a sensor working, then I want =
to use such a utility to QA sensor data and potentially to view it in =
environments such as SciPy. I don't want to use YSI's Ecowatch, at =
least not in a production environment. The data format is binary and as =
of yet I have no clue as to how to decypher it, so I am playing around =
with the file. BTW, anyone knows of a "binary file explorer" type of =
thing that would try and find patterns in arbitrary binary files and =
suggest data extraction schemes, e.g. firts 4 bytes are chars, next 600 =
are floats ... ?
Anyhow, the jist of this message is about the SWE initiative -- which I =
have just found out about today -- and its relation to MapServer. I was =
hoping to store sensor data in a sort of staging environment. At the =
first level, the original .DAT files. At the second level, the data =
would be stored in more pallatable formats for HTML/SVG output and it =
would have been QA'd. For that last level, I am not sure I would want =
to serialize the data in XML (but then, maybe yes!) before stuffing it =
in a DB. Since I am using Python, I thought maybe of pickling it to a =
flat file. That would allow me to write a QA module (checking for NaN's =
etc.) in Python and that would also let me use Python XML serialization =
modules to pipe it through (using mod_python ?) before sending out to =
the Web. That's the flatfile/ObjectDB option. Another would be to use =
the flat file for QA, then spew the results out in tabular format into =
PostGIS. That'a probably closest to what would be amenable to MapSever. =
What is the status of SWE in the context of MapServer ? I gather SWE =
it is not a spec yet, but is it coming strong?
The point is simple : I have sensor data that I want to show up on a map =
and above all store it so it can be "OGC Serviced" when SWE clients =
start to appear. I'll probably have to write a SWE client. If I do, I =
want to do it with Python (thanx Sean for that point on your slide, but =
I was already convinced!) and I'll want an adapter for Zope and maybe =
Twisted (also my problem). I just want to do it in the most "OGC =
compliant", Pythonic way.
Any pointers ideas out there ?
Thanx
Yves Moisan
------=_NextPart_000_01C2_01C46E47.1D053370
Content-Type: text/html;
charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<HTML><HEAD>
<META http-equiv=3DContent-Type content=3D"text/html; =
charset=3Diso-8859-1">
<META content=3D"MSHTML 6.00.2800.1400" name=3DGENERATOR>
<STYLE></STYLE>
</HEAD>
<BODY bgColor=3D#ffffff>
<DIV><FONT face=3DArial size=3D2>
<DIV><FONT face=3DArial size=3D2>Hi All,</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>One of the objectives I had attending =
the recent=20
OGIS conference, besides getting to know you all, was to understand how =
to=20
store/retrieve/process in situ sensor data, in my particular case water =
quality=20
data coming from YSI 6600 sondes, and deliver it in a map over the =
web. I=20
wanted to be able to draw the envelope of the "data service tier" vs any =
client=20
app, because I want to build my own client without bundling the data =
tier into=20
it. Data must remain accessible to other parties and of course=20
MapServer/Server is where the data tier ends. At the time, I =
figured=20
MapServer could spew out maps with little dots showing sensor locations, =
people=20
click on the dots, get the data in tabular format. Standard =
stuff. I=20
hadn't thought of sensor data *standard storage* per se, =
though.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>I am currently developing a small =
Python utility to=20
gobble up YSI 6600 sonde data. For now, I want to be able to =
simulate=20
sonde data using a "data generator" that would write such binary files =
for the=20
sake of having a system pseudo-working. We are not getting a sonde =
any=20
time soos, but I want to get going. When I get a sensor working, =
then I=20
want to use such a utility to QA sensor data and potentially to view it =
in=20
environments such as SciPy. I don't want to use YSI's Ecowatch, at =
least=20
not in a production environment. The data format is binary and as =
of yet I=20
have no clue as to how to decypher it, so I am playing around with the=20
file. BTW, anyone knows of a "binary file explorer" type of thing =
that=20
would try and find patterns in arbitrary binary files and suggest data=20
extraction schemes, e.g. firts 4 bytes are chars, next 600 are floats =
...=20
?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Anyhow, the jist of this message is =
about the SWE=20
initiative -- which I have just found out about today -- and its =
relation to=20
MapServer. I was hoping to store sensor data in a sort of staging=20
environment. At the first level, the original .DAT files. At =
the=20
second level, the data would be stored in more pallatable formats for =
HTML/SVG=20
output and it would have been QA'd. For that last level, I am not =
sure I=20
would want to serialize the data in XML (but then, maybe yes!) before =
stuffing=20
it in a DB. Since I am using Python, I thought maybe of pickling =
it to a=20
flat file. That would allow me to write a QA module (checking for=20
NaN's etc.) in Python and that would also let me use Python XML=20
serialization modules to pipe it through (using mod_python ?) before =
sending out=20
to the Web. That's the flatfile/ObjectDB option. Another =
would be to=20
use the flat file for QA, then spew the results out in tabular format =
into=20
PostGIS. That'a probably closest to what would be amenable to=20
MapSever. What is the status of SWE in the context of MapServer =
? I=20
gather SWE it is not a spec yet, but is it coming strong?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>The point is simple : I have sensor =
data that I=20
want to show up on a map and above all store it so it can be "OGC =
Serviced" when=20
SWE clients start to appear. I'll probably have to write a SWE=20
client. If I do, I want to do it with Python (thanx Sean for that =
point on=20
your slide, but I was already convinced!) and I'll want an adapter for =
Zope and=20
maybe Twisted (also my problem). I just want to do it in the most =
"OGC=20
compliant", Pythonic way.</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Any pointers ideas out there =
?</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Thanx</FONT></DIV>
<DIV><FONT face=3DArial size=3D2></FONT> </DIV>
<DIV><FONT face=3DArial size=3D2>Yves =
Moisan</FONT></DIV></FONT></DIV></BODY></HTML>
------=_NextPart_000_01C2_01C46E47.1D053370--
More information about the mapserver-users
mailing list