[OSGeo-Discuss] Mobile GPS data collection, summary of sorts.
Just van den Broecke
just at justobjects.nl
Thu Aug 3 03:27:22 PDT 2006
Bob Basques wrote:
>
>
> Just van den Broecke wrote:
>
>> (Back from holiday...just went through the MGPSDC threads...wew..)
>>
>> What I can offer is the GeoTracing (www.geotracing.com) platform, not
>> as much not as the solution but as a way to try out, share ideas,
>> design issues and experience. If the FOSS4G submission is accepted I
>> hope to present and/or demo in Lausanne (and/or have interesting
>> discussions with you).
>
> I actually looked at this when you posted (just before you went on
> Vacation??).
yes, I thought to expand some more and give out the (hidden) URLs to try
out things
>
>>
>> I plan to move the code to geotracing.codehaus.org somewhere this
>> month (time!). Documentation (time!) is lacking, but several working
>> applications have been delivered that do personalized GPS tracking
>> from a mobile phone through a server finally into a (MySQL
>> non-spatial) DB.
>
> Geez, somebody else that think like me. Non-Spatial database, oh no!! :c)
the data/queries were simple (see your comment below)
>
> So it this set up as a standalone operation entirely on the phone (as an
> optional mode of operation) or is the data sent directly to a Mainserver
> for Storage?
>
> MYSQL work great for simple datasets, even simple to moderately complex
> Spatial databases, as long as the data are indexed approriately.
ok, still need to figure out if one can set indexes on REAL (lat/lon)
>
>> Application-requirements were the main driver for the project. The
>> underlying Media/CMS platform KeyWorx (www.keyworx.org) that e.g.
>> describes protocol concepts is reasonably documented:
>> http://kwx.dev.waag.org/docs
>>
>> If you have a smartphone with bluetooth (JSR-82) plus bt gps you may
>> try out the MobiTracer client by signing up and downloading OTA:
>> http://www.geotracing.com/gt/my
>> The related TraceLand webapp runs at http://www.geotracing.com/gt
>> For MacOSX a _very_ experimental browser-AJAX-based client is available:
>> http://www.geotracing.com/gt/webtracer and
>> http://www.geotracing.com/gt/gpsdoor (small CLI Python program similar
>> to gpsd but HTTP-service-based)
>
> Is there a TREO 650/Socket (BT) GPS package?
no(t yet)
>
>>
>> Other:
>> - client/server comms uses lightweight XML over HTTP or TCP/IP
>> - hence client technology is independent of server
>
> As it should be :c)
>
>> - authorization: users logon before any other action like submitting data
>
> How is this mnanaged on the server side, just an on/off proposiition, or
> can things be controlled at a particular layer of information?
(not entirely clear what you mean) the server manages
users/roles/permissions; users and data items are first class objects
(records), protocol requests are intercepted for authorization
>
>> - personalization: all submitted data (GPS, media, POIs) is linked to
>> user (in DB).
>> - basically client does remote track recording through the XML
>> protocol (create-track, write-track, suspend/resume (creates
>> segments), etc)
>
> Very similar to how we set up out Listener some years back.
>
>> - media files (photo/video) can be uploaded (HTTP) or emailed
>> - geotagging media is attempted by matching EXIF timestamp data with
>> GPS points or else submission time
>> - raw GPS (NMEA) samples are sent to server, who parses, checks for
>> quality and spikes (GPS data can be "filthy") before appending to track
>
> A little different from our version, we did the cleaning at the server
> the same way, but all points were time indexed and IDed seperately and
> simply recorded in the database, The AVL hardware seems to have been a
> bit more stable than GPS receivers I think.
>
>> - client does some GPS-prefiltering: taking best sample in an
>> configurable interval
>
> We grabbed everything, got a point every 1.5 seconds or so by polling
> the remote capture device (IP polling), this was plenty of data to build
> a track for following vehicles.
track-data is file-oriented with meta-data in DB records (a KeyWorx
feature to manage large data blobs like media files and appendable
tracks with transactional integrity), since 99% of the use is about
managing individual tracks not points in tracks.
>
>> - time is a nasty issue: though a GPS has exact time, some phones do
>> not provide correct UTC, photos do not provide timezone info in EXIF etc
>> - no local storage/later syncing yet (a much wanted option since
>> connectivity may not be present in e.g. rural areas)
>
> Almost a must if you need to do any sort of trail analysis. Make the
> dispatching system more effective for example.
yes,
>
>> - for desktop and possibly/hopefully some PDA-type clients
>> (http://weblogs.mozillazine.org/dougt/archives/016719.html) I see
>> browser-AJAX-based solutions as a way to go over dedicated apps. You
>> will then only need small components to handle GPS and possibly local
>> storage.
>> - Opera Mobile on Symbian is not yet there, Minimo for CE is hopeful
>> with AJAX support.
>> - data organisation: server stores tracks per user divided into
>> segments (like GPX)
>
> Can you explain this a bit further, segment of time, Distance, other ???
I think you hit an interesting point: how to manage/organize/store GPS
data. The GeoTracing approach is to store GPS-data as Tracks. Each Track
is related to a User. Any media/POIs are both related to the User and
the Track (KeyWorx supports a generic N-M DB relationship).
A Track (in GT) is not a contiguous number of points but divided in
Segments. At least two reasons: (1) statistics: e.g. sports events where
you e.g. skate a trail but pause inbetween (suspend/resume) (2) within a
single Track for various reasons (crashes/network unavailability, or
simply not having switched the system on) one may resume at some
distance further along. When rendering the Track you don't want to draw
a straight line between two too distant points (but rather have a hole).
Segments can be created explicitly with the track-suspend/resume service
or implicitly when one restarts the client-app using the same track.
There is always an active track. Creating a new Track archives the
previous. (and we may add a third reason to implicitly create segments
based on minimal distance between consecutive points or even minimal
movement).
It depends on the intended application, but wherever one needs to
collect GPS-data and treat the data as polylines you have to consider
these issues and consider using segments. I think this is like most
handheld GPS-es manage track data ? At least I got the idea from the
GPX-standard.
At least when you start implementing GPS-data collection especially in a
constrained (in connectivity, data cost) network environment like GPRS
there's among others these issues to consider:
1) filter out faulty GPS data (xDOP, spikes, drifts). Strangely some
Sirf III GPS-es (like the Holux GPSlim 236) appear to work indoor/under
foliage but give large spikes.
2) cope with GPS availability
3) cope with network availability (ok need local store/later sync)
4) cope Bluetooth connection errors
5) bandwidth/data storage optimization: e.g.
a) only send/store a point if it has some minimal distance from its
previous point (e.g. if a user stays at a single point)
b) e.g. driving a straight line, no need to send/store intermediate points
6) also store raw GPS-NMEA samples (tracklogs), not just extracted
lon/lat. You may regret that later (e.g. if your cleanup algoritm
appeared incorrect or you forgot to store e.g. elevation or quality)
7) don't treat a Track as a single polyline but use segments (e.g. N
polylines). For example DB each record could have lat/lon/time/ele etc
plus a trackid + segmentid.
8) Consider criteria for creating tracks (explicit by user or by day)
and segments (suspend/resume, minimal distance etc).
9) use configurable parameters for the above GPS-filtering (e.g. min
xDOP), minimal distance, sample rate, max speed etc.
These were at least some of the issues I encountered.
>
>> - REST service to acquire data as XML (e.g. GPX export)
>> - realtime tracking in browser with AJAX-based Pushlets
>> (www.pushlets.com)
>> - yesyes, GeoTracing does not comply with OpenGIS stacks yet
>>
>> I realize the above is somewhat long/unordered and not a direct reply
>> but I am enthousiastic about this topic and many issues discussed are
>> familiar to those that I have been working on the past year.
>
> Same here, don't worry about it, all good stuff from what I can tell.
>
> How would you guess this would fit into what has been described so far?
Good/difficult question. What I found missing (or maybe overlooked) in
the discussions are the intended applications. With GeoTracing
development was application-driven (rather than starting a platform)
more from a User/Multimedia/CMS viewpoint than a GIS-viewpoint. I would
like to see a list of domains/applications for which Mobile Data
Collection is intended. From there we may extract common requirements
and come up with an architecture.
Back to fitting in:
- like said: I don't see GeoTracing as the solution, what is mainly lacking:
a) openGIS-compliance, WFS etc (I sort of feel many of you require this)
b) broad client-support (currently J2ME Mobile and experimental
Desktop-browser)
c) local storage/remote sync
d) developers (now mainly a solo-project) and thus time
Just
>
> bobb
>
>>
>> best,
>>
>> --Just
>>
>> Just van den Broecke just at justobjects.nl
>> Just Objects B.V. tel +31 65 4268627 Skype: justb4
>> The Netherlands http://www.justobjects.nl
>>
>>
>>
>>
>>
>>
>>
>> Bob Basques wrote:
>>
>>> First off, I'm about ready to put some of this into a WIKI, just
>>> figured it needs to be hashed out here first. :c)
>>>
>>> Mateusz Loskot wrote:
>>>
>>>> Bob Basques wrote:
>>>>
>>>
>>>
>>> Development of a BETA package for Mobile Data capture:
>>>
>>>>> The CLI approach seems to most sane to me. This will most like
>>>>> change over time with some of the seperate utilities being combined
>>>>> into bigger and better things later on. One reason I like this
>>>>> approach is that it's very easy to segment the utilities into smaller
>>>>> work area/groups. As long as a good Comm directive is followed, ann
>>>>> utilities should work in harmony together. This "Utility"
>>>>> approach also affords us the opportunity to work precisely on the
>>>>> things that need to be worked on, while at the same time using the
>>>>> tools already developed for other things.
>>>>>
>>>>
>>>>
>>>>
>>>> What about the idea of modularized package + UI, for example command
>>>> line interface at the first stage?
>>>>
>>>
>>>
>>> I like thinking about this as a CGI sort of request handler. If this
>>> method were followed, then it's just that much easier to use
>>> components via a Webserver when the time comes, or even to combine
>>> utilities into a piped structure as handlers.
>>>
>>>>> Related to the above is the question of what build.
>>>>> I keep pointing
>>>>> to LINUX as the first version of things because it will require the
>>>>> least amount of NEW work to get a prototype operational.
>>>>> Basic requirements (as I see them):
>>>>>
>>>>> Every Device that is considered for porting, should have:
>>>>>
>>>>> * Have a Web Server of some sort (in the near term, nothing
>>>>> exotic should be attempted on the LINUX side, just keep it
>>>>> short and sweet so that the capabilities are easier to port
>>>>> to other OS's.
>>>>>
>>>>
>>>>
>>>>
>>>> What is the idea of communication, could you elaborate?
>>>>
>>>
>>>
>>> Since my notion is that the Main server (mothership) and the portable
>>> device (figher ship) will each have about the same capabilites
>>> (ideally THE same), the only real problem is with syncing the data
>>> between them. The next biggest piece will be in updating tracking
>>> layers at each remote device, but the DB sysning should take care of
>>> this easily.
>>>
>>>>
>>>>
>>>>> * Have a Web Browser that has at least some Javascript
>>>>> capabilities, (most do, BTW, although most are also
>>>>> incomplete when talking about what features they support,
>>>>> don't support)
>>>>>
>>>>
>>>>
>>>>
>>>> Yes, though it may be a problem with portability.
>>>> What Web browsers do you consider?
>>>>
>>>
>>>
>>> Since I wouldn't advocate anything to wierd in the Client GUI, it
>>> could be just about anything with Javascript support, Blazer (PALM),
>>> Opera(lot's of portable OS's), WinCE, etc. Even some of the Phone
>>> browser should support basic Javascript operations. I say we shoot
>>> for most, but not all up front.
>>>
>>>>> * These might be optional items
>>>>> * Have a way of generating a Map, even in a simplified
>>>>> form as a standalone operation.
>>>>>
>>>>
>>>>
>>>>
>>>> What does "generate a map" means?
>>>> Do you mean it should be able to display map on a device screen?
>>>>
>>>
>>>
>>> Yup, the portable device acts just like a desktop version of thing,
>>> even when disconnected (it has some basic mapping data resident on
>>> the machine. Some smaller form factors will have limited space and
>>> the Syncing process could be made a bit more robust in these
>>> situations, where only a localized version of the data is resident on
>>> the portable device based on dispatching etc.
>>>
>>>>> * Have a way of connecting to a AVL hardware stack ??(etc.)
>>>>>
>>>>
>>>>
>>>>
>>>> I'm not experienced with technical specification of AVL but OK :-)
>>>>
>>>
>>>
>>> I explain some of this in the earlier Email, but the important piece,
>>> is that they too also capture GPS data as a built in Capability.
>>>
>>>>> If we can agree to the above, then what we'll need to build (short
>>>>> list):
>>>>>
>>>>> * Collection format for the data (Points only to start with?)
>>>>>
>>>>
>>>>
>>>>
>>>> What does "collection format" means?
>>>> Is this about log GPS locations or also from manual map editing?
>>>>
>>>
>>>
>>> Yes, and yes, with my description in the previous Email, I think you
>>> can see how this would work usig a database storage technique.
>>>
>>>>> * Database, Flat File, etc.
>>>>>
>>>>
>>>>
>>>>
>>>> IMHO one of the most important requirement is insert/update/query
>>>> performance with good indexing.
>>>> So, I'd stick to SQLite.
>>>>
>>>
>>>
>>> Any SQL database, I'm most familiar with MYSQL. But since I'm
>>> suggesting that no special data formatting is needed, that the data
>>> could reside in anything with a SQL retrieval mechanism.
>>>
>>>>
>>>>
>>>>> * Server comm LIB, Main (MotherShip) to Secondary (Attack
>>>>> Fighter) web servers.
>>>>> * Syncing of captured data points
>>>>> * Syncing of Mapping data between servers fore
>>>>> re-distribution.
>>>>> * Daemon (listener) to monitor GPS (could be a wrapper around
>>>>> any of the options already available for reading
>>>>> GPS devices.) coordinate locations, and other digital
>>>>> devices, Compass, etc.
>>>>>
>>>>
>>>>
>>>>
>>>> As Paulo has suggested, gpsd should work well.
>>>>
>>>
>>>
>>> See my notes on this in other Emails.
>>>
>>>>> * Some sort of GUI (web Based, even when running standalone)
>>>>> Do you mean it will be a JavaScript/AJAX application>
>>>>>
>>>
>>>
>>> Yes.
>>>
>>>>> * MapService, for providing Automated location services.
>>>>>
>>>>
>>>>
>>>>
>>>> I imagine it can be one of the biggest problem to find/create services
>>>> infrastructure.
>>>>
>>>
>>>
>>> I'm not sure I understand this statement. If by it you mean a
>>> Mapping system with this type data already present, I have access to
>>> this type of thing. It's a big reason why I'm pushing hard to make
>>> this component based, as the CLIENT end of anything will end up in
>>> our Main Spatial service, not as a seperate application, but as an
>>> enhancement to the existing one.
>>>
>>>>> could be a start for pulling a project proposal out of.
>>>>>
>>>>
>>>>
>>>> There are a bunch of details I'm glossing over here, but I think this
>>>>
>>>> Although it's hard for me to comment a Web-based interface for
>>>> GPS-driven application for data collecting, I think the plan is good.
>>>>
>>>> Here are my additional questions:
>>>> 1. What's the suggested development platform for the prototype?
>>>> 2. Are we still going to create and use underlying core components for
>>>> data accessing, processing, analysing, computing, etc?
>>>> 3. What about services infrastructure?
>>>>
>>>
>>>
>>> I think these questions should start to become clearer for all those
>>> reading so far, let me know if anyone is still needing some aspects
>>> (that I have described) explained further. I'm very interested in
>>> hearing about why my ideas are hair brained, etc. Don't worry about
>>> offending me either, I have this same problem with every application
>>> team I get on it seems. :c)
>>>
>>> bobb
>>>
>>>> Best regards
>>>>
>>>
>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: discuss-unsubscribe at mail.osgeo.org
>>> For additional commands, e-mail: discuss-help at mail.osgeo.org
>>>
>>>
>>>
>>
>>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: discuss-unsubscribe at mail.osgeo.org
> For additional commands, e-mail: discuss-help at mail.osgeo.org
>
>
>
--
--Just
Just van den Broecke just at justobjects.nl
Just Objects B.V. tel +31 65 4268627 Skype: justb4
The Netherlands http://www.justobjects.nl
More information about the Discuss
mailing list