[FOSS-GPS] Using the open GSM cell databases

[Onen]

Hi Baruch,

Baruch Even wrote:
> Onen wrote:
>  > What limitations do you have exactly in mind?
> 
> The APIs currently do not allow for synchronizing the cell database to 
> the user. This is the main deficit.
> 

Well my work will bring the database of positions to the user. Not the 
raw data though.

> Other things I'd like to keep for calculations is the tav, time advance, 
>  this information is only available during a phone call but it gives a 
> 547 meter multiplier of distance to the serving cell. It could help to 
> make cell location pretty good (I mean here to process of deciding where 
> the cell is location not where you are located in relation to the cell).
> 

I am aware of this. The openBmap logger client for 
Freesmartphone.org/openmoko phone, has the underlying code for this. 
Nevertheless I never activated it because as you said, the timing 
advance is only valid under certain circumstances. Thus the code would 
need to be tested in order to be sure the values are only recorded when 
valid. But so far lack of time has let it deactivated.

AFAIK under the openmoko phone, when timing advance is invalid, it 
returns a value which is also a valid value. Thus there is no way to 
distinguish in this specific case if the value is good or not :-(

> My user API is going to include two methods, upload and sync, the upload 
> is going to include any data point that *may* be relevant and the sync 
> should take a mcc/mnc pair and give you all the cell locations for that.
> 

Well why should you think about what may or may not be useful to upload? 
Don't you think that this should be the server side work, to consider 
only what makes sense for him, and provide as much raw data to people?

The sync aspect is for me taken care in the D-Bus location service.

> Obviously, the software will download information for the current 
> mcc/mnc pair, it should also allow the user to specify another pair, or 
> maybe just a country if the user plans to travel to that country.
> 

For now, I plan to provide directly a unique db for the whole world. The 
file is not that big.

> I would like to make the software "just work(TM)", no need to remember 
> it and maintain it. The user will install it, it will load for the first 
> time, see the mcc/mnc and immediately sync to get all cell locations for 
> that pair. Periodically it should try to sync with the server. The sync 
> should download only the differences in the cell locations.
> 

Yes, that is what I am working on for the location service.

> The server shall calculate the cell locations, I intended to implement 
> several algorithms and see which one is better, but the algorithms are 
> not critical, getting something to work all around is more important to 
> me. The algorithms can be refined later once we have it working and have 
> a user base to test them. My plan was to implement a simple cell sample 
> average as a cell location and another average of all seen cells for the 
> user location.
> 

You don't need to have a complex server with sync and so on, to try 
algorithms. The raw data are enough for you to evaluate different 
approaches. The server should provide the best algorithm to our 
knowledge. I don't see the point at developing a complete multi 
algorithms approach on the server side, don't you think?

>  >> I've started writing some bits of the code in Vala. I'd be happy to
>  >> cooperate to get a single solution that just works for everyone.
>  >>
>  >
>  > My service is almost ready. It is very basic though. It is written in 
> Python. Nevertheless I am not strongly minded to the programming 
> language, thus a shift is possible from my side.
> 
> I wanted something that works in the background on my FreeRunner so I 
> wanted something that takes little memory and little CPU. Python may be 
> good for prototyping but I don't like it on my FR.
> 

Agreed. As I said, I am not strongly minded about this. Nevertheless, if 
possible I would be interested in developing code that could be used 
under other platforms (Android, Nokia, etc.). Would Vala not tie us to 
the openmoko platform only (mostly)?

> 
>  > What are the functionalities which are implemented on your side?
> 
> For the client I only have code that collects the data for upload 
> purposes so as to update the server automatically about cell information.
> 
> The plan there is to have it listen to the real GPS all the time, if 
> there is a GPS signal (the user activated it) the data is collected 
> every 10 seconds with the GSM and GPS info to be automatically uploaded 
> to the server. This way the user both gets a location service and helps 
> to update it without needing to do anything manually.
> 

What I have had in mind for a long time is: a D-Bus service to 
log/upload data. This service could be configured to work automagically 
(I think this is what you are looking for), but could be driven manually 
by a GUI for people who want to keep control (for privacy protection for 
example).

What do you think, this way we could work together, at least on a common 
D-Bus logging engine?
(I only speak here of my "areas", as I am working on the logging client, 
and the location service. Nick is working on the server side).

> 
> As you can see, the automatic and "just work" is a theme for me. I have 
> a problem with getting into the car and fiddling with the FR to make it 
> do what I want. It should just work!
> 

We disagree on the automatic point. But as usual, I propose to make this 
configurable, because you are not the first person who tells me this is 
how he wants the logging to work.

To the "just work", I would add easy and stable. This are two points I 
always try to pay attention for the openBmap client.

> I'm currently in the middle of the work on the db schema and basic logic 
> to calculate cell locations using a dumb algorithm (simple averaging) 
> and then I planned to implement a sync logic on the server to fetch data 
> from cellhunter, simply because that's where I upload my data to, and 
> then implement the sync on the server and client and then I can 
> implement a simple user location algorithm and export it via a limited 
> Gypsy interface.
> 
> If it will be possible to work on the server side on an existing server 
> it would be much nicer, but I'd still like to have all the data I 
> mentioned above (tav is notoriously missing, I think openbmap doesn't 
> keep the rxlev either)

See above about tav, it is in openBmap client, only deactivated so far.

Rxlev (and a lot more ;-) ) is being logged by the obm client. Last time 
someone looked at it, openBmap was logging the biggest number of 
information AFAIK [1].

  and continuously (weekly basis) to sync with
> opencellid/cellhunter/openbmap to get have the largest cell database 
> possible.
> 

I am wondering if most (not all, agreed) of what you want is not already 
available...

> How does it sound to you?
> 

Pretty good ;-)

Onen

[1] http://lists.openmoko.org/pipermail/community/2009-June/049241.html