[postgis-users] Checking Road Network Topology

[Mark Wynter]

Hi List

Interesting topic.  I’ve developed a solution for processing large scale road networks (OSM) using techniques that strk and R?mi-C have mentioned.

The solution uses PostGIS for “breaking" and “nodeing” . GRASS for snapping and patching.

The PostGIS step uses a custom SQL function to break lines according to a set of thresholds and rules
- the function diligently deals with bridges and tunnels, and special cases such as motorways, motorway_links, etc, to break lines where they need to be broken
- the function does not rely on PostGIS topology - just plain vector operations.
- the PostGIS step can be executed in parallel (if you tessellate the network first to produce “vector” tiles)  - not dissimilar to the concept of Map Reduce.  For example, the whole of Australia (1.6M+ roads) takes 12 minutes to process in PostGIS on a 32 vCPU AWS instance.  Scaling out to process even bigger road networks is no problem because each tile can be processed independently, and stitched back together afterwards.

The GRASS step involves 5 operations
1) v.clean tool=snap # no bridges, no tunnels
2) v.clean tool=snap # bridges only
3) v.clean tool=snap # tunnels only
4) v.patch (steps 1-3)
5) v.clean tool=snap # on the patched layer, with a much smaller tolerance to that used in steps 1-3 to again avoid say tunnel nodes inadvertently snapping to bridge nodes.

GRASS snapping and patching when done for whole of Australia in one go takes 48 minutes on the same 32vCPU machine setup - but with 31 CPUs sitting idle :-((.   I could cut the GRASS time down massively if I were to parallel process each vector tile.  The GRASS setup uses the PG Driver (PostgreSQL) to store the road attribute tables and v.out.postgis -f to write the final vector geometries back to PostGIS.

Figuring out this process to produce a highly representative network topology has not been trivial.  The workflow is scripted in bash, and uses psql, GNU Parallel.   I use QGIS Desktop 2.6 (multithreaded rendering) installed on the processing server to visualise the PostGIS outputs.

This may not give immediate comfort to Stephan who may need something here and now.

Longer term, I’m happy to look at how I can contribute this solution to the PostGIS / OSM community - and to see if I can pick up paid work in this space.   I’ve funded all of this myself (forgoing income for myself and my family in the process).  I’ve also been developing OSM map matching techniques, where you process “crowd sourced” GPS feeds to infer actual travel speeds along each road segment (time of day, day of week etc) - which can then be used to inform service planning processes, and routing algorithms using accurate information about local traffic conditions, congestion etc.

I’m open to any ideas, leads or offers of help.

Mark


On Wed, Jan 21, 2015 at 09:11:16AM +0100, R?mi Cura wrote:
> Hey,
> we previously discussed this matter of cleaning topology.
> Postgis topology is about iso topology storage and use, not cleaning.

That said, you can still use PostGIS for cleaning the data you're willing
to store in a topological model. It might need you to make your own
cleaning procedures, but we're happy to accept contributions :)

--strk;


Yeah sure , it was discussed on the user list :

http://lists.osgeo.org/pipermail/postgis-users/2014-November/039758.html

I personnaly tried the grass v.clean and it is fast and efficient (altough
the grass UI is terrible in my opinion)

Cheers,
R?mi-C