[GRASS-user] speeding up v.clean for large datasets

[Markus Metz]

On Fri, Apr 19, 2013 at 9:06 AM, Mark Wynter <mark at dimensionaledge.com> wrote:
> Hi All, we're looking for ways to speed up the cleaning of a large OSM road network (relating to Australia).  We're running on a large Amazon AWS EC2 instance.
>
> What we've observed is exponential growth in time taken as number of linestrings increases.
>
> This means it's taking about 3 days to clean entire network.
>
> We were wondering if we were to split the dataset into say 4 subregions, and clean each separately, is it then possible to patch them back together at the end without having to run v.clean afterwards?  We want to be able to run v.net over the entire network spanning the subregions.
>
> Alternatively, has anyone found a way to speed up v.clean for large network datasets?

Yes, implemented in GRASS 7 ;-)

Also, when breaking lines it is recommended to split the lines first
in smaller segments with v.split using the vertices option. Then run
v.clean tool=break. After that, use v.build.polylines to merge lines
again. Or use in GRASS 7 the -c flag with v.clean tool=break
type=line. The rmdupl tool is then automatically added, and the
splitting and merging is done internally.

Markus M


>
>
>
>
> GRASS 6.4.3svn (road_network):/data/grassdata > v.clean input=osm_roads output=osm_roads_cleaned tool=break,rmdupl
> --------------------------------------------------
> Tool: Threshold
> Break: 0.000000e+00
> Remove duplicates: 0.000000e+00
> --------------------------------------------------
> Copying vector lines...
> Rebuilding parts of topology...
> Building topology for vector map <osm_roads_cleaned>...
> Registering primitives...
> 971074 primitives registered
> 13142529 vertices registered
> Number of nodes: 1458192
> Number of primitives: 971074
> Number of points: 0
> Number of lines: 971074
> Number of boundaries: 0
> Number of centroids: 0
> Number of areas: -
> Number of isles: -
> --------------------------------------------------
> Tool: Break lines at intersections
>
>
> _______________________________________________
> grass-user mailing list
> grass-user at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/grass-user