[gdal-dev] New OGR driver to read OpenStreetMap .osm / .pbf files

[Even Rouault]

>
> Windows 7, 64-bit, SATA disk and 2x3 GHz is converting Finland.osm.pbf in
> about
> 8 minutes for me. But execution time does not increase at all linearly.
> Germany.osm.pbf is about 10 times larger in filesize but ogr2ogr had to work
> about 8 hours with it, thus roughly 70 times longer. Obviously I would save 6
> hours by splitting the German pbf file into 10 smaller ones, running ogr2ogr
> ten
> times and combining results with ogrtileindex for use with Mapserver.

I suppose there is a big discontinuity at some point. While the temporary
database can fit into RAM, and then in the I/O cache of the operating system,
the performance must be reasonably good. But when it grows above, you'll get
disk access for almost every way and this will be sluggish.

Do you have comparisons of the performance with osm2pgsql on the same PC and
with the same data ? I'd be curious if that slow down effect is found with every
tool, or if it is something specific to the way sqlite is used, or if other
tools do more clever things when indexing or retrieving nodes.

I suppose splitting could help, but I don't see an obvious reason why an
intelligent splitting would be faster. I mention "intelligent" because you can
do a rough splitting based on bounding box for example, but this will lead to
ways that have unresolved nodes.

Actually, you could try that by adding a -spat argument to your ogr2ogr command
line. Nodes that fall outside of the spatial filter are not added to the
temporary database. But of course ways that cross the boundary of the bounding
box will be truncated and some relations too. And no guarantee it will speed up
the global process, because some time will still be lost trying to resolve the
nodes of ways that are outside of the spatial filter.

>
> -Jukka Rahkonen-
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev
>