[gdal-dev] OSM Driver and World Planet file (pbf format)

[Even Rouault]

Le lundi 23 juillet 2012 19:25:22, Smith, Michael ERDC-CRREL-NH a écrit :
> Even,
> 
> [osmusr at bigserver-proc osm]$ ogr2ogr -progress -f oci
> oci:user/pass at tns:tmp planet-latest.osm.pbf -lco dim=2 -lco srid=4326 -lco
> geometry_name=geometry -lco launder=yes --debug on  2>osm_debug.log
> 0...10...20...30...40...50...60Š70
> [osmusr at bigserver-proc osm]$
> 
> 
> 
> From the debug output
> 

Michael,

The debug output would suggest that there was no more data to process, which 
is strange. I've tested a bit with a planet file dating back to a few weeks, 
with a modified OSM driver that does basically no processing except the 
parsing, and it managed to parse until the end of file. So in your situation, 
I'd assume that there was a parsing error, but I'm not 100% positive (might be 
something wrong in the "interleaved reading" mode ?)

I've commited in r24707 a change that is mainly a custom indexation mechanism 
for nodes (can be disabled with OSM_USE_CUSTOM_INDEXING=NO) to improve 
performances (Improve them about by a factor of 2 on a 1 GB PBF on my PC) 
Along with that change, I've added some facility for extra error outputs. If a 
parsing error occured, an error message will be printed. And, before 
recompiling, you can edit ogr/ogrsf_frmts/osm/gpb.h and uncomment (by removing 
the // at the beginning of //#define DEBUG_GPB_ERRORS) line 40. This should 
report a more precise error if there's something wrong during the GPB parsing.
You might also retry with --debug OSM and, at the end of the processing, 
you'll see a trace "Number of bytes read in file : XXXXXXXXXXX" : you can check 
that the value is the same as the size of the PBF file.

Even