[gdal-dev] OSM Driver and World Planet file (pbf format)

[Smith, Michael ERDC-CRREL-NH]

Even, 

[osmusr at bigserver-proc osm]$ ogr2ogr -progress -f oci
oci:user/pass at tns:tmp planet-latest.osm.pbf -lco dim=2 -lco srid=4326 -lco
geometry_name=geometry -lco launder=yes --debug on  2>osm_debug.log
0...10...20...30...40...50...60Š70
[osmusr at bigserver-proc osm]$



>From the debug output

OCI: Flushing 100 features on layer POLYGONS
OCI: Flushing 100 features on layer POLYGONS
OCI: Flushing 100 features on layer POLYGONS
OCI: Flushing 100 features on layer POLYGONS
OCI: Flushing 100 features on layer POLYGONS
OCI: Flushing 100 features on layer POLYGONS

OCI: Flushing 100 features on layer POLYGONS
OSM: Switching to 'lines' as they are too many features in 'polygons'
OGR2OGR: 32827 features written in layer 'POLYGONS'
OCI: In Create Layer ...
OCI: Prepare(CREATE TABLE "MULTILINESTRINGS" ( OGR_FID INTEGER, geometry
MDSYS.SDO_GEOMETRY ))
OGR2OGR: 0 features written in layer 'MULTILINESTRINGS'
OCI: In Create Layer ...
OCI: Prepare(CREATE TABLE "MULTIPOLYGONS" ( OGR_FID INTEGER, geometry
MDSYS.SDO_GEOMETRY ))
OGR2OGR: 0 features written in layer 'MULTIPOLYGONS'
OCI: In Create Layer ...
OCI: Prepare(CREATE TABLE "OTHER_RELATIONS" ( OGR_FID INTEGER, geometry
MDSYS.SDO_GEOMETRY ))
OGR2OGR: 0 features written in layer 'OTHER_RELATIONS'
OCI: Flushing 23 features on layer POINTS
OCI: Flushing 99 features on layer LINES
OCI: Flushing 27 features on layer POLYGONS
OSM: nNodeSelectBetween = 50006
OSM: nNodeSelectIn = 94362
VSI: ~VSIUnixStdioFilesystemHandler() : nTotalBytesRead = 12682608949


(note that I removed some alter table lines for clarity)


Mike

On 7/23/12  9:16 AM, "Smith, Michael ERDC-CRREL-NH"
<Michael.Smith at usace.army.mil> wrote:

>Even,
>
>It stopped cleanly (no segfault) at 70%. OS is RHEL 6.2 64 bit. Import
>time was about 340 min.
>
>Command was 
>
>ogr2ogr -progress -f oci oci:user/pass at tns:tmp planet-latest.osm.pbf -lco
>dim=2 -lco srid=4326 -lco geometry_name=geometry -lco launder=yes
>
>I'm rerunning now with the debug log to a file.
>
>Mike
>
>-- 
>Michael Smith
>
>Remote Sensing/GIS Center
>US Army Corps of Engineers
>
>
>
>On 7/23/12  7:05 AM, "Even Rouault" <even.rouault at mines-paris.org> wrote:
>
>>Le lundi 23 juillet 2012 12:56:12, Smith, Michael ERDC-RDE-CRREL-NH a
>>écrit :
>>> I'm finding that the new OSM Driver (I tested again with r24699) has a
>>> problem when working with the whole planet file. When I tried with the
>>>US
>>> Northeast subset, I got multipolygons and multilinestring entries. When
>>> reading the whole planet file, I did not. It gets to 70% and then ends
>>> (but without an error message). I also got fewer polygons than I was
>>> expecting. It seems like the reading got interrupted by some non
>>>reported
>>> error.
>>> 
>>> I was writing to Oracle for this importing but got the same results
>>>writing
>>> to sqlite. It seems that smaller extracts work fine but the are some
>>> reading issues with the whole planet file (in pbf format). I can try
>>>with
>>> the .osm format.
>>
>>I didn't try yet with whole planet files. Takes too much time :-)
>>
>>Which command line did you use exactly ?
>>
>>Did it stop cleanly or with a segfault ? In the latter case, (assuming
>>you are 
>>on Linux), running under gdb might be useful.
>>
>>What is your OS, 32/64 bit ? Perhaps, you could add --debug on. I'd
>>suggest 
>>redirecting standard error file to a file because the log file can be
>>huge.
>>_______________________________________________
>>gdal-dev mailing list
>>gdal-dev at lists.osgeo.org
>>http://lists.osgeo.org/mailman/listinfo/gdal-dev
>
>_______________________________________________
>gdal-dev mailing list
>gdal-dev at lists.osgeo.org
>http://lists.osgeo.org/mailman/listinfo/gdal-dev