[gdal-dev] How to set OGR_INTERLEAVED_READING when using the OSM driver

[Paolo Corti]

Hi Even and Jukka, please follow me inline:

On Fri, Mar 15, 2013 at 10:47 PM, Even Rouault
<even.rouault at mines-paris.org> wrote:
>>
>> According to the documentation anyway, the option should not be set at
>> all, as "the ogr2ogr application has been modified to use that
>> OGR_INTERLEAVED_READING mode without any particular user action".
>>
>> Ideas? Am I doing something obviously wrong?
>
> Indeed, there's nothing special to do. The "ogr2ogr test.shp myfile.pbf lines"
> line is sufficient and I've just tested it successfully.
>
> My hypothesis, from the above debug log, would be that you don't run the
> ogr2ogr binary from trunk, but perhaps the one from a previous release...
>

You are right, I had a previous GDAL version on the computer I am
working (I think 1.9.2), but after building from trunk the GDAL
commands seem to have been correctly updated:

$ ogr2ogr --version
GDAL 1.10dev, released 2011/12/29

> Configuration options are given as --config OPTION VALUE
>
> This is the command I have been using for converting OSM data into Spatialite
>
> ogr2ogr finland.sqlite finland.osm.pbf -f SQLite -dsco SPATIALITE=YES
> -gt 10000 --config OGR_SQLITE_SYNCHRONOUS OFF

Thanks for the information, but it is not working for me.
I am trying to load to a PostGIS layer using the following syntax. It
seems that first time the table is only created, without uploading
features to it.
Using ogr2ogr with the -append option after it, I will be able to
manage to upload features to it, but only 10k at a time, as the
OGR_INTERLEAVED_READING configuration seems to be ignored when
appending:

The first time I run ogr2ogr, PostGIS layer is correctly created, but
features not uploaded:

$ ogr2ogr -f PostgreSQL -lco GEOMETRY_NAME=the_geom -nln
myschema.osm_roads PG:"dbname='mydb' user='me' password='mypassword'"
myosm.pbf lines --config OGR_INTERLEAVED_READING YES --debug ON
OGR: OGROpen(myosm.pbf/0x91e21c8) succeeded as OSM.
PG: DBName="'mydb'"
PG: PostgreSQL version string : 'PostgreSQL 9.1.8 on
i686-pc-linux-gnu, compiled by gcc (Ubuntu/Linaro 4.6.3-1ubuntu5)
4.6.3, 32-bit'
PG: PostGIS version string : '2.0 USE_GEOS=1 USE_PROJ=1 USE_STATS=1'
PG: Layer 'osm_roads' geometry type: POINT:Point, Dim=2
OGR_PG_NOTICE: NOTICE:  CREATE TABLE will create implicit sequence
"osm_roads_ogc_fid_seq" for serial column "osm_roads.ogc_fid"
OGR_PG_NOTICE: NOTICE:  CREATE TABLE / PRIMARY KEY will create
implicit index "osm_roads_pk" for table "osm_roads"
PG: Primary key name (FID): ogc_fid
PG: Using column 'ogc_fid' as FID for table 'osm_roads'
PG: Layer 'osm_roads' geometry type: LINESTRING:Line String, Dim=2
OSM: OGR_INTERLEAVED_READING = 1
OSM: Got bounds : minx=11.361760, miny=40.717850, maxx=14.099890, maxy=42.916190
OSM: For layer points, new max size is 128
OSM: For layer points, new max size is 320
OSM: For layer points, new max size is 608
OSM: For layer points, new max size is 1040
OSM: For layer points, new max size is 1688
OSM: For layer points, new max size is 2660
OSM: For layer points, new max size is 4118
OSM: For layer points, new max size is 6305
OSM: For layer points, new max size is 9585
OSM: For layer points, new max size is 14505
OSM: For layer points, new max size is 21885
OSM: For layer points, new max size is 32955
OSM: For layer points, new max size is 49560
OSM: Switching to 'points' as they are no more feature in 'lines'
OGR2OGR: 0 features written in layer 'myschema.osm_roads'
OSM: Number of bytes read in file : 14874118

And after running again ogr2ogr with the append option, the first
10.001 features are appended to the layer:

$ ogr2ogr -f PostgreSQL -lco GEOMETRY_NAME=the_geom -nln
myschema.osm_roads PG:"dbname='mydb' user='me' password='mypassword'"
myosm.pbf lines -append
WARNING: Layer creation options ignored since an existing layer is
         being appended to.
ERROR 1: Too many features have accumulated in lines layer. Use
OGR_INTERLEAVED_READING=YES mode

If I try to use the OGR_INTERLEAVED_READING option with -append, the
process won't work as "Layer creation options ignored since an
existing layer is being appended to." according to the output

thanks for your invaluable feedback
p

-- 
Paolo Corti
Geospatial software developer
web: http://www.paolocorti.net
twitter: @capooti
skype: capooti