[gdal-dev] Alternative to GPKG format

[koji higuchi]

Hi Even
I managed to write gpkg without ogr without fiona using class
ShapeConverter(osmium.SimpleHandler).
still, speed seems slow.
Any idea to increase it up.

On Thu, Nov 1, 2018 at 7:05 PM koji higuchi <koji51guchi at gmail.com> wrote:

> Hi Even
>
> I am trying to replace using fiona by pure ogr.
> but having difficult to collapse the code with class and objects.
> If i could extract geom and tags in a ogr pythonic way, then it be better
>
>
> On Thu, Nov 1, 2018 at 6:45 PM Even Rouault <even.rouault at spatialys.com>
> wrote:
>
>> You are perhaps using a too old version of Fiona. Recent versions insert
>> features within OGR transactions. Should be fine with latest Fiona 1.8.0
>> See https://github.com/Toblerity/Fiona/issues/476
>>
>> > Hi, following is the code I used:
>> >
>> > Import os, osmium, fiona
>> >
>> > fi = 'europe-latest.osm.pbf'
>> > fo = 'europe-latest.dpkg'
>> >
>> > drv = 'DPKG'
>> >
>> > crs = {'no_defs': True, 'ellps': 'WGS84', 'datum': 'WGS84', 'proj':
>> > 'longlat'}
>> >
>> >  schema = {'geometry': 'LineString',
>> >                    'properties': {'id': 'float', 'name' : 'str', 'kind'
>> :
>> > 'str'}}
>> >
>> > outfile = fiona.open(fo, 'w', driver=drv, crs=crs, schema=schema)
>> >
>> > geomfab = osmium.geom.GeoJSONFactory()
>> >
>> > class ShapeConverter(osmium.SimpleHandler):
>> >       def way(self, w):
>> >            if 'place' in w.tags:
>> >                rec = {'geometry' : eval(geomfab.create_linestring(w)),
>> >                       'properties' : {'id' : float(w.id),
>> >                                      'name' : w.tags.get('name'),
>> >                                        'kind' : w.tags['place']}}
>> >                outfile.write(rec)
>> >
>> > ShapeConverter().apply_file(fi, locations=True)
>> >
>> > On Thu, Nov 1, 2018 at 4:10 PM jratike80 <
>> >
>> > jukka.rahkonen at maanmittauslaitos.fi> wrote:
>> > > koji higuchi wrote
>> > >
>> > > > Hi
>> > > > I am extracting .osm.pbf file into .gpkg; but the writing rate is
>> very
>> > > > slow.
>> > > > When I write into .shp, rate is faster but its limit is 4gb.
>> > > > So, what other format is better for larger than 4gb requirement?
>> > > > Thanks for your ideas.
>> > > > Koji
>> > >
>> > > Hi,
>> > >
>> > > Show the exact command that you are using so we can see if it could be
>> > > made
>> > > faster. For example if you happen to use -skipfailures for handling
>> the
>> > > invalid geometries of the OSM data then you will make GDAL to do a new
>> > > transaction for each row and that certainly is slow. But let's have a
>> look
>> > > at your command first.
>> > >
>> > > -Jukka Rahkonen-
>> > >
>> > >
>> > >
>> > > --
>> > > Sent from: http://osgeo-org.1560.x6.nabble.com/GDAL-Dev-f3742093.html
>> > > _______________________________________________
>> > > gdal-dev mailing list
>> > > gdal-dev at lists.osgeo.org
>> > > https://lists.osgeo.org/mailman/listinfo/gdal-dev
>>
>>
>> --
>> Spatialys - Geospatial professional services
>> http://www.spatialys.com
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20181101/69322f59/attachment.html>