[GRASS-dev] how to increase performance of v.out.ogr to geopackage ?

[Markus Neteler]

On Thu, Dec 27, 2018 at 11:02 AM Markus Neteler <neteler at osgeo.org> wrote:
> On Thu, Dec 20, 2018 at 10:34 AM Markus Metz
> <markus.metz.giswork at gmail.com> wrote:
> > On Thu, Nov 29, 2018 at 3:43 PM Moritz Lennert <mlennert at club.worldonline.be> wrote:
> > >
> > > Hi,
> > >
> > > I have large amount of vector maps (~ 400) which each has hundreds of
> > > thousands of areas and over 150 attribute columns. I'm looking for the
> > > most efficient way to export these to a format I can share with
> > > non-GRASS GIS users. My favourite would probably be Geopackage, amongst
> > > others because the attribute column names are sometimes fairly long,
> > > thus making export to a DBF based format not easily feasible.
> ...
> > For GDAL/OGR, you can set configuration options in various ways [0], e.g.
> >
> > OGR_SQLITE_CACHE=1024 OGR_SQLITE_SYNCHRONOUS=OFF v.out.ogr ...

# default settings
GRASS 7.6.svn (latlong_wgs84):~ > time -p v.out.ogr
input=brd_plz_gebiete_pop2011 output=brd_plz_gebiete_pop2011.gpkg
Exporting 45618 areas (may take some time)...
...
v.out.ogr complete. 47331 features (Polygon type) written to
<brd_plz_gebiete_pop2011> (GPKG format).
real 10.41
user 6.28
sys 7.16

# CACHE 1024
GRASS 7.6.svn (latlong_wgs84):~ > time -p OGR_SQLITE_CACHE=1024
v.out.ogr input=brd_plz_gebiete_pop2011
output=brd_plz_gebiete_pop2011_2.gpkg
Exporting 45618 areas (may take some time)...
...
v.out.ogr complete. 47331 features (Polygon type) written to
<brd_plz_gebiete_pop2011> (GPKG format).
real 9.71
user 6.11
sys 6.91

# CACHE 2048
GRASS 7.6.svn (latlong_wgs84):~ > time -p OGR_SQLITE_CACHE=2048
v.out.ogr input=brd_plz_gebiete_pop2011
output=brd_plz_gebiete_pop2011_3.gpkg
Exporting 45618 areas (may take some time)...
...
v.out.ogr complete. 47331 features (Polygon type) written to
<brd_plz_gebiete_pop2011> (GPKG format).
real 9.72
user 5.98
sys 6.96

# default again
GRASS 7.6.svn (latlong_wgs84):~ > time -p v.out.ogr
input=brd_plz_gebiete_pop2011 output=brd_plz_gebiete_pop2011_4.gpkg
Exporting 45618 areas (may take some time)...
...
v.out.ogr complete. 47331 features (Polygon type) written to
<brd_plz_gebiete_pop2011> (GPKG format).
real 9.84
user 6.12
sys 7.01

# CACHE AND SYNCHRONOUS=OFF
time -p OGR_SQLITE_CACHE=2048 OGR_SQLITE_SYNCHRONOUS=OFF v.out.ogr
input=brd_plz_gebiete_pop2011 output=brd_plz_gebiete_pop2011_5.gpkg
Exporting 45618 areas (may take some time)...
...
v.out.ogr complete. 47331 features (Polygon type) written to
<brd_plz_gebiete_pop2011> (GPKG format).
real 9.99
user 6.31
sys 6.95

--> (Here) the difference is very small.

Perhaps more striking with larger maps?

markusN