[GRASS-dev] how to increase performance of v.out.ogr to geopackage ?

[Markus Metz]

On Thu, Dec 20, 2018 at 10:34 AM Markus Metz <markus.metz.giswork at gmail.com>
wrote:
>
>
>
> On Thu, Nov 29, 2018 at 3:43 PM Moritz Lennert <
mlennert at club.worldonline.be> wrote:
> >
> > Hi,
> >
> > I have large amount of vector maps (~ 400) which each has hundreds of
> > thousands of areas and over 150 attribute columns. I'm looking for the
> > most efficient way to export these to a format I can share with
> > non-GRASS GIS users. My favourite would probably be Geopackage, amongst
> > others because the attribute column names are sometimes fairly long,
> > thus making export to a DBF based format not easily feasible.
> >
> > I've seen several sources of information about possible performance
> > increases, either in the SQLite world (as a Geopackage is based on
> > SQLite) or in the OGR SQLite driver page.
> >
> > For OGR [1], there is the -gt option to ogr2ogr which allows increasing
> > the number of features per transaction (or even running the entire
> > conversion in one single transaction).
[...]
> > How about the
> > transaction option ?
>
> I'm afraid the transaction option (equivalent to ogr2ogr -gt) is out of
control of v.out.ogr because it is format-specific and v.out.ogr does not
use the output format drivers directly, instead it uses the generic OGR
interface. However, we could try
GDALDatasetStartTransaction()/GDALDatasetCommitTransaction() in v.out.ogr.

v.out.ogr already uses the OGR interface to start and commit transactions
when possible.

Another reason for slow export can be a buggy NFS implementation if you
export to a NFS mount: check if v.out.ogr is running at 100%. If not you
could export to a local temp dir and then copy the GeoPackages to the final
destination.

Markus M
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20181220/e12adde2/attachment.html>