[gdal-dev] Improve clipping of shapefiles

[Paul Meems]

Thanks Mike for your reply.

After clipping I'm doing a dissolve. The clipping takes 5 min, which is
more or less acceptable for 2.8 million shapes.
But the dissolve is taking over 30 min. for 680k shapes.

I've added a spatial index using
dsShapefile.ExecuteSQL("CREATE SPATIAL INDEX ON " + layerName, null,
"sqlite");
but this didn't help with the clipping.

Before dissolving I add an attribute index:
dsShapefile.ExecuteSQL($"CREATE INDEX ON {layerName} USING Avg1", null,
null);
and to dissolve I use
.OpenEx(input, 4, null, null, null);
.wrapper_GDALVectorTranslateDestName();

with these options:
var options = new[]
{
    "-f", "ESRI Shapefile",
    "-overwrite",
    "-explodecollections",
    "-dialect", "sqlite",
    "-sql", "SELECT ST_Union(geometry), Avg1 FROM {layerName} GROUP BY Avg1"
};

When I import the shapefile in PostGIS and do the same query it just takes
6 min.
So I keep thinking I'm not doing it right with GDAL/OGR.

As Jan Heckman suggested I tried using the Clipper library. Luckily it is
implemented in MapWinGIS as well.
The dissolve using Clipper is just taking 4 min.
I prefer not to use yet another library.

So I'm still open for more suggestions.




Paul

*Paul Meems *
Release manager, configuration manager
and forum moderator of MapWindow GIS.
www.mapwindow.org

Owner of MapWindow.nl - Support for
Dutch speaking users.
www.mapwindow.nl


*The MapWindow GIS project has moved to GitHub
<https://github.com/MapWindow>!*


Download the latest MapWinGIS mapping engine.
<https://github.com/MapWindow/MapWinGIS/releases>

Download the latest MapWindow 5 open source desktop application.
<https://github.com/MapWindow/MapWindow5/releases>

2017-09-11 23:34 GMT+02:00 Mike Toews <mwtoews at gmail.com>:

> On 11 September 2017 at 21:09, Paul Meems <bontepaarden at gmail.com> wrote:
> > I have a large shapefile with over 2.8 million shapes (fishnet) and I
> have a
> > border file with only 1 multipolygon.
> >
> > I'm trying to clip the fishnet with the border.
> > Using code is takes about 5 min. using command line it takes even longer.
> >
> > This is my command:
> > ogr2ogr fishnetClipped.shp fishnet.shp -clipsrc border.shp
> > ...
> > Can I somehow improve the speed?
>
> If the fishnet shapefile does not have a spatial index, try creating
> one. There are a few ways to do this:
> http://www.gdal.org/drv_shapefile.html
>
> I'm not sure if -clipsrc datasource will use the spatial index, but
> it's worth trying.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20170912/48e87c12/attachment-0001.html>