[gdal-dev] GDAL/OGR 2.0.2 RC1 Available for Review

[Even Rouault]

Hi Jürgen,

your change looks sane, but I'm unable to reproduce an issue with the current 
code, as before each insertion in skip_failures mode, the previous transaction 
is committed (since -gt is forced to 1) (which seems to have the effect of a 
rollback, but that perhaps only works in some contexts) and restarted. I'd be 
curious to have a reproducer of the issue your encountered. From what I can 
see, this must have always existed in ogr2ogr (unless other changes in the PG 
driver itself now exhibit an issue that was hidden before?)

Here's what I tried:

$ ogrinfo pg: -sql "create table test(id varchar unique, foo varchar)"

$ printf "id,foo\n1,1\n3,3\n" > test.csv

$ ogr2ogr -append pg: test.csv

$ printf "id,foo\n1,1\n2,2\n3,3\n" > test.csv

$ ogr2ogr -append pg: test.csv -skip
ERROR 1: ERROR:  duplicate key value violates unique constraint "test_id_key"
DETAIL:  Key (id)=(1) already exists.

ERROR 1: INSERT command for new feature failed.
ERROR:  duplicate key value violates unique constraint "test_id_key"
DETAIL:  Key (id)=(1) already exists.

Command: INSERT INTO "test" ("id", "foo") VALUES ('1', '1')
ERROR 1: ERROR:  duplicate key value violates unique constraint "test_id_key"
DETAIL:  Key (id)=(3) already exists.

ERROR 1: INSERT command for new feature failed.
ERROR:  duplicate key value violates unique constraint "test_id_key"
DETAIL:  Key (id)=(3) already exists.

Command: INSERT INTO "test" ("id", "foo") VALUES ('3', '3')

$ ogrinfo pg: test

Layer name: test
Geometry: None
Feature Count: 3
Layer SRS WKT:
(unknown)
id: String (0.0)
foo: String (0.0)
OGRFeature(test):0
  id (String) = 1
  foo (String) = 1

OGRFeature(test):1
  id (String) = 3
  foo (String) = 3

OGRFeature(test):2
  id (String) = 2
  foo (String) = 2

I also tried with "--config pg_use_copy yes" and it also works.

Even

> On Fri, 22. Jan 2016 at 12:14:32 +0100, Even Rouault wrote:
> > ... and now a GDAL/OGR 2.0.2 release candidate. Please review and test.
> 
> I added it to OSGeo4W as experimental.
> 
> I found one problem that causes ogr2ogr to break when inserting duplicate
> data into postgres although -skipfailures is used.
> 
> Following patch fixes that for me (but I'm not sure if it affects other
> cases):
> 
> diff -ur ../orig/gdal-2.0.2/apps/ogr2ogr.cpp ./apps/ogr2ogr.cpp
> --- ../orig/gdal-2.0.2/apps/ogr2ogr.cpp 2016-01-22 11:51:09.000000000 +0100
> +++ ./apps/ogr2ogr.cpp  2016-01-23 00:43:21.392717800 +0100
> @@ -4049,6 +4049,17 @@
>              {
>                  CPLDebug( "OGR2OGR", "Unable to write feature "
> CPL_FRMT_GIB " into layer %s.\n", poFeature->GetFID(),
> poSrcLayer->GetName() ); +
> +                if( bLayerTransaction )
> +                {
> +                    poDstLayer->RollbackTransaction();
> +                    poDstLayer->StartTransaction();
> +                }
> +                else
> +                {
> +                    poODS->RollbackTransaction();
> +                    poODS->StartTransaction(bForceTransaction);
> +                }
>              }
> 
>  end_loop:
> 
> 
> Jürgen

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com