[PostGIS] #110: shp2pgsql new option to not batch commit

PostGIS trac at osgeo.org
Mon Jun 22 12:40:34 PDT 2026


#110: shp2pgsql new option to not batch commit
--------------------------+---------------------------
  Reporter:  robe         |      Owner:  mcayland
      Type:  enhancement  |     Status:  closed
  Priority:  low          |  Milestone:  PostGIS 2.0.0
 Component:  postgis      |    Version:  master
Resolution:  fixed        |   Keywords:
--------------------------+---------------------------
Changes (by komzpa):

 * milestone:  PostGIS Fund Me => PostGIS 2.0.0


Old description:

> '''What steps will reproduce the problem?'''
> 1. If you have some geometries that fail insert such as polygons without
> closed rings, it kills the current batch and gives a more or less
> meaningless message
> 2. This has happened to me a lot and a lot of users we train.  A lot of
> the
> time the data should just rightfully not be added.
> 3. Write now to overcome this I generate a .sql file and use sed to
> remove
> the begin commits which means I can't use the normal | to directly load
>
> I think the simplest option is to allow users with a flag to have the
> option to not have begin commits so that everything is in its own
> transaction.  That way at least a whole batch of good records aren't lost
> because of one bad apple.
>
> I know we have talked about other options such as allowing these beasts
> into the database or nulling the geometry if invalid etc -- though those
> options are trickier to implement and may not handle all cases.

New description:

 '''What steps will reproduce the problem?'''
 1. If you have some geometries that fail insert such as polygons without
 closed rings, it kills the current batch and gives a more or less
 meaningless message
 2. This has happened to me a lot and a lot of users we train.  A lot of
 the
 time the data should just rightfully not be added.
 3. Write now to overcome this I generate a .sql file and use sed to remove
 the begin commits which means I can't use the normal | to directly load

 I think the simplest option is to allow users with a flag to have the
 option to not have begin commits so that everything is in its own
 transaction.  That way at least a whole batch of good records aren't lost
 because of one bad apple.

 I know we have talked about other options such as allowing these beasts
 into the database or nulling the geometry if invalid etc -- though those
 options are trickier to implement and may not handle all cases.

--
-- 
Ticket URL: <https://trac.osgeo.org/postgis/ticket/110#comment:18>
PostGIS <http://trac.osgeo.org/postgis/>
The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.


More information about the postgis-tickets mailing list