[postgis-users] Dealing with bogus records

[david.garnier at etudier-online.com]

En réponse à Andy Turk <andy at streetlight.com>:

> On Sunday 26 May 2002 01:47 pm, you wrote:
> > Hello,
> > I'm using PostGIS for a project with large shapefiles (for now, at
> least
> > 600.000 records) and I have a problem with bogus records. In my
> shapefiles,
> > some records are bogus, things like "A11" in a integer column,
> 81120-9
> > instead of 81129, this kind of things. I think that there is less than
> 10
> > problematic records per shapefile.
> 
> I noticed similar problems in converting TIGER data. My fix was to
> change the 
That's the data I use.
> schema so that varchar is used instead of integer for the offending
> columns.
This isn't a really good solution since you lose some metadata about your 
Shapefiles and indexing on strings is less efficient than indexing on numbers.

Obviously, shp2pgsql isn't very good at picking up the correct type (like char
(2) instead of varchar(), but how could it? I would suggest everyone using 
shp2pgsl to manually adjust the table schema: editing the output of shp2pgsql -
c is fine.

Finally, I solved by problem by manually commenting out the code inserting the 
transaction statements ("begin;" and "end;"). It makes things just a little bit 
slower.

Best regards,
David Garnier