[postgis-users] ogr2ogr error message for Tiger to PostgreSQL conversion

[Frank Warmerdam]

Timsy Bir wrote:
> Hi
> 
> I am trying to parse the Tiger data to a database. Someone recommended 
> the ogr2ogr tool, and I tried this:
> 
>   ogr2ogr -f "PostgreSQL" PG:dbname=mingle03 TGR08021.RT1
> 
> but got the error:
> 
>   ERROR 3: Failed to read record 500 of TGR08021.RT5
>   ERROR 1: INSERT command for new feature failed.
>   ERROR:  invalid UTF-8 byte sequence detected near byte 0xa2
> 
>   Command: INSERT INTO "polygon" ("module", "file", "cenid", "polyid", 
> "tract", "blocksufcu", "rs_a1", "aianhhfpcu", "aianhhcu", "aihhtlicu", 
> "anrccu", "aitscecu", "aitscu", "concitcu", "cousubcu", "sdelmcu", 
> "sdseccu", "sdunicu", "msacmsacu", "pmsacu", "necmacu", "rs_a5", 
> "rs_a6", "rs_a7", "rs_a8", "rs_a11", "rs_a13", "state", "aianhh", 
> "aihhtli", "aitsce", "aits", "cousub", "submcd", "place", "sdelm", 
> "sdsec", "sduni", "msacmsa", "cd106", "cd108", "puma5", "puma1", 
> "zcta5", "zcta3", "taz", "ua") VALUES ('TGR08021', 8021, 'C1506', 2, 
> 92071, '9', '7', 47001, 93, '3', 3, 0, 501, 0, 5100, '    0', '8021', '  
> ', 0, 0, 0, ',¢', '¿.£', '
>                           ·Ðo', ', '~¢¿', 'Àð·', 0, 802, '1', 920, 71, 
> 974, 70010, 59, 5, 0, 1, 6, 0, 81, 140, 0, '02080', '21C', '1506', 30)
> 
>   ERROR 1: Terminating translation prematurely after failed translation 
> of layer Polygon
> 
> Any suggestions on how to solve this? The postgis.sql and 
> spatial_ref_sys.sql scripts are loaded onto the database, but other than 
> that, no tables have been set up. I am a beginner and would appreciate 
> any suggestions or pointers.

Timsy,

Per the PG driver document:

  By default it is assumed that text being sent to Postgres is in the UTF-8
  encoding.  This is fine for plain ASCII, but can result in errors for
  extended characters (ASCII 155+ for instance).  Whle OGR provides no direct
  control over this, you can set the PGCLIENTENCODING environment variable
  to indicate the format being provided.  For instance, if you text is
  LATIN1 you could set the environment variable to LATIN1 before using OGR
  and input would be assumed to be LATIN1 instead of UTF-8. <p>

Best regards,
-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | President OSGF, http://osgeo.org