[postgis-users] Shp2pgsql : unable to convert french UTF8encoded shapefiles

[Paul Ramsey]

That's ISO 8859-1, not UTF-8

http://en.wikipedia.org/wiki/ISO_8859-1

In UTF-8, é is 0xc3a9

P.

On 4-Feb-08, at 7:40 AM, Alexandre Gacon wrote:

> Hi Michael,
>
> I have checked my data and it is UTF8 (the 'é' character is encoded  
> as E9).
>
> The "SHOW client_encoding" return UNICODE as result. The encoding  
> field of
> my database displays UTF8.
>
> I join a small shapefile to test with.
>
> I use the following commands
>
> shp2pgsql fra94034______00 teleatlas_2007.testdb -W "UTF-8" >  
> myfile.sql
>
> And
>
> pgsql -d gis_database -f myfile.sql -U postgres.
>
> Sincerelly,
>
> Alexandre Gacon
>
> -----Message d'origine-----
> De : postgis-users-bounces at postgis.refractions.net
> [mailto:postgis-users-bounces at postgis.refractions.net] De la part  
> de Michael
> Fuhr
> Envoyé : lundi 4 février 2008 16:28
> À : PostGIS Users Discussion
> Objet : Re: [postgis-users] Shp2pgsql : unable to convert french  
> UTF8encoded
> shapefiles
>
> On Mon, Feb 04, 2008 at 10:29:17AM +0100, Alexandre Gacon wrote:
>> I am trying to load several shapefiles provided by TéléAtlas in a
>> postgis database with shp2pgsql on Windows XP and I have several  
>> problems.
>>
>> The shapefiles contains the data for France and are encoded in UTF8.
>
> Are you certain the data is UTF-8?  Have you verified by looking at  
> a hex
> dump?  Might some or all of it be in another encoding?
>
>> If I generates pgsql files without indicating the encoding to use,
>> shp2pgsql works fine but I can't execute the file with psql : I  
>> got an
>> error about an "Illegal byte sequence". If I open a pgsql file with
>> the pgAdmin tool, I manage to execute the sql request and it works  
>> fine.
>
> What does "SHOW client_encoding" return in pgAdmin?
>
>> This solution helps me but it is not enough : I can't do this for
>> large shapefiles (the pgAdmin tool can not open the file) and I have
>> several hundreds of files to load.
>>
>> If I try to generate the pgsql file with the -W "UTF8" option, I got
>> the same error than the psql command : "utf8 : Illegal byte  
>> sequence".
>
> Do you have a small shapefile you could make available for others  
> to test
> with?  What are the exact shp2pgsql and psql commands you're using?
>
> --
> Michael Fuhr
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis- 
> users<94034______.zip>_______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users