[GRASS-dev] new v.in.geonames: problems with UTF-8 Unicode text

[Glynn Clements]

Markus Neteler wrote:

> I am writing v.in.geonames to easily read in data from
> http://download.geonames.org/export/dump/
> 
> The script is essentially using v.in.ascii to read in the CSV file encoded
> in UTF-8 Unicode text. There are placenames in various languages including
> Japanese.
> v.in.ascii isn't able to read them properly and fails on such lines, example:
> 
> 3165456 Torre del Greco Torre del Greco Torre d%27%27o Grieco,Torre
> d''o Grieco,Torre d''o Grieco,Torre del Greco,トッレ��デル・グレーコ
> 40.7839209532791        14.3708038330078        P       PPL     IT
>          04      NA      063084          90607           72
> Europe/Rome     2008-06-28
> 
> (I have slightly improved the v.in.ascii error message, not yet submitted):
> 
> ERROR: Unparsable latitude value in column <4>: 'o Grieco,Torre d''o
>        Grieco,Torre del Greco,トッレ・デル・グレーコ
> 
> How to fix this problem?

Can you please provide accurate and sufficient information about the
problem?

I.e. the exact data being fed to v.in.ascii (*before* it has been
mangled by the various components of the email chain), the v.in.ascii
command which is failing, etc.

-- 
Glynn Clements <glynn at gclements.plus.com>