[GRASS-dev] new v.in.geonames: problems with UTF-8 Unicode text

[Hamish]

Markus:
> I am writing v.in.geonames to easily read in data from
> http://download.geonames.org/export/dump/
> 
> The script is essentially using v.in.ascii to read in the
> CSV file encoded in UTF-8 Unicode text.

I think I may have found a buffer overflow in their DB...  (NZ.zip export)

2181761	Taumatawhakatangihangakoauauotamateapokaiwhenuakitanatahu	Taumatawhakatangihangakoauauotamateapokaiwhenuakitanatahu
	Taumata-whakatangihanga-koauau-a-Tamatea-pokai-whenua-ki-tana-tahu,Taumata-whakatangihanga-kōauau-a-Tamatea-pōkai-whenua-ki-tana-tahu,Taumatauakatangiangakoauauotamateaturipukakapikimaungakhoronukupokanuehnuakitanatakhu,Taumatawhakatangihangakoauau,Taumatawhakatangihangakoauauotamateapokaiwhenuakitanatahu,Taumatawhakatangihangakoauauotamateaturipukakapikimaungahoronukupokaiwhenuakitanatahu,Taumatawhakatangihangakoauotamateaturipukakapikimaungahoronukupokaiwhenuakitanataha,Tetaumatafakatangikhangakoauaotamateaurekhaeaturipukapikhimaungakhoronukupokaifenuaakitanarakhu,Tetaumatawhakatangihangakoauaotamateaurehaeaturipukapihimaungahoronukupokaiwhenuaakitanarahu,ТауматауакатангиангакоауауотаматеатурипукакапикимаунгаÑ
оронукупокануэнуакÐ
¸Ñ‚анатаÑ
у,ТетауматафакатангиÑ
ангакоауаотаматеауреÑ
аеатурипукапиÑ
имаунгаÑ
оронукупокаифенуаакитанараÑ
у,タウマタファカタンギハンガコアウアウオタマテアポカイフェヌアキタナタフ,塔乌玛塔法卡塔尼哈娜可阿乌阿乌欧塔玛提亚坡凯费努啊奇塔娜塔胡	-40.35	176.55	T	HLL	NZ	NZ	00				0		191	Pacific/Auckland	2007-02-17


v.in.ascii chokes there for obvious reasons.


Move over small welsh towns!
http://en.wikipedia.org/wiki/Taumatawhakatangihangakoauauotamateapokaiwhenuakitanatahu
[checkout the translation(s)]



Hamish