[GRASS-dev] new v.in.geonames: problems with UTF-8 Unicode text

[Glynn Clements]

Markus Neteler wrote:

> On Tue, Jul 1, 2008 at 7:12 PM, Glynn Clements <glynn at gclements.plus.com> wrote:
> >
> > Markus Neteler wrote:
> ...
> >> I assume that the escape magic in db/drivers/postgres/execute.c isn't
> >> perfect. Any suggestions here?
> >
> > Is there actually a problem? All I see are warnings.
> >
> > If I try inserting the above into a test table using the psql utility,
> > I get exactly what I would expect.
> ...
> > Note that the backslash is present in the file, which appears to be an
> > error; many other entries have literal apostrophes without the
> > backslash.
> 
> Ah, I overlooked that there is misfit in the input data.
> 
> (Un)related a segfault: When copying the map from the postgresql mapset
> into the sqlite mapset, it crashes:
> 
> # in SQLite mapset:
> GRASS 6.4.svn (latlong_tbe_climate): > gdb g.copy
> GNU gdb 6.6-3mdv2008.0 (Mandriva Linux release 2008.0)
> ...
> This GDB was configured as "x86_64-mandriva-linux-gnu"...
> Using host libthread_db library "/lib64/libthread_db.so.1".
> (gdb) r vect=it at postgresql,test --o
> Starting program:
> /home/neteler/grass64/dist.x86_64-unknown-linux-gnu/bin/g.copy
> vect=it at postgresql,test --o
> [Thread debugging using libthread_db enabled]
> [New Thread 46964157956272 (LWP 10388)]
> warning: Lowest section in /usr/lib64/libicudata.so.36 is .gnu.hash at
> 0000000000000120
> Copy vector <it at postgresql> to current mapset as <test>
> WARNING: Vector map <test> already exists and will be overwritten
> WARNING: Table <test> linked to vector map <test> does not exist
> 
> Program received signal SIGSEGV, Segmentation fault.
> [Switching to Thread 46964157956272 (LWP 10388)]
> 0x00002ab6aa9e8888 in db_execute_immediate (driver=0xab8be18d88e1a38c,
> SQLstatement=0x7fff00de1240)
>     at c_execute.c:16
> 16          db__set_protocol_fds (driver->send, driver->recv);
> (gdb) bt full
> #0  0x00002ab6aa9e8888 in db_execute_immediate
> (driver=0xab8be18d88e1a38c, SQLstatement=0x7fff00de1240)
>     at c_execute.c:16
>         ret_code = 32767
> #1  0x00002ab6aa9ea276 in db__copy_table (from_drvname=0x610420 "pg",

[snip]

>     nvals=-1846924119) at copy_tab.c:321

At the top of db__copy_table (lib/db/dbmi_client/copy_tab.c) is:

    char buf[1000]; 

I suspect that this is overflowing at line 295:

294			db_double_quote_string ( &value_string );
295			sprintf (buf, "'%s'", db_get_string(&value_string) );
296			db_append_string ( &sql, buf);

I suggest:

			db_double_quote_string ( &value_string );
			db_append_string ( &sql, "'");
			db_append_string ( &sql, db_get_string(&value_string));
			db_append_string ( &sql, "'");

-- 
Glynn Clements <glynn at gclements.plus.com>