[GRASS-dev] [GRASS GIS] #2513: v.colors freezes using the pg driver and attr
GRASS GIS
trac at osgeo.org
Wed Dec 24 01:06:29 PST 2014
#2513: v.colors freezes using the pg driver and attr
-------------------------------------+--------------------------------------
Reporter: jamesp670 | Owner: grass-dev@…
Type: defect | Status: new
Priority: normal | Milestone: 7.0.0
Component: Database | Version: svn-trunk
Keywords: freeze in G_fatal_error | Platform: Linux
Cpu: x86-64 |
-------------------------------------+--------------------------------------
Description changed by martinl:
Old description:
> Hi Grass Devs,
>
> You've got a big fan in me. I acknowledged GRASS on my AGU (see agu.org)
> poster this year and I'll make sure it's a cite next year.
>
> I have a bug report. This is getting complex and perhaps you can help?
>
> The setup:
>
> My uname output:
>
> : GRASS 7.0.0svn (rlis-master-for-seven):~ > uname -a
> : Linux login.cluster 2.6.32-431.17.1.el6.x86_64 #1 SMP Wed May 7
> 23:32:49 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
>
> I built grass from source, using primarily deps supplied by
>
> : yum install http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/\
> : pgdg-redhat93-9.3-1.noarch.rpm
>
> The build is painless. GRASS works great, except for this bug.
>
> I run GRASS v.colors to obtain a color map for TIGER census, as in:
>
> : GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
> map=tract2010censusdp1_clipped_02 column=popden10 layer=1
> rgb_column=GRASSRGB color=grey
>
> What I expect:
>
> GRASSRGB column is created and populated with greyscale values.
>
> What happens instead, is that v.colors freezes. Specifically, I see this
> sequence:
>
> : GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
> map=tract2010censusdp1_clipped_02 column=popden10 layer=1
> rgb_column=GRASSRGB color=grey
> : Option <column> given, assuming <use=attr>...
> : Converting color rules into categories...
> : Writing color rules...
> : DBMI-PostgreSQL driver error:
> : Unable to execute:
> : ALTER TABLE tract2010censusdp1_clipped_02 ADD COLUMN GRASSRGB
> VARCHAR(11)
> : ERROR: column "grassrgb" of relation "tract2010censusdp1_clipped_02"
> already exists
> :
> :
> : DBMI-PostgreSQL driver error:
> : Unable to execute:
> : ALTER TABLE tract2010censusdp1_clipped_02 ADD COLUMN GRASSRGB
> VARCHAR(11)
> : ERROR: column "grassrgb" of relation "tract2010censusdp1_clipped_02"
> already exists
> :
> :
> : ERROR: Unable to add column <GRASSRGB> to table
> : <tract2010censusdp1_clipped_02>
> : no database is open
> : no database is open
> : C-c C-c
> : GRASS 7.0.0svn (rlis-master-for-seven):~ >
>
> Where the C-c C-c is, I am tapping C-c to break the task, which is
> blatently frozen.
> I've debugged it. The sequence of events is:
>
> : print "no database is open"
> : db_shutdown_driver(driver=00661C40)
> : doing waitpid(pid=1583)
> : [freeze]
>
> During the freeze, proc 1583 is still running, and it's a db/pg process.
> If I "kill 1583", the v.colors job ends immediately with no additional
> output,
> as in
>
> : no database is open
> : no database is open
> : [I kill the db/pg process by hand]
> : GRASS 7.0.0svn (rlis-master-for-seven):~ >
>
> I've tried v.colors in the trunk and release SVN branches. The layer is
> fine,
> I can query it, view attributes just fine. db.test output looks fine.
> It's
> 100% reproducable.
>
> I build from source, debug with gdb, insert print statements. That's
> what got
> me this far, but now this IPC with the grass-7.0.0svn/driver/db/pg
> process is
> getting pretty complex. If I had to hazard a guess, I'd say that closing
> the
> communication FILEs in shutdown.c should cause the db/pg process to end.
> It
> is not for some reason. That then also begs the question of why the
> shutdown
> is called before the colors are written at all. The stack is:
>
> : (gdb) where
> : ,#0 0x0000003ab72ac8ce in waitpid () from /lib64/libc.so.6
> : ,#1 0x00002aaaab17f3ad in G_wait (i_pid=28458) at spawn.c:981
> : ,#2 0x00002aaaaad37348 in db_shutdown_driver (driver=0x661ec0) at
> shutdown.c:59
> : ,#3 0x00002aaaaad35e60 in error_handler_driver (p=0x661ec0) at
> handler.c:24
> : ,#4 0x00002aaaab1630ad in G__call_error_handlers () at handler.c:108
> : ,#5 0x00002aaaab15ee93 in G_fatal_error (
> : msg=0x405368 "Unable to add column <%s> to table <%s>") at
> error.c:176
> : ,#6 0x00000000004042b1 in write_rgb_values (Map=0x7fffffffcfb0,
> layer=1,
> : column_name=0x654830 "GRASSRGB", colors=0x7fffffffce90) at
> write_rgb.c:37
> : ,#7 0x0000000000402f83 in main (argc=7, argv=0x7fffffffd988) at
> main.c:350
>
> If I delete grassrgb by hand, v.colors does return neatly and works
> great, as in:
>
> : psql (9.3.5)
> : =>
> : => alter table tract2010censusdp1_clipped_02 drop column grassrgb;
> : ALTER TABLE
> : =>
>
> then
>
> : GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
> map=tract2010censusdp1_clipped_02 column=popden10 layer=1
> rgb_column=GRASSRGB color=grey
> h: Option <column> given, assuming <use=attr>...
> : Converting color rules into categories...
> : Writing color rules...
> : Column <GRASSRGB> added to table <tract2010censusdp1_clipped_02>
> :
> : Color table for vector map <tract2010censusdp1_clipped_02 at PERMANENT>
> set to
> : 'grey'
> : GRASS 7.0.0svn (rlis-master-for-seven):~ >
>
> The content of GRASSRGB looks fine after that.
>
> Removing the color table using -r before running doesn't work, I get
>
> : GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors -r
> map=tract2010censusdp1_clipped_02 rgb_column=GRASSRGB
> : WARNING: Color table of vector map <tract2010censusdp1_clipped_02> not
> : found
> : GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
> map=tract2010censusdp1_clipped_02 column=popden10 layer=1
> rgb_column=GRASSRGB color=bgyr
> : [freeze]
>
> I obviously don't understand the difference between the color table and
> the rgb_column. v.colors seems to
> lack the usual --overwrite.
>
> So I have a workaround: remove the column by manual sql entry before
> running
> v.colors. The freeze is obviously a bug, and there seems to be a lack of
> clarity RE: the lack of --overwrite and the use of -r.
>
> How you can reproduce this at home:
>
> 1. Open http://rlisdiscovery.oregonmetro.gov/
>
> 2. CLick on "Download" for 2000 Block Groups.
>
> 3. Save it into ~/tmp
>
> 4. unzip blockgrp2000.zip
>
> 5. grass70 -gui
>
> 6. Location Wizard
>
> 7. Project Location: rlus-bug-report
>
> 8. Read projection ... data file
>
> 9. Georef file: blockgrp2000.shp
>
> 10. Do *not* Import the data (although that is a nice grass 7 touch
> there!)
>
> 11. start grass.
>
> 12. db.connect driver=pg database="host=myhost,dbname=mydb"
> [substitute your database of course]
>
> 13. v.in.ogr dsn=/home/powellj/tmp/blockgrp2000.shp output=blockgrp2000
> (4385 primitives imported in about a minute).
>
> 14. v.colors --verbose map=blockgrp2000 column=pop08 layer=1
> rgb_column=GRASSRGB color=bgyr
> [works great, finishes in < 1s]
>
> 15. Repeat step 14. It freezes immediately, reading "no database is
> open".
New description:
Hi Grass Devs,
You've got a big fan in me. I acknowledged GRASS on my AGU (see agu.org)
poster this year and I'll make sure it's a cite next year.
I have a bug report. This is getting complex and perhaps you can help?
The setup:
My uname output:
{{{
: GRASS 7.0.0svn (rlis-master-for-seven):~ > uname -a
: Linux login.cluster 2.6.32-431.17.1.el6.x86_64 #1 SMP Wed May 7 23:32:49
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
}}}
I built grass from source, using primarily deps supplied by
{{{
: yum install http://yum.postgresql.org/9.3/redhat/rhel-6-x86_64/\
: pgdg-redhat93-9.3-1.noarch.rpm
}}}
The build is painless. GRASS works great, except for this bug.
I run GRASS v.colors to obtain a color map for TIGER census, as in:
{{{
: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
map=tract2010censusdp1_clipped_02 column=popden10 layer=1
rgb_column=GRASSRGB color=grey
}}}
What I expect:
GRASSRGB column is created and populated with greyscale values.
What happens instead, is that v.colors freezes. Specifically, I see this
sequence:
{{{
: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
map=tract2010censusdp1_clipped_02 column=popden10 layer=1
rgb_column=GRASSRGB color=grey
: Option <column> given, assuming <use=attr>...
: Converting color rules into categories...
: Writing color rules...
: DBMI-PostgreSQL driver error:
: Unable to execute:
: ALTER TABLE tract2010censusdp1_clipped_02 ADD COLUMN GRASSRGB
VARCHAR(11)
: ERROR: column "grassrgb" of relation "tract2010censusdp1_clipped_02"
already exists
:
:
: DBMI-PostgreSQL driver error:
: Unable to execute:
: ALTER TABLE tract2010censusdp1_clipped_02 ADD COLUMN GRASSRGB
VARCHAR(11)
: ERROR: column "grassrgb" of relation "tract2010censusdp1_clipped_02"
already exists
:
:
: ERROR: Unable to add column <GRASSRGB> to table
: <tract2010censusdp1_clipped_02>
: no database is open
: no database is open
: C-c C-c
: GRASS 7.0.0svn (rlis-master-for-seven):~ >
}}}
Where the C-c C-c is, I am tapping C-c to break the task, which is
blatently frozen.
I've debugged it. The sequence of events is:
{{{
: print "no database is open"
: db_shutdown_driver(driver=00661C40)
: doing waitpid(pid=1583)
: [freeze]
}}}
During the freeze, proc 1583 is still running, and it's a db/pg process.
If I "kill 1583", the v.colors job ends immediately with no additional
output,
as in
{{{
: no database is open
: no database is open
: [I kill the db/pg process by hand]
: GRASS 7.0.0svn (rlis-master-for-seven):~ >
}}}
I've tried v.colors in the trunk and release SVN branches. The layer is
fine,
I can query it, view attributes just fine. db.test output looks fine.
It's
100% reproducable.
I build from source, debug with gdb, insert print statements. That's what
got
me this far, but now this IPC with the grass-7.0.0svn/driver/db/pg process
is
getting pretty complex. If I had to hazard a guess, I'd say that closing
the
communication FILEs in shutdown.c should cause the db/pg process to end.
It
is not for some reason. That then also begs the question of why the
shutdown
is called before the colors are written at all. The stack is:
{{{
: (gdb) where
: ,#0 0x0000003ab72ac8ce in waitpid () from /lib64/libc.so.6
: ,#1 0x00002aaaab17f3ad in G_wait (i_pid=28458) at spawn.c:981
: ,#2 0x00002aaaaad37348 in db_shutdown_driver (driver=0x661ec0) at
shutdown.c:59
: ,#3 0x00002aaaaad35e60 in error_handler_driver (p=0x661ec0) at
handler.c:24
: ,#4 0x00002aaaab1630ad in G__call_error_handlers () at handler.c:108
: ,#5 0x00002aaaab15ee93 in G_fatal_error (
: msg=0x405368 "Unable to add column <%s> to table <%s>") at
error.c:176
: ,#6 0x00000000004042b1 in write_rgb_values (Map=0x7fffffffcfb0,
layer=1,
: column_name=0x654830 "GRASSRGB", colors=0x7fffffffce90) at
write_rgb.c:37
: ,#7 0x0000000000402f83 in main (argc=7, argv=0x7fffffffd988) at
main.c:350
}}}
If I delete grassrgb by hand, v.colors does return neatly and works great,
as in:
{{{
: psql (9.3.5)
: =>
: => alter table tract2010censusdp1_clipped_02 drop column grassrgb;
: ALTER TABLE
: =>
}}}
then
{{{
: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
map=tract2010censusdp1_clipped_02 column=popden10 layer=1
rgb_column=GRASSRGB color=grey
h: Option <column> given, assuming <use=attr>...
: Converting color rules into categories...
: Writing color rules...
: Column <GRASSRGB> added to table <tract2010censusdp1_clipped_02>
:
: Color table for vector map <tract2010censusdp1_clipped_02 at PERMANENT> set
to
: 'grey'
: GRASS 7.0.0svn (rlis-master-for-seven):~ >
}}}
The content of GRASSRGB looks fine after that.
Removing the color table using -r before running doesn't work, I get
{{{
: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors -r
map=tract2010censusdp1_clipped_02 rgb_column=GRASSRGB
: WARNING: Color table of vector map <tract2010censusdp1_clipped_02> not
: found
: GRASS 7.0.0svn (rlis-master-for-seven):~ > v.colors --verbose
map=tract2010censusdp1_clipped_02 column=popden10 layer=1
rgb_column=GRASSRGB color=bgyr
: [freeze]
}}}
I obviously don't understand the difference between the color table and
the rgb_column. v.colors seems to
lack the usual --overwrite.
So I have a workaround: remove the column by manual sql entry before
running
v.colors. The freeze is obviously a bug, and there seems to be a lack of
clarity RE: the lack of --overwrite and the use of -r.
How you can reproduce this at home:
1. Open http://rlisdiscovery.oregonmetro.gov/
2. CLick on "Download" for 2000 Block Groups.
3. Save it into ~/tmp
4. unzip blockgrp2000.zip
5. grass70 -gui
6. Location Wizard
7. Project Location: rlus-bug-report
8. Read projection ... data file
9. Georef file: blockgrp2000.shp
10. Do *not* Import the data (although that is a nice grass 7 touch
there!)
11. start grass.
12. db.connect driver=pg database="host=myhost,dbname=mydb"
[substitute your database of course]
13. v.in.ogr dsn=/home/powellj/tmp/blockgrp2000.shp output=blockgrp2000
(4385 primitives imported in about a minute).
14. v.colors --verbose map=blockgrp2000 column=pop08 layer=1
rgb_column=GRASSRGB color=bgyr
[works great, finishes in < 1s]
15. Repeat step 14. It freezes immediately, reading "no database is
open".
--
--
Ticket URL: <http://trac.osgeo.org/grass/ticket/2513#comment:2>
GRASS GIS <http://grass.osgeo.org>
More information about the grass-dev
mailing list