[GRASS-dev] Some progress on Win32 attribute deadlock problem

[Benjamin Ducke]

Dear GRASS friends,

I know it is getting boring, but this problem with attribute data on
Win32 is totally wrecking the show for me, so I took a few more hours of
tedious Win32 (brrrr!) debugging to see what I can find.

Again, I used this data set:

ftp://ftp.rz.uni-kiel.de/pub/ufg/qgis-test.zip

and v.out.ogr to export all points (ca. 3000) in vector map 'dgm' incl.
attributes.
Deadlock occurs randomly on attribute data access, hanging v.out.ogr (or
any (!) other module that accesses attribute data -- I confirmed this at
least for DBF and sqlite tables).

Since debugging under Win32 is a nightmare, I decided to put in lots of
debugging messages by hand to try and locate the problem.

In unaltered form, v.out.ogr managed to export all points before hanging
in one(!) of 50 tries on a Machine with just 176 megs RAM.

In ca. 50% of the cases, hangs occured when Vect_read_line was called
(but there were several other funcs involved, too).

So I removed Vect_read_line from the main program loop and only called
it once before entering the loop to have a valid struct in memory to
work with.

Now, v.out.ogr finished in ca. 50% of the cases. So Vect_read_line must
be involved somehow, but it is not the original source of the problem
(read on).

Next, I reduced vector API calls to only Vect_read_line and
Vect_cat_get. Got rid of all attribute handling code. Now everything
worked fine. Out of 30 attempts, 30 worked. So these two funcs by
themselves don't cause any problems.

Next, I put some attribute handling code back in bit by bit to see when
the trouble would start again. It did not take long. Basically, managing
SQL strings with db_init_string and db_set_string works fine, but as
soon as the attribute database is queried (using db_open_select_cursor),
the lockups come back in full number!

SUMMARY

* The problem does not originate in a specific DB driver (such as
DBF.EXE), but is buried deep in the basic functions if the GRASS DBMI lib.

* The basic Vect_read_line and Vect_cat_get functions work fine on Win32
by themselves. Trouble starts when there is interaction with DBMI calls.

* The function db_open_select_cursor is at least one of the possible
culprits, but chances are that more functions are involved. In any case,
lockups occur more frequently when calls to Vect_read_line are done
often, suggesting that there is some unwholesome interaction or
competition for resources between Vect and DBMI libs.

* For some reason, this problem does not show up in Linux.
There is strong evidence that it is a system resource allocation problem
(mem, file handlers???), because lockups occur less frequently on
systems with more available RAM. The problem exists on both Windows 2000
and Windows XP systems (Vista untested).



For today, I am giving up.
At least now it seems to become clearer where to look for the culprit.
If any of you have any bright ideas, I'd be grateful to hear about them.

Best,

Benjamin

-- 
Benjamin Ducke, M.A.
Archäoinformatik
(Archaeoinformation Science)
Institut für Ur- und Frühgeschichte
(Inst. of Prehistoric and Historic Archaeology)
Christian-Albrechts-Universität zu Kiel
Johanna-Mestorf-Straße 2-6
D 24098 Kiel
Germany

Tel.: ++49 (0)431 880-3378 / -3379
Fax : ++49 (0)431 880-7300
www.uni-kiel.de/ufg