[GRASS5] GRASS 5.3 new bug in s.in.ascii

[Michael Barton]

I tried this on a different computer, but still a Mac OSX machine 
running the same version of GRASS 5.3. I get the same results. Note: I 
have used both nedit (xwindows) and bbedit (mac) to save the file--in 
unix text format.

I am using a 13-02-2004 build of GRASS 5.3. I guess I will try updating 
to a newer verision as soon as I can and test again--though this 
shouldn't make any difference.

Michael Barton



On Monday, March 22, 2004, at 11:12  PM, Hamish wrote:

>>> I have hit a bug in GRASS 5.3--the first I've run into in a couple
>>> months of pretty solid work with this version.
>>>
>>> When trying to import a simple text file into the sites format, I
>>> get an error if the file contains non-numeric strings. Here is the
>>> file I was trying to import
>>>
>>> 501175.0877193,3836254.44444444,1968,room
>>> 502782.10526316,3839709.53216374,1915,room
>>> 502621.40350877,3842789.64912281,1889,room
>>> 503183.85964912,3844798.42105263,1874,room
>>> 501175.0877193,3842602.16374269,1894,roomblock
>>> 506505.02923977,3837566.84210526,1941,roomblock
>>>
>>> s.in.ascii sites=clearcreek_sites input=clearcreek_sites.txt fs=,
>
>
> I can import that file without any problem.
> GRASS53: > s.in.ascii in=barton_test.txt sites=barton_test fs=,
>
> result:
> name|barton_test
> desc|s.in.ascii sites=barton_test input=barton_test.txt fs=
> 501175.0877193|3836254.44444444|#1 %1968 @room
> 502782.10526316|3839709.53216374|#2 %1915 @room
> 502621.40350877|3842789.64912281|#3 %1889 @room
> 503183.85964912|3844798.42105263|#4 %1874 @room
> 501175.0877193|3842602.16374269|#5 %1894 @roomblock
> 506505.02923977|3837566.84210526|#6 %1941 @roomblock
>
>
> [Note you can just create ascii files with the correct format and put
> them in the $MAPSET/site_lists/ directory as a last resort..]
>
>
> I see the "desc|... fs=" has lost its "," .. need to fix that.
>
>
>>> It doesn't matter whether or not I put quotes around the character
>>> strings. The error I get is:
>>>
>>> *** malloc[14292]: error for object 0x357b: Pointer being
>>> reallocated was not allocated
>>> ERROR: G_realloc: out of memory
>>>
>>> If I get rid of the string fields (i.e., 'room' and 'roomblock'), it
>>> imports just fine.
>
> Can you make this fail with a clean & up to date install of GRASS
> 5.0, 5.3, and 5.7? All have different versions now (as of today).
>
>
>>> I'd SWEAR this was working just fine in a previous version of
>>> s.in.ascii I was using last November. I didn't notice anything had
>>> been changed in the CVS for s.in.ascii in the past few months so I
>>> don't understand why this WAS working but isn't now.
>
> Can you reproduce it on another computer? Is it only on 5.7?
>
>
>> One possibility is changes to src/libes/gis/sites.c.
>
> The 5.0/5.3 version hasn't been touched in 22 months.
> The 5.7 version had a change to a memory call 2 months ago.. -> !!??
>   see grass51/lib/sites/sites.c  G_sites_close()
>
>
>> Another possibility is that the bug has always been present, but you
>> haven't been bitten by it before; bugs relating to malloc()ed memory
>> are often like that.
>
>
> Upon doing some updates to s.in.ascii yesterday (see below) I came
> across something, but didn't figure it out fully. It's a bit of a grey
> area but I think it fails in a bad way (assigning data to sites which
> don't have any). It isn't seen with uniform data coverage, which is the
> usual case.
>
>
> Take this input file:
> 2649375 6503980
> 2682513 5996973 5
> 2473745 5735185
> 2308057 5483339 text
> 2155623 5413749
> 2496942 6036738 2 word
> 2609610 6235565
> 2649375 6503980 two text
> 2682513 5996973
> 2473745 5735185 2.34 5.67
> 2308057 5483339
> 2155623 5413749
>
> s.in.ascii produces:
>
> name|test
> desc|s.in.ascii sites=test fs=space
> 2649375|6503980|#1
> 2682513|5996973|#2 %5
> 2473745|5735185|#3 %5
> 2308057|5483339|#4 %5 @text
> 2155623|5413749|#5 %5 @text
> 2496942|6036738|#6 %2 @word
> 2609610|6235565|#7 %2 @word
> 2649375|6503980|#8 %2 @two @text
> 2682513|5996973|#9 %2 @two @text
> 2473745|5735185|#10 %2.34 %5.67 @two @text
> 2308057|5483339|#11 %2.34 %5.67 @two @text
> 2155623|5413749|#12 %2.34 %5.67 @two @text
>
>
> From memory this is a long standing problem. As it can assign data
> values to sites which don't have them, I'd throw this in the "bad" bug
> box as it will silently give you the wrong answer.
>
> In the past I've used @" " as placeholders for no-data records to get
> around this. Note that most site modules will be confused by mixed
> records like these; really I guess they should check against the "form"
> header field but nothing does.
>
> get_site.c only calls G_site_new_struct() once, after that the 
> structure
> is never freed or blanked, just overwritten. Thus site->dbl_alloc and
> site->str_alloc go up but never down. Maybe ok (or even desirable) for
> this to happen, but the values should be reset to NaN's or blank spaces
> at least.
>
> I tried freeing it in main.c after G_site_put_new() is called, and
> having get_site() call G_site_new_struct() each time it is run, but 
> that
> lead to a segfault when a string att was used. Also tried a few other
> things but no luck -- but I'm not very good with memory issues so the
> problem isn't necessarily a difficult one. Maybe someone else can see
> it more clearly.
>
>
> -==--==-
>
>
> Updates (I'll add to 5.7 in the next day or so):
>
> * s.in.ascii should now deal with Mac OS9 and DOS text files without
> cryptic error messages. Mac OS9 files are rejected while DOS files are
> recovered with a warning message.
>
> * now you can feed it x,y positions without a label or numeric 
> attribute
> and it will still work.  (bug #1721)
>
> Please test!!
>
>
> Hamish
>
______________________________
Michael Barton, Professor & Curator
Department of Anthropology
Arizona State University
Tempe, AZ  85287-2402
USA

voice: 480-965-6262; fax: 480-965-7671