[GRASSLIST:9150] Re: major data loss bug with i.points!

[Hamish]

> Hi all, I just experienced some VERY weird behavior using grass
> 6.1cvs.

Hi Ian,

I though I'd try and add something to this thread, although I suspect
we'll never really know the cause.


>   The end result is that somehow GRASS erased an entire location 
> (several 100's of MB), without much prompting by me.  Here is how it 
> happened.

As far as I am aware GRASS contains no code at all anywhere in it to
intentionally remove an entire database, location, or mapset. There was
an unfortunate bug in a script that deleted someone's home directory
once when a temporary directory was not created properly ("cd $tmp ; rm
-f ${tmp}*" when the setting of $tmp failed [ouch, terribly sorry about
that]). But nothing like you describe has been reported before AFAIK.

Certainly it is a serious issue whenever hours of work and important
data is lost, at minimum it is a sobering reminder that computers are
complex beasts and random bad things do happen from time to time.

 
> I was using i.points in an xy location when I mistakenly resized the 
> i.points terminal window as I was entering points.  The terminal
> window  looked a little funky (mishapen i.points display), and I was
> presented  with a grass prompt, instead of the normal i.points
> display.  My cursor  no longer responded on the monitor, though it
> still displayed the  typical 'cross' cursor from i.points.

if the terminal was smaller than 80x24 the i.points text may have done
weird things when it tried to draw the ascii "boxes". See bug #1107
  https://intevation.de/rt/webrt?serial_num=1107


> I tried to control-c to kill  i.points, but my cursor would not
> change. Everytime I did this the  grass prompt would repeat itself
> without a carriage return such that it looked like:
> 
> GRASS 6.1.cvs (xy):~ >     GRASS 6.1.cvs (xy):~ >      GRASS 6.1.cvs 
> (xy):~ >
> 
> rather than
> 
> GRASS 6.1.cvs (xy):~ >
> GRASS 6.1.cvs (xy):~ >
> 
> Carriage returns yielded the same behavior.  I decided things looked a
> little screwy, so I typed exit.  Grass seemed to quit in a normal way 
> (D.m disappeared along with the terminal).

I have found that this same kind of terminal corruption has happened to
me from time to time over a ssh connection to another machine on the
network. If a lot of random crap gets sent to the display (e.g. "cat
binaryfile"), one of the million monkey strings gets interpreted as a
command for the terminal program to do something and things get weird.
Closing the terminal (or leaving it open with just "logout" from the
remote machine) gets rid of the problem. This is not a GRASS specific
error, but rather input handling issue in the terminal program (unless
of course a GRASS program spews out binary data to stdout).

The prompt losing newline capability has been pretty common to me,
especially, but not exclusively, when using the RXVT term.

I'm sure others have had the experience of pasting a large text document
to the terminal by mistake too.


> Upon immediately restarting  Grass and firing up i.points again, I
> discovered that I had lost my  target location!  This included ALL
> data in the target location (a few  hundred megabytes, many hours of
> work).  I have searched thoroughly on  my system, and it is all gone
> (not in the trash either and I haven't  emptied it).  My XY location

Stuff only shows up in the trash that you have dragged there. Anything
deleted on the command prompt (MS Windows too) or by the C unlink()
command will not be saved to the trash.


> is still there, but the target location  has disappeared.  The target
> files of the imagery groups all still  point to the lost location. 
> This all happened in a span of 2 minutes.   Out of fear I have not
> attempted to recreate this scenario.

seems like a valid response.

 
> The last commands from my .bash_history file don't reveal much:
> i.points
> fg
> exit
> i.points
> I tried to see if i.points was running in the background after its 
> initial weird behavior.

Since i.points is mentioned, we should probably raise the priority of
fixing this bug, if only as a precaution.

"i.points: error in tmp file handling"
  https://intevation.de/rt/webrt?serial_num=3798

i.points has:
imagery/i.points/main.c:    unlink (tempfile1);
imagery/i.points/main.c:    unlink (tempfile2);
imagery/i.points/main.c:    unlink (cell_list);
imagery/i.points/main.c:    unlink (group_list);
imagery/i.points/main.c:    unlink (vect_list);
imagery/i.points/main.c:    unlink (digit_points);
imagery/i.points/main.c:    unlink (digit_results);

if the tempfile (or other) string got corrupted via this bug, it is
possible (but I'd say very unlucky) that the resultant string would be a
valid and exact pathname pointing to your mapset with the tempfile name
missing, and that is where things went wrong..? The left over garbage
might have corrupted the terminal prompt too?


> I feel pretty comfortable with grass (regular user for 4 years), and 
> this is the first time I have ever experienced anything remotely 
> similar to this.  It definitely scares me as I have a lot of data I 
> don't care to lose.  Anyone have any ideas out there about this?  
> Developers?  Is this something that has happened before with a cvs 
> version?

I think you are the only unlucky one so far.. hopefully the last.

> I have been very happy with GRASS, but a bug like this really makes me
> question my use of the program.  I know one should make regular 
> backups, but this is very bothersome.

I think over 23 years the core code of GRASS has become pretty stable,
unfortunately sometimes new bugs do get inadvertently added and some old
bugs are only triggered under very rare circumstances. It just sucks
when you happen to be the one that finds one...

> Mac OS 10.3.9 (latest update), GRASS 6.1cvs from Lorenzo (source Oct. 
> 22, 2005)

suggestions for all:

fix the above two bugs starting with #3798. Bug 1107 may be a "won't
fix; replace the program's interactive functionality with something
better."

audit lib/init/clean_temp.c and other modules that call unlink(),
scripts that use "rm -f".

An old & important blue-water sailing rule, but I think it applies:
It is best to take action (eg backup your data) when the thought first
crosses you mind.



Hamish