[GRASSLIST:5561] Re: [Fwd: whinging about GRASS again]

[Michael Barton]

Michael,

Thanks for passing this along. I'll try to respond to these issues. It may
be an opportunity to help explain some things to new users.

Perhaps others who have additional perspectives on GIS may want to add some.


> From: Michael Tiemann <tiemann at redhat.com>
> Date: Sat, 29 Jan 2005 20:28:02 -0500
> To: <michael.barton at asu.edu>
> Subject: [Fwd: whinging about GRASS again]
> 
> Here it is.  I think Russ makes many very clear, cogent points.
> 
> M
> ?
> From: Russell Nelson <nelson at crynwr.com>
> Date: Sun, 23 Jan 2005 15:37:28 -0500
> To: <grass5 at grass.itc.it>
> Subject: whinging about GRASS again
> 
> I tried GRASS 5.7 again.  The last time I tried it was almost three
> years ago, in May of 2002.

First, a question. Since you are trying the most current version of GRASS
(something I indeed recommend), why are you testing some version of 5.7 when
6.0.0 beta 1 has been released. Depending on which version of 5.7 you are
looking at, this could make an enormous difference in the functionality of
GRASS.


> I know somewhat more about GIS than I did
> the last time, but apparently not enough.  I'm still off-put by the
> barriers to entry to GRASS use.  I never got over them last time.
> GRASS has been unavailable to me the whole time.  The problem that I
> wanted to solve remains unsolved.

GRASS has been available and being improved in a variety of ways over that
time. However, that won't help limited knowledge of GIS. GRASS has a
reasonably difficult learning curve--less so than some other complex and
powerful analytical tools (e.g., R for statistics)--but it still requires
considerable effort. However, this is the case for the other GIS software
I've used. For that matter, this is the case with GIMP and Photoshop, and
with stat programs including my favorite (because of its UI) JMP. You can
open a file and do stuff with it, but it won't mean anything unless you
understand statistics.

This is not an excuse for a needlessly difficult user experience. Just an
observation that software tools for complex tasks tend to have a
considerable learning curve because it is necessary to learn the concepts
and methods of the analytical system in addition to the nuts and bolts of
using the software. Most people know how to read and write (even if poorly)
so they only need to understand how to work the software for using a word
processor (which has a number of odd UI quirks--from an objective point of
view--that we've simply become used to). Most people do not know anything
about hydrologic modeling, buffering, or projections for spatial data. It is
necessary to learn about these things in addition to simply learning how to
make the software work, making the whole process considerably more
difficult. 

IMHO, GRASS is no harder (nor any easier) to learn than other GIS software
I've encountered--including MapInfo, ArcView/ArcGIS, and Idrisi. This
doesn't yet get at the issues you raise below, but needs to be kept in mind.

> 
> You still need to be an expert to use GRASS.  The GRASS tutorials
> don't help, because the problem is not that people can't learn to use
> GRASS, the problem is that they shouldn't HAVE TO be an expert to use
> it.  The GRASS GUI front-ends don't help, because the problems with
> GRASS are in the middle of GRASS, not at the edges.

In many respects, I agree with the first and last sentence here. What I said
above speaks directly to that. I'm not sure about the second, however. Can
someone make a word processor that can be used by someone who can neither
read nor write? Or a spreadsheet for someone who does not have an
understanding of math?

The critical question is can GIS software be made more useable for people
who understand GIS but are not GIS professionals?

> 
> The trouble with GRASS is that, at its core, its defaults are all
> wrong.
> 
> First, you have to set a projection BEFORE you load any data.  That's
> backwards.  There should be a default projection into which all loaded
> data is projected.  If you load a raster UTM map into a lat/lon
> projection, then it should be reprojected.  Or else perhaps the
> initial projection should be "None", and when you load something, that
> becomes the default projection.
> 

The second sentence is mistaken and I suspect that most people who use GIS
enough to understand the importance of projections would disagree with the
third sentence--regardless of how appealing and commonsensical it might seem
to people who don't use GIS.

Unlike ArcView, for example, you don't 'set' the projection in GRASS at all.
You can't. Furthermore, as far as I know, there is no projection information
stored internally with GRASS files that can be used to indicate their
projection (neither do ArcView shapefiles). This has some advantages and
disadvantages in terms of file storage and manipulation. Nevertheless, it is
critical that projection parameters for all maps that are going to be used
together be the same. GRASS handles this critical necessity by assuming that
all files within a "location" have the same projection
parameters--parameters that the user sets when a location is created.

So is this a good idea? In MapInfo you must set the projection parameters
for a viewing session and you can set a default set in case you don't want
to set them each time. All maps will be automatically reprojected into this
projection as they are displayed. In ArcView, you can do something similar,
but only if you have physically stored the shape file in latlon format (I
think ArcGIS is a bit more flexible in this regard). In both cases, this
only works for vector files, where the computational overhead to transform
the display is modest. In neither case is this done for raster files
(MapInfo still doesn't really work with raster files like ArcView and
GRASS). If you were to do this with raster files, then hundreds of
thousands--perhaps millions--of pixels would have to to recomputed each time
you displayed the map. ArcView doesn't even do raster reprojection; you need
a third part extension to do this--and it takes a long time. This would make
for extremely slow displays even with today's computers. GRASS is heavily
invested in raster GIS where it excels in modeling and image analysis. There
is a much faster response by not doing this kind of reprojection on the fly.
However, GRASS does do automatic interpolation of raster displays to the
current resolution for better display. A more pertinent question, perhaps,
is whether GRASS files should be redesigned to include a metadata header
that includes projection information. This then could be used by the display
module. However, you still need to know what a map projection is to make
sure that multiple maps overlay correctly. In ArcView (which does not embed
projection information in its files) it is possible to overlay a shapefile
stored in UTM NAD 27 over a shapfile stored in UTM NAD 83 and have no way of
knowing that you are doing it--and suffering unknown errors of 100m or more.
You CAN fool GRASS too, but it is harder.

> Second, you have to set a region BEFORE you load any data.  That's
> just stupid.  Who has any idea what your region should be without
> first loading the data.  Whenever you load data (raster or vector), it
> should *set* your region to encompass the entirety of the data that
> you load.  If you DO NOT want to do that (and I can see where you
> might not), THEN you would use an option.

This seems like a very straightforward idea. I noticed that you proposed
submitting some kind of change to the GRASS CVS to implement this. However,
there are some complications here also. First you DON'T have to set the
region before you display data (GRASS and other GIS programs don't "load"
data like spreadsheet, they display GIS files stored on disk. This seems a
small, but critical difference. But see below). When you create a "location"
to store your GIS files, you are asked to define a default region. You don't
HAVE to do this (you can just put 0 and 1 in the extents in fact), but it is
a good idea. This becomes the default 'window on the world' you look through
when you display GIS data. Of course, you can change this window  on the
world (i.e., the region) in many ways--specifying coordinates, making it
match the extents of a map (as you suggest), or by drawing a box with a
mouse. When you next open GRASS, it will automatically use the region you
defined in your last session. MapInfo works this way and it is very handy.

ArcView works the way you suggest. It creates a window on the world that
matches the extents of the first map you display. This is occasionally
handy, but usually annoying while you wait for it to display the entire map
before you can zoom back to the place of interest. Of course you can save a
workspace file with the zoom setting you want. You can save a workspace in
GRASS too. So if we were to switch to the ArcView way of displaying to a map
extents initially I could live with it. But I would miss the GRASS/MapInfo
method. 

A zoom to extents command (g.region vect=[mapname] or g.region
rast=[mapname]) could be issued in the GIS Manager when the first time a map
is listed for display, but I suspect that most users would find that a real
pain. Perhaps a better compromise would be to add a flag to d.vect and
d.rast to have them optionally set the region to the map extents if the flag
is enabled. Then you could have it both ways.

> 
> Third, when you load a dataset (raster or vector), the default should
> be to display it.  Again, there should be an option to not display,
> but the default should be the sensible thing.  It's a principle of
> user interaction that an action should be followed by a response.
> When an action causes no response, people think they did something
> wrong.  If you flip a light switch, and you see no change, you think
> that the light is burned out, or that you flipped the wrong switch, or
> that you didn't actually flip the switch.

As I noted before, GRASS does not load a file, it displays it. And when a
display command is issued the file is displayed immediately. D.vect
[mapname] or d.rast [mapname] will display [mapname] in the current display
window. I suspect what you are thinking of is how maps are listed and
displayed in the GIS Manager (formerly Display Manager) interface. Maps can
be listed and arranged to be displayed; displaying the overlaid set of maps
is a separate button. This is very much like ArcView/ArcGIS that "loads"
maps for display with the display checkbox unchecked. When you check it, the
maps are displayed. The GIS Manager could easily redisplay the maps in the
list every time a new map is added. Of course, this slows things down a lot
when you have a complex set of maps to display--and redisplay each time you
add one to the set or rearrange them. On the other hand, it would be nicer
for new users with one or a very few maps. The display check boxes could
probably initiate the display, as they do in ArcView if others want that.
However, it is often nice also to be able to arrange maps before displaying.

The thing that is very annoying about displaying maps in GRASS is the need
to have a display window open prior to displaying a map. As I understand it,
this is a result of using the somewhat dated xdriver for display. The TclTk
GIS Manager gets around this by issuing a d.mon command prior to the display
commands to make sure there is a window to display in. If you use d.vect or
d.rast directly however, you need to have a window open AND selected. It is
nice to have the option of multiple display windows, but having to
explicitly open and select a window before displaying is a pain. Perhaps
d.vect and d.rast could be modified to check to see if a window is open and
selected, and open one if not.

Before going on, I'd like to suggest thinking a bit outside the box. We have
all become accustomed to an interface of windows, mouse activated pull-down
menus, and dialog boxes to set options. In this sense, GRASS is very
standard and this probably helps the learning curve at the very beginning.
By now, everyone using a computer knows what to do with a pull-down menu
even if they don't know what the menu does. However, an interface initially
designed commercially to manage Macintosh files, word processors, and
spreadsheets may not be the best one to manage complex spatial data. Is
there a better way? I don't know. What about something that has thumbnails
of maps that you can stack, rearrange and group? Maybe you could even form
the map thumbnails into equations to do map algebra. Then you could resize
the stacks or equation results dynamically to see it. Maybe you could
resize them in 3 dimensions if you wanted to do something NVIZ like. Anyway
enough for this aside.

> 
> Fourth, the data organization of GRASS is completely non-Unixy.  It's
> incredibly clumsy to try to use ordinary Unix commands to manipulate
> GRASS files.  Yes, GRASS has its own file manipulations commands, but
> that's just wrong.

The GRASS file manipulation commands and very useful for a variety of
reasons, but I agree with your first sentence.

> Unix has subdirectories into which a set of
> related files can be placed.  The GRASS organization of putting
> related files into different folders organized by the type of the file
> is wrong.  It's just wrong.  Different file types have different file
> extensions.  foo.c, foo.o, and foo are how you organize a program; not
> src/foo.c, object/foo.o, and executable/foo.

This is the way that vectors ARE organized in GRASS 5.7 on up. Rasters are
still organized the old way. A few months ago, Markus Neteler suggested
re-organizing raster structure along the same lines. I think it is a GREAT
idea. However, many others were reluctant to do so because of backward
compatibility with large investment in GIS data. However, if you think this
is bad, you should see ESRI grid files. At least with GRASS you can actually
move the files using OS level file management commands. This is not possible
with ESRI grid files, with part of the information embedded in a single
/info directories and the rest distributed among various grid directories.

> 
> Fifty, nobody runs GRASS on a shared machine anymore.  They just
> don't.  If you want to work on something, it's going to be in a
> subdirectory.  Introducing unfamiliar ideas like "database",
> "location", and "mapset" (I'm just reading these off the very first
> GRASS screen) is not a way to endear yourself to someone new to GRASS.

Like 'coverage', 'theme', and 'view' are any more transparent terminology???
Actually research teams DO exist and DO access GRASS spatial data from
shared data servers. Interdisciplinary research teams are being STRONGLY
encouraged by national funding for the sciences at present. GRASS is the
only GIS with a data structure that is set up to work with teams. ArcView is
a nightmare if you want to share data. Beyond that, the location/mapset
structure is the way that GRASS handles projections (as discussed above) and
these names seem pretty straightforward. The "database" or even worse
"GISDBASE" could perhaps be better renamed the 'GRASS data directory'

> 
> Sixth, the command options of GRASS are completely non-Unixy.  If I'm
> going to load a raster using g.in.gdal, I should be able to say
> "g.in.gdal filename".  Since the current set of parameters are all
> tagged parameters, it should be a no-brainer to add positional
> parameters.  

Many (most?) GRASS commands with simple arguments DO work this way. You can
use d.vect [mapname] or d.vect map=[mapname] to display a map. R.in.gdal
requires at least input and output map arguments, and can take quite a few
more. I don't know if it will take r.in.gdal [inmap] [outmap], but it might.

> And while you're at it, change "foo=bar" to "-foo bar",
> please.  If you don't have single-character options, you don't need
> "--" for options.  A single dash is fine.

GRASS started out as a UNIX program, but is now designed to run on may
platforms--more than any other GIS. As a Mac user (who does use the
terminal), I find foo=bar easier to follow than -foo bar; I think most other
new users would too. Note that it is the same amount of typing. Also, GRASS
uses the dash for flags, which is pretty unixy.

v.command foo=bar -x

> 
> Seventh, the command prompt of GRASS is all TOO Unixy.

Huh??

> If you type
> "help", you get the shell's help.

If you type a command with the -h switch, you get the command usage.
If you type a command without arguments, you get an autogenerated GUI
dialog. There is a help button on each of these that takes you to a nice
html page of the manual for the command.
If you type g.manual you'll get an html formatted help for the entire
program opened in your current browser.

This is pretty flexible, combining both unix-like help and features more
like the online help of commercial programs.

However, we could really use people to help update and improve these manual
pages, and other GRASS documentation like the tutorial and FAQ. This is
something even non-programmers can do.

> The concept of GRASS is that you'll
> use shell scripts to automate GRASS.  That's silly.  Nobody writes
> shell scripts anymore unless they really really have to.

Actually I like to do shell scripts. Perhaps I'm silly, but they work very
well. For people without much programming experience, they are a GREAT way
to customize GRASS. It is easy to make a reasonably nice UI from shell
scripts in GRASS too.

> It would be
> much much easier for people to learn enough Python to automate GRASS,
> and present GRASS to them as Python modules.

My understanding is that you CAN use GRASS from Python and Perl. So you have
a choice of scripting environments

> It would work like this:
> 
> from grass import r, d, v
> 
> display = d.monitor()
> map = r.in.gdal("stlawrencecounty.jpg")
> schooldistricts = v.in.ogr("schooldistricts.shp")
> display.add(map)
> display.add(schooldistricts)
> 
> Then you'd see, on the display, a raster map of St. Lawrence County,
> overlayed by a set of vectors which are the school districts.
> 

Uh it is pretty easy to do this using GRASS commands in a shell script, Perl
script, or Python script. The following 5 commands (same number as you have)
in script run from within GRASS would do exactly what you want.

r.in.gdal input=stlawrencecounty.jpg output=map
v.in.ogr dsn=[path to schooldistricts.shp] output=schooldistricts/
layer=schooldistricts
d.mon x0
d.rast map
d.vect schooldistricts

If you want to run this outside the GRASS environment, you can access the
grass libraries to do so.

It's late and I guess I'll leave the rest of the more ranty comments below
for others to answer. A lot of the GRASS structure and organization is there
for good reasons. However, other aspects have just been inherited and may
even be problematic. GRASS can definitely be improved. The changes from 5.0
to 6.0 are substantial and show the potential of the software. However, this
has been done with a very small community of very dedicated developers. To
have more improvements requires more user contribution to GRASS development.
This can empower users to help create better software that they use on a
daily basis. This is why I became involved a year ago. You are doing the
right thing by offering to help making improvements to the program. However,
you also need to get sufficiently involved in the project and learn enough
GIS in order to suggest changes that can both benefit the existing user
community and encourage new users as well.

Michael

> Finally, I (obviously) think that GRASS needs a lot of changes.  I
> realize that every GRASS user has invested substantial time in
> learning how to use GRASS.  These users will resist these changes.
> Probably most GRASS developers are, in the grand tradition of open
> source, first and foremost GRASS users.  So my hopes of having my
> suggestions accepted are low.  The last time I made similar
> suggestions several years ago, instead of people saying "yes, those
> are good suggestions", people said "if that's all you want to do,
> GRASS is the wrong package".  I agree!  If all you want to do is use
> GRASS, GRASS is the wrong package.
> 
> I think that, for GRASS to achieve a larger user base (and
> consequently larger developer base), it needs changes which will piss
> off all current grass users.  GRASS needs to be completely refactored
> into a completely different kind of GIS system.  I think that, instead
> of trying to improve GRASS, it would be better to treat GRASS as the
> ruins of a GIS package, to be scavenged over for useful bits of code.
> In that manner, GRASS development could continue unimpeded, while the
> (insert name here) package could move forward without having to worry
> about backwards compatibility with current user training.
> 
> Am I completely kooky here?  Am I saying anything obviously stupid?
> Is there some technical requirement for GRASS to be hard to use that I
> fail to comprehend?  Does anybody else feel the same way that I do?
> Or have they all left to go work on QGIS and Thuban?  I recognize that
> I'm suggesting huge amounts of work, but the reward is large as well.
> 
> Imagine: an open source GIS package that was both capable AND usable!
> 
> -- 
> --My blog is at angry-economist.russnelson.com  | Freedom means allowing
> Crynwr sells support for free software  | PGPok | people to do things the
> 521 Pleasant Valley Rd. | +1 315-323-1241 cell  | majority thinks are
> Potsdam, NY 13676-3213  | +1 212-202-2318 VOIP  | stupid, e.g. take drugs.


____________________
C. Michael Barton, Professor of Anthropology
School of Human Evolution and Social Change
PO Box 872402
Arizona State University
Tempe, AZ  85287-2402
USA

Phone: 480-965-6262
Fax: 480-965-7671
www: <www.public.asu.edu/~cmbarton>