[GRASS5] New IPC XDRIVER vs. fifo vs. UNIX Sockets???

Andreas Lange Andreas.Lange at Rhein-Main.de
Thu Nov 16 16:16:07 EST 2000


Hi Markus, Eric, all,

as i got the impression from personal mails that there is still some
confusion about the XDRIVER, FIFOs, Sockets and IPC message queues i'll
try to explain what i found out.
Please be warned that i am not an expert on this and am not very much
interested in system programming.

The system is right now as follows:
All commands creating graphical output (d.rast, d.vect, d.sites etc.)
use functions from the Raster library. The output of these calls are fed
into a FIFO and read out by the DRIVER (XDRIVER for X Window output,
CELL driver for CELL output etc.). The driver sits in the background and
waits for data coming in the FIFO and renders this output to the monitor
or to a file. The d.mon start command starts up this driver and does
some checks for locks, already running drivers etc. 
Writing a new driver (see in the programming manual) is done by breaking
out the files of the device driver lib and adapt only some output
routines to the new driver. The driver consists then of the code to
start/stop, connect with the FIFOs and render the output. All graphics
commands not directly provided by the device driver are done with
functions from the device driver library (e. g. if you only provide a
function for drawing lines all other graphics output is done with this
graphical primitive). 

FIFOs have the big advantage that they are an early invention in the
history of unix systems and that they are standardized with POSIX.1. So
nearly every unix has FIFOs.
Cygwin claims POSIX.1 compatibility, with just two system calls missing,
one is mkfifo() for the FIFOs. The problem with the FIFO implementation
within Cygwin is that it is impossible for architectural reasons to
create the server part of the FIFOs with Win9x/ME. I' am not shure if
there is a FIFO implementation for Win NT, but the UWIN port (that sadly
is not free in the GPL's sense) has the FIFOs. But excluding the
Win9x/ME users is IMHO not acceptable.  

The "new" IPC message queue based driver was written in 1994 by Ronald
Wiener, mainly because the message queues are faster on some systems (or
have been faster around 1994!). My tests show that FIFOs and IPC message
queues are the same on IRIX, but on Linux the message queues are faster.
Any Solaris testing?
The IPC (interprocess communication) comes from System V and is therefor
not implemented in some BSD-descendend/derived systems (such as Mac OS
X). So message queues will _not_ enhance portability. One other problem
with the message queues is that if the process crashes or dies without
closing/reading the contents of the message queue (or calling
msgrcv/msgctl), the contents and the message queue structure is not
released until reboot or until removed manually with the ipcrm command. 

With the FIFOs the fifo file is released if the associated process is
killed or exits. 

If we decide to switch from the FIFO setup to a IPC message queue based
system or a Unix sockets based system we will have to change the part of
the library that does the writing to the driver from the raster lib
(io.c) and all drivers that read from the FIFO/msgq/socket and we'll
have to implement a new locking scheme for sockets as Eric described. So
we will not get rid of the XDRIVER or the CELL driver, just change them. 

I think that the windows port is very important for the acceptance of
GRASS. But on the other hand i think that it would be bad to change
GRASS to something that causes much work and a lot of trouble _only_
because Cygwin is missing something. At least we should check that there
are no known problems/deficiencies with the Cygwin sockets or message
queues implementation. The outdated FAQ says: "... the network support
in Cygwin is supposed to provide the Unix API, not the Winsock API. ...
Because of this, cygwin.dll does a lot of nasty stuff behind the scenes,
trying to persuade various winsock functions to do what a Unix select
would do.". 

If _only_ graphics output for a windows port is needed, it should be
possible to use something completely different on Win32 than on Unix
(even without the need for a XFree Server). The old tcltkgrass code in
unused/tcltkgrass could be a starting point. The tk superview object can
display grass raster, vector and site data, but is not in working
condition (needs to be updated to the new 5.0 API, dumps core etc.). I
remember that the commercial GRASS for Windows uses something like this.
But compiling a tk8.x wish shell under Windows requires a MS compiler.  


So i can not advise anything, but i will participate in whatever the
outcome of the discussion will be. I just don't want that we wind up
with something that works only for some OSs or causes too much trouble.
We need to consult an windows expert first. And we should do some
prelimnary tests first to check the speed of a sockets driver. 


Andreas

Markus Neteler wrote:
> 
> Hi Eric, hi all,
> 
> On Sun, Nov 12, 2000 at 09:47:13PM -0800, Eric G . Miller wrote:
> > As we look to implement this new IPC messaging XDRIVER, it's come up
> > that some *nix variants may not support SYSV message queues.  That's
> > kind of a problem since the whole idea is to improve compatibility and
> > possibly performance.  Unfortunately, the FIFOs don't work under CygWin
> > (correct ?).  So, a third alternative is using sockets.
> 
> Up to now the "new" IPC messaging XDRIVER has not been announced
> here, it was somewhat unofficial. Huidae Cho has implemented the
> IPC messaging XDRIVER from Ronald Wiemer (from 1994, developed for
> GRASS4.1). Myself I was misleaded and thought it would be a sockets driver,
> therefore former mails might have been confusing.
> 
> CygWin does not support fifos, therefore it is very important to get rid of
> the fifo XDRIVER/CELL driver etc.
> 
> The current IPC messaging XDRIVER is somewhat working, but has severe bugs
> which would have been to be fixed. But some platforms might not support
> message queues (like Macintosh?). So we should not concentrate on it.
> 
> > I've been rummaging around and noted that:
> > a) CygWin apparently supports UNIX sockets (presumably both AF_INET and
> >    AF_LOCAL).
> > b) Virtually all *nix's support local UNIX sockets (is this not true?).
> > c) XFree86 4.0.1 runs under CygWin now (haven't looked at details).
> 
> So it seems that sockets would be the only option to be platform
> independent. Maybe the new IPC XDRIVER could be again modified to sockets,
> at least we know, where changes are required.
> 
> Eric already tried that:
> > So, I've been playing with updating parts of necessary code to use local
> > unix sockets (AF_LOCAL, sockaddr_un).  Anyway, I'm no expert in this
> > area, but think this could be a viable solution (if it's decided that
> > portability for the IPC mechanism is a problem).  However, several
> > bits of code would have to change, because:
> >
> > a) The whole fifo thing would have to disappear.
> > b) The socket file must not exist before bind() is called.  This
> >    would necessitate a change in many of the semantics for testing
> >    whether a monitor is running or not.
> > c) Only one file descriptor is needed.
> > d) Need to be able to pass around a 'struct sockaddr_un'.
> > e) There'd need to be a different mechanism for identifying the
> >    socket file associated with the monitors (perhaps some different
> >    environment variables?).
> > f) Other things I haven't thought of or come across...
> >
> > I've focused on local sockets only because it reduces some of the
> > security checks that are necessary.  Anyway, I know some of you problem
> > have more experience with this stuff than I, so I'd appreciate any
> > comments.
> 
> Me, too :-)
> 
> If the others agree to change the XDRIVER to sockets, we should go for it.
> In my opinion a Windows port is very important for the reputation of GRASS.
> And it announced such a long time... We should finish it: "just" the XDRIVER
> is missing, the modules are already working on CygWin. But a GIS without
> graphical output is not much fun.

-- 
Andreas Lange, 65187 Wiesbaden, Germany, Tel. +49 611 807850
Andreas.Lange at Rhein-Main.de - A.C.Lange at GMX.net



---------------------------------------- 
If you want to unsubscribe from GRASS Development Team mailing list write to:
minordomo at geog.uni-hannover.de with
subject 'unsubscribe grass5'



More information about the grass-dev mailing list