[GRASS-dev] python scripts and standard input

[Massimo Di Stefano]

Hi Glynn,

thanks for your help!

actually i'm using :

out = subprocess.Popen(['m.proj', '-o'], stdout=subprocess.PIPE,  stdin=subprocess.PIPE).communicate("%s %s" % (x,y))[0]


reading the code you point me, looks similar.


for now i only need to convert to lon-lat the point-position "picked" by the mouse on the map-canvas 
(it is a short action so i can avoid to use thread).
in the next days i have to write a v.in.asci based code that read data from standard-input (a serial port connection) ... the thread problem will come.


Glynn, List,

This topic, maybe is good to discuss a problem i'm having about subprocess,
please apologize me if it is OT, tell me to open a new thread.

The error i'm having is strictly connected with subprocess and how grass handle it.
The weird is that the error i'm having comes up only on OSX while on linux all works fine.

this the error log  :

GRASS 6.5.svn (spearfish60):~ > python2.6 /Users/sasha/Desktop/OssimPlanetSasha/try.py 
Traceback (most recent call last):
  File "/Users/sasha/Desktop/OssimPlanetSasha/try.py", line 1765, in <module>
    p.init()
  File "/Users/sasha/Desktop/OssimPlanetSasha/try.py", line 75, in init
    self.DataW = DataWork()
  File "/Users/sasha/Desktop/OssimPlanetSasha/datatools.py", line 37, in __init__
    self.vector = self.VectorList()
  File "/Users/sasha/Desktop/OssimPlanetSasha/datatools.py", line 543, in VectorList
    v = list_strings('vect')
  File "/Applications/GRASS-6.5.app/Contents/MacOS/etc/python/grass/script/core.py", line 636, in list_strings
    return ["%s@%s" % pair for pair in list_pairs(type)]
  File "/Applications/GRASS-6.5.app/Contents/MacOS/etc/python/grass/script/core.py", line 621, in list_pairs
    for mapset, maps in list_grouped(type).iteritems()])
  File "/Applications/GRASS-6.5.app/Contents/MacOS/etc/python/grass/script/core.py", line 562, in list_grouped
    for line in read_command("g.list", type = type).splitlines():
  File "/Applications/GRASS-6.5.app/Contents/MacOS/etc/python/grass/script/core.py", line 210, in read_command
    return ps.communicate()[0]
  File "/System/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/subprocess.py", line 663, in communicate
    stdout = self.stdout.read()
IOError: [Errno 4] Interrupted system call
GRASS 6.5.svn (spearfish60):~ > 


the line that generate the error is : 

v = list_strings('vect')

if i comment out all the related code, the error comes up in other lines that use the grass-python code.

like :

s = read_command("g.region", flags='l')

or 

units = read_command("g.proj", flags='p')

it is a "random error" :-(


important note :
the error come up "ever" if i have Ossimplanet already open and running 
if i quit it and try again, the % of the error are drastically reduced to 1 : 10 , (every 10 times i start the application , i have 1 error)

the error easly comes up also when more application are opened on my laptop and more intense  is the cpu usage.

Sinverly i have no clue,

probably it can depends by  a (my) bad programming skill.

PlanetSasha hat too many lines and files .. so i is hard to be used as debug code
i'll try to set up a little script to replicate the problem.


thanks!

Massimo.



Il giorno 28/apr/2010, alle ore 23.16, Glynn Clements ha scritto:

> 
> Massimo Di Stefano wrote:
> 
>> i'm tring to use grass modules that needs a "standard input" method,
>> 
>> like "m.proj"
>> 
>> here an example where i'musing a "file" as input :
>> 
>> coords = read_command('m.proj', input=coordsfile, flags='i') 
>> 
>> 
>> how to use standard input, like in bash :
>> 
>> echo '599537.80 4920995.38' | m.proj -o
>> 
>> 
>> there is a python way to do it, or i need to use "system call" like
>> os.system or subprocess ?
> 
> If you want to read from the child's stdout, there's pipe_command()
> and read_command(). If you want to write to the child's stdin, there's
> feed_command() and write_command().
> 
> If you want to do both, you need to do it yourself using
> start_command() (on which the other functions are based), e.g.:
> 
> import grass.script as grass
> 
> def read_write_command(*args, **kwargs):
>    input = kwargs['stdin']
>    kwargs['stdout'] = grass.PIPE
>    kwargs['stdin'] = grass.PIPE
>    ps = grass.start_command(*args, **kwargs)
>    return ps.communicate(input)[0]
> 
> This behaves like a combination of read_command() and write_command(). 
> The input string is passed via the "stdin" parameter as for
> write_command(), while the data from the child's stdout is returned as
> for read_command().
> 
> If you want a pipe_command/feed_command interface with file handles,
> it gets problematic. You need to use non-blocking I/O or multiple
> threads; otherwise, you can get deadlock, with the child blocked
> waiting for the parent to read the child's stdout while the parent is
> blocked writing to the child's stdin.
> 
> If you really need this, look at the implementation of
> subprocess.communicate(); the Unix version uses select() while Windows
> uses threads.
> 
> Alternatively, redirect the child's stdout to a file and have the
> parent read the file; writing to a file never blocks.
> 
> BTW, if you need even lower-level access, use grass.Popen() or
> grass.call(). These are almost identical to subprocess.Popen() and
> subprocess.call() except that, on Windows, they set shell=True by
> default. This allows commands to be either scripts or binary
> executables, and doesn't require the extension; with shell=False, the
> command must be a binary executable and must include the ".exe"
> suffix.
> 
> -- 
> Glynn Clements <glynn at gclements.plus.com>