[Qgis-user] Sextante on Mac OS X: the solution for running R (problem of shell = True with Python subprocess on Unix systems)

[Martin Laloux]

I could not use R commands on Mac OS X, so, like with GRASS, I decided to
analyze the Python scripts and the sextante log to understand what happens

This time, no error in the log but no results either, no resulting
shapefile in /Users/martinlaloux/sextante/tempdata. The temporary script
/Users/martinlaloux/sextante/*sextante_script.r* has been created  so the
problem is again in the parameters of the Python module subprocess in *
Rutils.py* (line 58):

command = ["R", "CMD","BATCH", "--vanilla", RUtils.getRScriptFilename(),
RUtils.getConsoleOutputFilename()]
proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE,
stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True)

Usually when I want to run an R script in Python, I use the command (see
"QGIS: run Python scripts from the Python Console  or with Script Runner
(without creating an extension)" (in french  QGIS : lancer des scripts
Python ou des commandes Shell depuis la Console Python ou avec Script
Runner
<http://www.portailsig.org/content/qgis-lancer-des-scripts-python-ou-des-commandes-shell-depuis-la-console-python-ou-avec-scrip>
):<http://www.portailsig.org/content/qgis-lancer-des-scripts-python-ou-des-commandes-shell-depuis-la-console-python-ou-avec-scrip>

subprocess.Popen(['Rscript','sextante_script.r'],stdout=subprocess.PIPE))
-> *works*

*But what about the solution of the script ?*
If I try to run the procedure directly in the Python shell (with
sextante_script.r obtained by the script):

>>> import subprocess
>>> doc=subprocess.Popen(['R', 'CMD', 'BATCH',
'sextante_script.r'],stdout=subprocess.PIPE) -> *works*
>>> doc=subprocess.Popen(['R', 'CMD', 'BATCH',
'sextante_script.r'],stdout=subprocess.PIPE,
stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True) ->
*works*
>>> doc=subprocess.Popen(['R', 'CMD', 'BATCH', 'sextante_script.r'], shell=
True, stdout=subprocess.PIPE,
stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True) ->
*no result *

*Why ? *
*
The problem is shell= True* *with Unix systems. *I have already
encountered this
problem many times in my work
**A good explanation is given in the stackoverflow.com question: Getting
output from Python script in Python
tests<http://stackoverflow.com/questions/9095733/getting-output-from-python-script-in-python-tests/9096104#9096104>

* "If you're using shell=True, don't pass the program and its arguments as
a list*"  *but whith a string*

- replacing the line 58 of Rutil.py by

     *command = "R CMD BATCH --vanilla " + RUtils.getRScriptFilename() + "
"+ RUtils.getConsoleOutputFilename()*
     proc = subprocess.Popen(command, shell=True, stdout=subprocess.PIPE,
stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True)
*
works because the result of command is a string*

- or
      command = ["R", "CMD","BATCH", "--vanilla",
RUtils.getRScriptFilename(), RUtils.getConsoleOutputFilename()]
      *proc = subprocess.Popen(command, stdout=subprocess.PIPE,
stdin=subprocess.PIPE,stderr=subprocess.STDOUT, universal_newlines=True)*

*works also because shell=false.*

The complete explanation is (from the question of stackoverflow):

> *On Unix, with shell=True: If args is a string, it specifies the command
> string to execute through the shell. This means that the string must be
> formatted exactly as it would be when typed at the shell prompt. This
> includes, for example, quoting or backslash escaping filenames with spaces
> in them. If args is a sequence, the first item specifies the command
> string, and any additional items will be treated as additional arguments to
> the shell itself.*
>
> I hope you understand why the original script does not work on Mac OS X, a Unix
system. By modifying line 58, no more problem
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/qgis-user/attachments/20120712/584beca6/attachment.html>