[R-spatial-devel] [STATSGRASS] speed of GRASS/R interface

[Roger Bivand]

On Mon, 8 Oct 2007, Carlos "Guâno" Grohmann wrote:

> Hi,
>
> I just downloaded/compiled/installed the current sourceforge CVS for
> sp and spgrass6, and it's WAY faster than before.

Thanks, I'll back out of the spgrass6 changes, which are in sp too. I see 
very big speed-ups in rgdal too, and have R CMD check'ed the released 
gstat for problems. I have no idea why the methods recursed into the new 
SpatialGrid object, but they obviously did.

Roger

>
> thanks, Roger!
>
> Carlos
>
>
> On 10/6/07, Roger Bivand <Roger.Bivand at nhh.no> wrote:
>> Hi,
>>
>> Modification also committed in sp on sourceforge, speedup from from ~ 12
>> minutes to 6.5 seconds on a 900 by 3600 raster, and no memory explosion.
>> I'm seeing similar speedups in rgdal. Still no idea why - a change in the
>> methods package? When we release a fresh sp, the need for special fixes
>> will go away, I hope.
>>
>> Could somebody try the current sourceforge CVS of sp and see if you see
>> the same speedup? May need to wait for the anonymous CVS to synch.
>>
>> Best wishes,
>>
>> Roger
>>
>>
>> On Sat, 6 Oct 2007, Roger Bivand wrote:
>>
>>> On Sat, 6 Oct 2007, jarekj at amu.edu.pl wrote:
>>>
>>>>
>>>>  Roger!
>>>>
>>>>  I also notice that of Carlos wrote.
>>>>
>>>>  That hangs not occured when spgrass6 was based on r.out.ascii instead of
>>>>  binary input-output and in fact it was realably faster. Maybe back to
>>>>  previouis solution be good temporal solution. I also exchange data
>>>>  between grass and R very often so that hang make me sometimes inpatient
>>>>
>>>>  greetengs and regards
>>>>  Jarek
>>>
>>> The problem is in new("SpatialGrid", SPix):
>>>
>>> library(sp)
>>> sessionInfo()
>>> cellcentre.offset <- c(-16.821667, 13.060307)
>>> cellsize <- c(0.000833, 0.000833)
>>> cells.dim <- c(3629.000000, 921.00000)
>>> grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
>>> pts <- sp:::boguspoints(grd)
>>> pts at bbox[,1] = pts at bbox[,1] - 0.5 * grd at cellsize
>>> pts at bbox[,2] = pts at bbox[,2] + 0.5 * grd at cellsize
>>> proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
>>>  "+no_defs +towgs84=0.000,0.000,0.000"))
>>> SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
>>> SG <- new("SpatialGrid", SPix)
>>>
>>> with rapidly increasing memory use. I have no idea why. My system:
>>>
>>>>  sessionInfo()
>>> R version 2.6.0 (2007-10-03)
>>> i686-pc-linux-gnu
>>>
>>> locale:
>>> LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C
>>>
>>> attached base packages:
>>> [1] stats     graphics  grDevices utils     datasets  methods   base
>>>
>>> other attached packages:
>>> [1] sp_0.9-14
>>>
>>> loaded via a namespace (and not attached):
>>> [1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15
>>>
>>> and the message after Ctrl-C:
>>>
>>> Error in coordinates(coords) :
>>>  error in evaluating the argument 'obj' in selecting a method for function
>>> 'coordinates'
>>> Error in bbox(object) :
>>>   error in evaluating the argument 'obj' in selecting a method for function
>>>   'bbox'
>>>
>>> which makes me suspect that the new() is (now?) going to completely senseless
>>> places. This has the same effect:
>>>
>>> sG <- new("SpatialGrid", bbox=pts at bbox, proj4string=pts at proj4string,
>>>   coords=pts at coords, grid=grd, grid.index=integer(0))
>>>
>>> unfortunately. It also impacts the rgdal route, because its call to make the
>>> output SGDF is the same.
>>>
>>> However, replacing the call to new("SpatialGrid", ...) to one of building an
>>> empty object with just new(), and stuffing the slots manually with slot() <-
>>> reduces import from over 12 minutes to 6 seconds (!). Perhaps the problem is
>>> that S4 classes have got "cleverer" and dislike the definition of SpatialGrid
>>> as only having a "SpatialPixels" representation?
>>>
>>> I have committed the very draft readRAST6() to CVS on sourceforge, but the
>>> real solution needs doing in sp itself (end of the SpatialGrid() function in
>>> SpatialGrid-methods.R).
>>>
>>> I'll carry on playing ...
>>>
>>> Roger
>>>
>>>
>>>>
>>>>
>>>>  6/10/2007, "Roger Bivand" <Roger.Bivand at nhh.no> napisa?/a:
>>>>
>>>>>  On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
>>>>>
>>>>>>  I've been thinking about the time it takes to import a raster from
>>>>>>  GRASS to
>>>>>>  R. I have some rasters with about 2000 rows and 2500 columns.
>>>>>>  importing
>>>>>>  starts fine, creates the header, then goes to 100%, and it hangs there
>>>>>>  for
>>>>>>  about 10 minutes, before get ready for the next command.
>>>>>>  Any way of decrease this waiting time?
>>>>>
>>>>>  I'm looking into this, as far as I can see as of now it is somewhere in
>>>>>  the new("SpatialGridDataFrame", ...) call in
>>>>>  res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
>>>>>  Rprof() output points at unlist() taking 90% of the total run time. I'll
>>>>>  let you know where I get.
>>>>>
>>>>>  Roger
>>>>>
>>>>>>
>>>>>>  cheers
>>>>>>
>>>>>>  Carlos
>>>>>>
>>>>>>
>>>>>
>>>>>  --
>>>>>  Roger Bivand
>>>>>  Economic Geography Section, Department of Economics, Norwegian School of
>>>>>  Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>>>>>  Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
>>>>>  e-mail: Roger.Bivand at nhh.no
>>>>
>>>
>>>
>>
>> --
>> Roger Bivand
>> Economic Geography Section, Department of Economics, Norwegian School of
>> Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
>> e-mail: Roger.Bivand at nhh.no
>>
>
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no