[R-spatial-devel] [STATSGRASS] speed of GRASS/R interface

[Roger Bivand]

Hi,

Modification also committed in sp on sourceforge, speedup from from ~ 12 
minutes to 6.5 seconds on a 900 by 3600 raster, and no memory explosion. 
I'm seeing similar speedups in rgdal. Still no idea why - a change in the 
methods package? When we release a fresh sp, the need for special fixes 
will go away, I hope.

Could somebody try the current sourceforge CVS of sp and see if you see 
the same speedup? May need to wait for the anonymous CVS to synch.

Best wishes,

Roger


On Sat, 6 Oct 2007, Roger Bivand wrote:

> On Sat, 6 Oct 2007, jarekj at amu.edu.pl wrote:
>
>>
>>  Roger!
>>
>>  I also notice that of Carlos wrote.
>>
>>  That hangs not occured when spgrass6 was based on r.out.ascii instead of
>>  binary input-output and in fact it was realably faster. Maybe back to
>>  previouis solution be good temporal solution. I also exchange data
>>  between grass and R very often so that hang make me sometimes inpatient
>>
>>  greetengs and regards
>>  Jarek
>
> The problem is in new("SpatialGrid", SPix):
>
> library(sp)
> sessionInfo()
> cellcentre.offset <- c(-16.821667, 13.060307)
> cellsize <- c(0.000833, 0.000833)
> cells.dim <- c(3629.000000, 921.00000)
> grd <- GridTopology(cellcentre.offset, cellsize, cells.dim)
> pts <- sp:::boguspoints(grd)
> pts at bbox[,1] = pts at bbox[,1] - 0.5 * grd at cellsize
> pts at bbox[,2] = pts at bbox[,2] + 0.5 * grd at cellsize
> proj4string(pts) = CRS(paste("+proj=longlat +a=6378137 +rf=298.257223563",
>  "+no_defs +towgs84=0.000,0.000,0.000"))
> SPix <- new("SpatialPixels", pts, grid = grd, grid.index = integer(0))
> SG <- new("SpatialGrid", SPix)
>
> with rapidly increasing memory use. I have no idea why. My system:
>
>>  sessionInfo()
> R version 2.6.0 (2007-10-03)
> i686-pc-linux-gnu
>
> locale:
> LC_CTYPE=en_GB;LC_NUMERIC=C;LC_TIME=en_GB;LC_COLLATE=en_GB;LC_MONETARY=en_GB;LC_MESSAGES=en_GB;LC_PAPER=en_GB;LC_NAME=C;LC_ADDRESS=C;LC_TELEPHONE=C;LC_MEASUREMENT=en_GB;LC_IDENTIFICATION=C
>
> attached base packages:
> [1] stats     graphics  grDevices utils     datasets  methods   base
>
> other attached packages:
> [1] sp_0.9-14
>
> loaded via a namespace (and not attached):
> [1] grid_2.6.0      lattice_0.16-5  rcompgen_0.1-15
>
> and the message after Ctrl-C:
>
> Error in coordinates(coords) :
>  error in evaluating the argument 'obj' in selecting a method for function 
> 'coordinates'
> Error in bbox(object) :
>   error in evaluating the argument 'obj' in selecting a method for function
>   'bbox'
>
> which makes me suspect that the new() is (now?) going to completely senseless 
> places. This has the same effect:
>
> sG <- new("SpatialGrid", bbox=pts at bbox, proj4string=pts at proj4string,
>   coords=pts at coords, grid=grd, grid.index=integer(0))
>
> unfortunately. It also impacts the rgdal route, because its call to make the 
> output SGDF is the same.
>
> However, replacing the call to new("SpatialGrid", ...) to one of building an 
> empty object with just new(), and stuffing the slots manually with slot() <- 
> reduces import from over 12 minutes to 6 seconds (!). Perhaps the problem is 
> that S4 classes have got "cleverer" and dislike the definition of SpatialGrid 
> as only having a "SpatialPixels" representation?
>
> I have committed the very draft readRAST6() to CVS on sourceforge, but the 
> real solution needs doing in sp itself (end of the SpatialGrid() function in 
> SpatialGrid-methods.R).
>
> I'll carry on playing ...
>
> Roger
>
>
>> 
>>
>>  6/10/2007, "Roger Bivand" <Roger.Bivand at nhh.no> napisa?/a:
>> 
>> >  On Tue, 2 Oct 2007, Carlos "Guâno" Grohmann wrote:
>> > 
>> > >  I've been thinking about the time it takes to import a raster from 
>> > >  GRASS to
>> > >  R. I have some rasters with about 2000 rows and 2500 columns. 
>> > >  importing
>> > >  starts fine, creates the header, then goes to 100%, and it hangs there 
>> > >  for
>> > >  about 10 minutes, before get ready for the next command.
>> > >  Any way of decrease this waiting time?
>> > 
>> >  I'm looking into this, as far as I can see as of now it is somewhere in
>> >  the new("SpatialGridDataFrame", ...) call in
>> >  res <- SpatialGridDataFrame(...) in readBinGrid(). Confusingly, the
>> >  Rprof() output points at unlist() taking 90% of the total run time. I'll
>> >  let you know where I get.
>> > 
>> >  Roger
>> > 
>> > > 
>> > >  cheers
>> > > 
>> > >  Carlos
>> > > 
>> > > 
>> > 
>> >  --
>> >  Roger Bivand
>> >  Economic Geography Section, Department of Economics, Norwegian School of
>> >  Economics and Business Administration, Helleveien 30, N-5045 Bergen,
>> >  Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
>> >  e-mail: Roger.Bivand at nhh.no
>> 
>
>

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no