[GRASS-dev] ramdisk as mapset?

"Sören Gebbert" soerengebbert at gmx.de
Thu Jul 19 07:42:14 EDT 2007


Hi Hamish,
interesting idea. But lets put this idea into the library of grass. :)

I would like to suggest to extent the grass lib to
read the entire map into the memory if needed.
G_get_rast_row() will still work with no modification, but will
read the rows from memory and not from the disk.

And while implementing this, a tile caching mechanism would be good to.

IMHO we have to add new special --flags to allow every module to use this feature.
eg: --cache-all will read all tiles (in this case a tile is a row) into the memory.
--cache-size-y=<int> will set the number of rows which should be in the cache.

And in case we will have "real" tile's (x,y size), we need a flag like
--tile-size-x=<int> and --tile-size-y=<int> to set the size of each tile
which should be loaded into the cache. But we need to rewrite the modules
so they can benefit from this kind of tiles, because G_get_rast_row() needs extra overhead
to create a single row from different tiles.

IIRC that is approximately what Glynn suggested about a new implementation
of the raster map storage.

Just my 2 cent.

Sören

btw.:
A 3d tile cache mechanism is already present in the g3d lib. 


-------- Original-Nachricht --------
Datum: Thu, 19 Jul 2007 19:14:29 +1200
Von: Hamish <hamish_nospam at yahoo.com>
An: grass5 <grass-dev at grass.itc.it>
Betreff: [GRASS-dev] ramdisk as mapset?

> Hi,
> 
> * I have some scripts which are very heavy on raster map disk i/o.
>    r.cost chugs heavily on the hard drive & the script can take days
>    to loop through. I don't want to wear a hole in it if I don't have to.
> * I have many GB of RAM to play with (enough to hold the region as DCELL)
> * The raster modules typically don't use much ram at all. (low overheads
>    to compete with for RAM)
> 
> I am trying to think of a way to get the raster ops to happen all in RAM
> to save time & wear on the hard drive. (script spans a number of r.*
> modules)
> 
> ideas so far:
> 
> 1) [Linux] create a 2GB ramdisk using ramfs. use g.mapset to swich into
> it, do the heavy i/o. switch back to the original mapset, g.copy the
> results map back to the "real" mapset, then destroy the ramdisk.
> advantages: easy to do.
> disadvantages: it's more of a local hack than a general solution.
> 
>  mkdir /mnt/ramdrive
> 
>  # default max_size is 1/2 physical ram, auto-resizes 'til then
>  mount -t ramfs none /mnt/ramdrive
>  mkdir -p /mnt/ramdrive/tmp_mapset
>  TMP_MAPSET="/mnt/ramdrive/tmp_mapset"
>  ln -s "$TMP_MAPSET" $USER/grassdata/$LOCATION/tmp_mapset
>  cp $USER/grassdata/$LOCATION/$MAPSET/WIND "$TMP_MAPSET"
>  g.mapset mapset=tmp_mapset
>  ...
>  g.module in=map@$MAPSET out=result
>  ...
>  g.mapset mapset=$MAPSET
>  g.copy result at tmp_mapset,result
>  umount /mnt/ramdrive
> 
> 
> problem: how to set group ID and mode/umask for ramdrive without
> having to do chown+chmod as root?
> 
> 
> 2) Some backgrounded "grass_mapd" process to dynamically allocate and
> hold a single map in memory. It's a child of the main GRASS process so
> exiting GRASS tears it down. It could be a "virtual" map sort of like
> how a reclass map is just a wrapper for something else. This is just a
> very rough idea, probably not so easy to do; but if possible I reckon it
> would be a cool tool to have.
> 
> 
> anyone have ideas?
> 
> thanks,
> Hamish
> 
> _______________________________________________
> grass-dev mailing list
> grass-dev at grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass-dev

-- 
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen! 
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer




More information about the grass-dev mailing list