[GRASS-dev] ramdisk as mapset?
"Sören Gebbert"
soerengebbert at gmx.de
Thu Jul 19 07:42:14 EDT 2007
Hi Hamish,
interesting idea. But lets put this idea into the library of grass. :)
I would like to suggest to extent the grass lib to
read the entire map into the memory if needed.
G_get_rast_row() will still work with no modification, but will
read the rows from memory and not from the disk.
And while implementing this, a tile caching mechanism would be good to.
IMHO we have to add new special --flags to allow every module to use this feature.
eg: --cache-all will read all tiles (in this case a tile is a row) into the memory.
--cache-size-y=<int> will set the number of rows which should be in the cache.
And in case we will have "real" tile's (x,y size), we need a flag like
--tile-size-x=<int> and --tile-size-y=<int> to set the size of each tile
which should be loaded into the cache. But we need to rewrite the modules
so they can benefit from this kind of tiles, because G_get_rast_row() needs extra overhead
to create a single row from different tiles.
IIRC that is approximately what Glynn suggested about a new implementation
of the raster map storage.
Just my 2 cent.
Sören
btw.:
A 3d tile cache mechanism is already present in the g3d lib.
-------- Original-Nachricht --------
Datum: Thu, 19 Jul 2007 19:14:29 +1200
Von: Hamish <hamish_nospam at yahoo.com>
An: grass5 <grass-dev at grass.itc.it>
Betreff: [GRASS-dev] ramdisk as mapset?
> Hi,
>
> * I have some scripts which are very heavy on raster map disk i/o.
> r.cost chugs heavily on the hard drive & the script can take days
> to loop through. I don't want to wear a hole in it if I don't have to.
> * I have many GB of RAM to play with (enough to hold the region as DCELL)
> * The raster modules typically don't use much ram at all. (low overheads
> to compete with for RAM)
>
> I am trying to think of a way to get the raster ops to happen all in RAM
> to save time & wear on the hard drive. (script spans a number of r.*
> modules)
>
> ideas so far:
>
> 1) [Linux] create a 2GB ramdisk using ramfs. use g.mapset to swich into
> it, do the heavy i/o. switch back to the original mapset, g.copy the
> results map back to the "real" mapset, then destroy the ramdisk.
> advantages: easy to do.
> disadvantages: it's more of a local hack than a general solution.
>
> mkdir /mnt/ramdrive
>
> # default max_size is 1/2 physical ram, auto-resizes 'til then
> mount -t ramfs none /mnt/ramdrive
> mkdir -p /mnt/ramdrive/tmp_mapset
> TMP_MAPSET="/mnt/ramdrive/tmp_mapset"
> ln -s "$TMP_MAPSET" $USER/grassdata/$LOCATION/tmp_mapset
> cp $USER/grassdata/$LOCATION/$MAPSET/WIND "$TMP_MAPSET"
> g.mapset mapset=tmp_mapset
> ...
> g.module in=map@$MAPSET out=result
> ...
> g.mapset mapset=$MAPSET
> g.copy result at tmp_mapset,result
> umount /mnt/ramdrive
>
>
> problem: how to set group ID and mode/umask for ramdrive without
> having to do chown+chmod as root?
>
>
> 2) Some backgrounded "grass_mapd" process to dynamically allocate and
> hold a single map in memory. It's a child of the main GRASS process so
> exiting GRASS tears it down. It could be a "virtual" map sort of like
> how a reclass map is just a wrapper for something else. This is just a
> very rough idea, probably not so easy to do; but if possible I reckon it
> would be a cool tool to have.
>
>
> anyone have ideas?
>
> thanks,
> Hamish
>
> _______________________________________________
> grass-dev mailing list
> grass-dev at grass.itc.it
> http://grass.itc.it/mailman/listinfo/grass-dev
--
Der GMX SmartSurfer hilft bis zu 70% Ihrer Onlinekosten zu sparen!
Ideal für Modem und ISDN: http://www.gmx.net/de/go/smartsurfer
More information about the grass-dev
mailing list