[GRASSLIST:2301] Re: Import questions
whuber at quantdec.com
Tue Aug 7 12:52:14 EDT 2001
At 08:54 AM 8/7/01 -0700, Roderick A. Anderson wrote:
> > I'm working now with 10m DEMs and I have 15 adjacent quads patched
> > together. My system is a PII/333 with 327M RAM and the processing speed is
> > acceptable. I don't know where the speed bottleneck may be, but with a lot
> > of data it may be disk access.
>Probably. Do your systems use IDE or SCSI drives? An IDE based system
>has to use CPU resources to access the drives whereas a SCSI system uses
>hardly any CPU. Despite the 'proclaimed' speed of IDE drives they still
>take system resources to read the data that could be better used to
This rationale is good. I used to think exactly the same way. Then I
bought an ultra-wide SCSI drive and benchmarked it on GIS software.
When the processing time is CPU-bound, or the drive is shared among several
users, certainly SCSI can help, for all the reasons provided by Mr.
Anderson. But I haven't yet found a GIS situation where data access or
display were CPU-bound.
Consider that a 10m DEM quarter-quad spans 1/8 degree, or about 14 x 11
kilometers typically. That's 1.5 million elevations, each of them
represented as a float: four bytes. Fifteen of those therefore occupy 1.5
million * 4 * 15 = 90 million bytes on the disk. A reasonably fast disk
(for a system of your vintage) will spin off 2 Megabytes a second, leaving
you with 45 seconds access time. It doesn't matter whether the disk access
is IDE or SCSI if you're just waiting there for the data to be read.
Having a lot of RAM to cache the data once they have been read can help
immensely with re-display. Buying a high-density, high-RPM disk can
improve performance. (Buying a fast cheap IDE drive for your large
datasets on a personal workstation can do wonders.) Under Windows, using
FAT-32 instead of NTFS helps.
If you really need super disk performance, get a SCSI, SCA, or fiber optic
interface, buy matched fast (10,000 rpm) drives, and stripe the datasets
across three physical volumes. You should be able to read all those DEMs
in under 10 seconds with such a system. At that point your video display
will start to become the bottleneck...
I still buy IDE drives for our GIS workstations.
More information about the grass-user