[postgis-users] Best Filesystem?

Sat Apr 24 21:19:48 PDT 2004

I'd been using EXT3, and went from there to XFS.  I've gone to EXT2 now. 
  I have seen subjective improvements for certain.  Objectively, the 
load average on the machine has dropped with every change.

The machine is ingesting meteorological data on a rather grand scale, 
and is taking a good deal of it and stuffing it into PostGIS.  We're 
abstracting a time-dependent shapefile from PostGIS and using it in 
Mapserver.

Additionally, since going to EXT2, I've not had to prune old data.  I've 
got 5493888 rows currently and it's growing about 10,000 rows per day. 
And that's going to accelerate.

One other issue: I'm using the HighPoint RAID 404 RAID5 controllers, 2 
of them.  They're *SLOW*, limited to PCI-33.  I'm going to order some 
3Ware Escalades Monday that should improve performance.  I'm considering 
revamoing the RAID hardware to use SATA but that'll cost a bundle in 
drive replacements...

Tests between Promise P-ATA RAID controllers and software RAID under 
Linux by a colleague found that the kernel-based software RAID5 was a 
better performer than hardware RAID5.  The conjecture is that there's 
double-caching affecting latency for both reads and writes...

Gerry

Craig Miller wrote:
> Gerry,
> 
> What filesystem did your roll back from?  Did you see an actual performance
> gain?
> 
> --Craig
> 
> 
> -----Original Message-----
> From: Gerry Creager N5JXS [mailto:gerry.creager at tamu.edu]
> Sent: Saturday, April 24, 2004 1:06 AM
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] Best Filesystem?
> 
> 
> Interesting question, and a good one.  I recently took a RAID5 box back
> to ext2 to remove the OS caching for performance...
> 
> gerry
> 
> Craig Miller wrote:
> 
>>Hi,
>>
>>Has anyone benchmarked different filesystems with PostGIS?
>>
>>I have read that JFS offers the best performance for PostgreSQL, followed
>>closely by XFS.
>>
>>It is interesting... I am finding that almost every layer in the stack is
>>doing some form of caching and flushing.  I haven't checked to see what my
>>hard drives themselves are doing, but my RAID controller caches,
>>XFS/ReiserFS caches, and PostgreSQL caches.  While cacheing is generally a
>>good thing, it is bad if it is a) redundant or b) exposing me unnecessary
>>risk.  The latter is my largest concern.  If my system has it's power cord
>>bumped, I don't want to lose the data that is floating around in a cache.
>>
>>So, what is the "best" filesystem for running PostGIS based on performance
>>and data integrity.
>>
>>Looking forward to the responses,
>>--Craig
>>
>>
>>-----Original Message-----
>>From: Wood Brent [mailto:pcreso at pcreso.com]
>>Sent: Thursday, April 22, 2004 9:49 AM
>>To: postgis-users at postgis.refractions.net
>>Subject: Re: [postgis-users] interoperability/compatibility
>>
>>
>>
>>
>>
>>
>>>Dear users
>>>
>>>Can someone comment on general compatibility/interoperability of OSS with
>>>other OSS or with the preoprietary systems.
>>
>>
>>Perhaps useful references (pertaining to PostGIS) would be:
>>
>>http://qgis.sourceforge.net/docs/install.html
>>
>>QGIS is a simple GUI which allows you to look at PostGIS data, and import
>>data
>>from shapefiles intp PostGIS tables. It uses PostGIS, Postgres, GDAL,
> 
> Proj4,
> 
>>GEOS, Qt to work together to create a GIS suite from interoperating
>>components.
>>
>>The install guide describes the intallation of each of these components to
>>build the whole system.
>>
>>
>>
> 
> http://www.ing.unitn.it/~grass/conferences/GRASS2002/proceedings/proceedings
> 
>>/pdfs/Blazek_Radim.pdf
>>
>>This paper was presented at a GRASS conference a couple of years ago.
> 
> GRASS
> 
>>is perhaps the premier OpenSource GIS package, certainly in terms on
>>longevity
>>&
>>overall functionality. It describes the new vector data capability which
> 
> is
> 
>>based on PostGIS as the spatial vector data management tool.
>>
>>
>>http://www-stat.uni-klu.ac.at/~agebhard/preDSC2003.pdf
>>
>>This contains the preliminary slides from a presentation about integrating
>>PostGIS and R (the open source stats package)
>>
>>
>>www.safe.com/reader_writerPDF/postgis.pdf
>>
>>FME from SAFE is pobably the most complete commercial GIS data reformatter
>>available. They now support PostGIS as a standard geographic data source.
>>
>>
>>There are plenty of others, UMN mapserver supports PostGIS, in a similar
>>fashion to QGIS, so web mapping is also there.
>>
>>
>>Hopefully a few useful examples....
>>
>>Brent Wood
>>
>>_______________________________________________
>>postgis-users mailing list
>>postgis-users at postgis.refractions.net
>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>>
>>_______________________________________________
>>postgis-users mailing list
>>postgis-users at postgis.refractions.net
>>http://postgis.refractions.net/mailman/listinfo/postgis-users
> 
> 
> --
> Gerry Creager -- gerry.creager at tamu.edu
> Network Engineering -- AATLT, Texas A&M University
> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
> Page: 979.228.0173
> Office: 903A Eller Bldg, TAMU, College Station, TX 77843
> 
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
> 
> 
> 
> 
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users

-- 
Gerry Creager -- gerry.creager at tamu.edu
Network Engineering -- AATLT, Texas A&M University	
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
Page: 979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843