[postgis-users] Best Filesystem?

Sun Apr 25 08:27:20 PDT 2004

O'Reilly has a "Managing RAID on Linux" book but it's short on 
recommendations.  It does, however, do a pretty good job of laying out 
the alternatives and how things work.

It's been useful and a disappointment at the same time.

Gerry

Craig Miller wrote:
> Thanks for the useful info Gerry.  I found reports that indicated that JFS
> was faster than XFS for PostgreSQL
> (http://www.potentialtech.com/wmoran/postgresql.php#results).  I did not dig
> into how sound his testing methods were.  I saw this confirmed on one or two
> mailing list emails as well.
> 
> I really like my 3Ware, I am sure you will too.  I read that the Promise
> RAID controllers are actually software RAID controllers.  They are, really
> no more than re-packaged IDE controllers and a kernel driver that
> understands RAID.  The Linux software RAID is a better implementation than
> the Promise implementation both in terms of speed and features.  So, if you
> want software RAID, you are correct... use the software RAID in the Linux
> kernel.  I have the 3Ware Escalade and it is a true hardware RAID
> controller, works great under Linux, and supports all RAID 0,1,5, and 10.
> 
> I went with JFS for my initial install.  I have never run JFS, so I am a
> little leery.  If anyone is interested, I will post back an update after
> running heavy against it for a month or so.
> 
> This question has to have been researched to death by now by someone.  I am
> going to hit my local Borders bookstore tomorrow and see what sort of books
> on PostgreSQL tuning have any information on it.
> 
> --Craig
> 
> 
> -----Original Message-----
> From: Gerry Creager N5JXS [mailto:gerry.creager at tamu.edu]
> Sent: Saturday, April 24, 2004 9:20 PM
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] Best Filesystem?
> 
> 
> I'd been using EXT3, and went from there to XFS.  I've gone to EXT2 now.
>   I have seen subjective improvements for certain.  Objectively, the
> load average on the machine has dropped with every change.
> 
> The machine is ingesting meteorological data on a rather grand scale,
> and is taking a good deal of it and stuffing it into PostGIS.  We're
> abstracting a time-dependent shapefile from PostGIS and using it in
> Mapserver.
> 
> Additionally, since going to EXT2, I've not had to prune old data.  I've
> got 5493888 rows currently and it's growing about 10,000 rows per day.
> And that's going to accelerate.
> 
> One other issue: I'm using the HighPoint RAID 404 RAID5 controllers, 2
> of them.  They're *SLOW*, limited to PCI-33.  I'm going to order some
> 3Ware Escalades Monday that should improve performance.  I'm considering
> revamoing the RAID hardware to use SATA but that'll cost a bundle in
> drive replacements...
> 
> Tests between Promise P-ATA RAID controllers and software RAID under
> Linux by a colleague found that the kernel-based software RAID5 was a
> better performer than hardware RAID5.  The conjecture is that there's
> double-caching affecting latency for both reads and writes...
> 
> Gerry
> 
> Craig Miller wrote:
> 
>>Gerry,
>>
>>What filesystem did your roll back from?  Did you see an actual
> 
> performance
> 
>>gain?
>>
>>--Craig
>>
>>
>>-----Original Message-----
>>From: Gerry Creager N5JXS [mailto:gerry.creager at tamu.edu]
>>Sent: Saturday, April 24, 2004 1:06 AM
>>To: PostGIS Users Discussion
>>Subject: Re: [postgis-users] Best Filesystem?
>>
>>
>>Interesting question, and a good one.  I recently took a RAID5 box back
>>to ext2 to remove the OS caching for performance...
>>
>>gerry
>>
>>Craig Miller wrote:
>>
>>
>>>Hi,
>>>
>>>Has anyone benchmarked different filesystems with PostGIS?
>>>
>>>I have read that JFS offers the best performance for PostgreSQL, followed
>>>closely by XFS.
>>>
>>>It is interesting... I am finding that almost every layer in the stack is
>>>doing some form of caching and flushing.  I haven't checked to see what my
>>>hard drives themselves are doing, but my RAID controller caches,
>>>XFS/ReiserFS caches, and PostgreSQL caches.  While cacheing is generally a
>>>good thing, it is bad if it is a) redundant or b) exposing me unnecessary
>>>risk.  The latter is my largest concern.  If my system has it's power cord
>>>bumped, I don't want to lose the data that is floating around in a cache.
>>>
>>>So, what is the "best" filesystem for running PostGIS based on performance
>>>and data integrity.
>>>
>>>Looking forward to the responses,
>>>--Craig
>>>
>>>
>>>-----Original Message-----
>>>From: Wood Brent [mailto:pcreso at pcreso.com]
>>>Sent: Thursday, April 22, 2004 9:49 AM
>>>To: postgis-users at postgis.refractions.net
>>>Subject: Re: [postgis-users] interoperability/compatibility
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>>Dear users
>>>>
>>>>Can someone comment on general compatibility/interoperability of OSS with
>>>>other OSS or with the preoprietary systems.
>>>
>>>
>>>Perhaps useful references (pertaining to PostGIS) would be:
>>>
>>>http://qgis.sourceforge.net/docs/install.html
>>>
>>>QGIS is a simple GUI which allows you to look at PostGIS data, and import
>>>data
>>
>>>from shapefiles intp PostGIS tables. It uses PostGIS, Postgres, GDAL,
>>
>>Proj4,
>>
>>
>>>GEOS, Qt to work together to create a GIS suite from interoperating
>>>components.
>>>
>>>The install guide describes the intallation of each of these components to
>>>build the whole system.
>>>
>>>
>>>
>>
>>
> http://www.ing.unitn.it/~grass/conferences/GRASS2002/proceedings/proceedings
> 
>>>/pdfs/Blazek_Radim.pdf
>>>
>>>This paper was presented at a GRASS conference a couple of years ago.
>>
>>GRASS
>>
>>
>>>is perhaps the premier OpenSource GIS package, certainly in terms on
>>>longevity
>>>&
>>>overall functionality. It describes the new vector data capability which
>>
>>is
>>
>>
>>>based on PostGIS as the spatial vector data management tool.
>>>
>>>
>>>http://www-stat.uni-klu.ac.at/~agebhard/preDSC2003.pdf
>>>
>>>This contains the preliminary slides from a presentation about integrating
>>>PostGIS and R (the open source stats package)
>>>
>>>
>>>www.safe.com/reader_writerPDF/postgis.pdf
>>>
>>>FME from SAFE is pobably the most complete commercial GIS data reformatter
>>>available. They now support PostGIS as a standard geographic data source.
>>>
>>>
>>>There are plenty of others, UMN mapserver supports PostGIS, in a similar
>>>fashion to QGIS, so web mapping is also there.
>>>
>>>
>>>Hopefully a few useful examples....
>>>
>>>Brent Wood
>>>
>>>_______________________________________________
>>>postgis-users mailing list
>>>postgis-users at postgis.refractions.net
>>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>>>
>>>
>>>
>>>
>>>_______________________________________________
>>>postgis-users mailing list
>>>postgis-users at postgis.refractions.net
>>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>--
>>Gerry Creager -- gerry.creager at tamu.edu
>>Network Engineering -- AATLT, Texas A&M University
>>Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
>>Page: 979.228.0173
>>Office: 903A Eller Bldg, TAMU, College Station, TX 77843
>>
>>_______________________________________________
>>postgis-users mailing list
>>postgis-users at postgis.refractions.net
>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>>
>>_______________________________________________
>>postgis-users mailing list
>>postgis-users at postgis.refractions.net
>>http://postgis.refractions.net/mailman/listinfo/postgis-users
> 
> 
> --
> Gerry Creager -- gerry.creager at tamu.edu
> Network Engineering -- AATLT, Texas A&M University
> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
> Page: 979.228.0173
> Office: 903A Eller Bldg, TAMU, College Station, TX 77843
> 
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
> 
> 
> 
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users

-- 
Gerry Creager -- gerry.creager at tamu.edu
Network Engineering -- AATLT, Texas A&M University	
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
Page: 979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843