[postgis-users] Best Filesystem?

Sun Apr 25 01:48:45 PDT 2004

Thanks for the useful info Gerry.  I found reports that indicated that JFS
was faster than XFS for PostgreSQL
(http://www.potentialtech.com/wmoran/postgresql.php#results).  I did not dig
into how sound his testing methods were.  I saw this confirmed on one or two
mailing list emails as well.

I really like my 3Ware, I am sure you will too.  I read that the Promise
RAID controllers are actually software RAID controllers.  They are, really
no more than re-packaged IDE controllers and a kernel driver that
understands RAID.  The Linux software RAID is a better implementation than
the Promise implementation both in terms of speed and features.  So, if you
want software RAID, you are correct... use the software RAID in the Linux
kernel.  I have the 3Ware Escalade and it is a true hardware RAID
controller, works great under Linux, and supports all RAID 0,1,5, and 10.

I went with JFS for my initial install.  I have never run JFS, so I am a
little leery.  If anyone is interested, I will post back an update after
running heavy against it for a month or so.

This question has to have been researched to death by now by someone.  I am
going to hit my local Borders bookstore tomorrow and see what sort of books
on PostgreSQL tuning have any information on it.

--Craig

-----Original Message-----
From: Gerry Creager N5JXS [mailto:gerry.creager at tamu.edu]
Sent: Saturday, April 24, 2004 9:20 PM
To: PostGIS Users Discussion
Subject: Re: [postgis-users] Best Filesystem?

I'd been using EXT3, and went from there to XFS.  I've gone to EXT2 now.
  I have seen subjective improvements for certain.  Objectively, the
load average on the machine has dropped with every change.

The machine is ingesting meteorological data on a rather grand scale,
and is taking a good deal of it and stuffing it into PostGIS.  We're
abstracting a time-dependent shapefile from PostGIS and using it in
Mapserver.

Additionally, since going to EXT2, I've not had to prune old data.  I've
got 5493888 rows currently and it's growing about 10,000 rows per day.
And that's going to accelerate.

One other issue: I'm using the HighPoint RAID 404 RAID5 controllers, 2
of them.  They're *SLOW*, limited to PCI-33.  I'm going to order some
3Ware Escalades Monday that should improve performance.  I'm considering
revamoing the RAID hardware to use SATA but that'll cost a bundle in
drive replacements...

Tests between Promise P-ATA RAID controllers and software RAID under
Linux by a colleague found that the kernel-based software RAID5 was a
better performer than hardware RAID5.  The conjecture is that there's
double-caching affecting latency for both reads and writes...

Gerry

Craig Miller wrote:
> Gerry,
>
> What filesystem did your roll back from?  Did you see an actual
performance
> gain?
>
> --Craig
>
>
> -----Original Message-----
> From: Gerry Creager N5JXS [mailto:gerry.creager at tamu.edu]
> Sent: Saturday, April 24, 2004 1:06 AM
> To: PostGIS Users Discussion
> Subject: Re: [postgis-users] Best Filesystem?
>
>
> Interesting question, and a good one.  I recently took a RAID5 box back
> to ext2 to remove the OS caching for performance...
>
> gerry
>
> Craig Miller wrote:
>
>>Hi,
>>
>>Has anyone benchmarked different filesystems with PostGIS?
>>
>>I have read that JFS offers the best performance for PostgreSQL, followed
>>closely by XFS.
>>
>>It is interesting... I am finding that almost every layer in the stack is
>>doing some form of caching and flushing.  I haven't checked to see what my
>>hard drives themselves are doing, but my RAID controller caches,
>>XFS/ReiserFS caches, and PostgreSQL caches.  While cacheing is generally a
>>good thing, it is bad if it is a) redundant or b) exposing me unnecessary
>>risk.  The latter is my largest concern.  If my system has it's power cord
>>bumped, I don't want to lose the data that is floating around in a cache.
>>
>>So, what is the "best" filesystem for running PostGIS based on performance
>>and data integrity.
>>
>>Looking forward to the responses,
>>--Craig
>>
>>
>>-----Original Message-----
>>From: Wood Brent [mailto:pcreso at pcreso.com]
>>Sent: Thursday, April 22, 2004 9:49 AM
>>To: postgis-users at postgis.refractions.net
>>Subject: Re: [postgis-users] interoperability/compatibility
>>
>>
>>
>>
>>
>>
>>>Dear users
>>>
>>>Can someone comment on general compatibility/interoperability of OSS with
>>>other OSS or with the preoprietary systems.
>>
>>
>>Perhaps useful references (pertaining to PostGIS) would be:
>>
>>http://qgis.sourceforge.net/docs/install.html
>>
>>QGIS is a simple GUI which allows you to look at PostGIS data, and import
>>data
>>from shapefiles intp PostGIS tables. It uses PostGIS, Postgres, GDAL,
>
> Proj4,
>
>>GEOS, Qt to work together to create a GIS suite from interoperating
>>components.
>>
>>The install guide describes the intallation of each of these components to
>>build the whole system.
>>
>>
>>
>
>
http://www.ing.unitn.it/~grass/conferences/GRASS2002/proceedings/proceedings
>
>>/pdfs/Blazek_Radim.pdf
>>
>>This paper was presented at a GRASS conference a couple of years ago.
>
> GRASS
>
>>is perhaps the premier OpenSource GIS package, certainly in terms on
>>longevity
>>&
>>overall functionality. It describes the new vector data capability which
>
> is
>
>>based on PostGIS as the spatial vector data management tool.
>>
>>
>>http://www-stat.uni-klu.ac.at/~agebhard/preDSC2003.pdf
>>
>>This contains the preliminary slides from a presentation about integrating
>>PostGIS and R (the open source stats package)
>>
>>
>>www.safe.com/reader_writerPDF/postgis.pdf
>>
>>FME from SAFE is pobably the most complete commercial GIS data reformatter
>>available. They now support PostGIS as a standard geographic data source.
>>
>>
>>There are plenty of others, UMN mapserver supports PostGIS, in a similar
>>fashion to QGIS, so web mapping is also there.
>>
>>
>>Hopefully a few useful examples....
>>
>>Brent Wood
>>
>>_______________________________________________
>>postgis-users mailing list
>>postgis-users at postgis.refractions.net
>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>>
>>
>>
>>
>>_______________________________________________
>>postgis-users mailing list
>>postgis-users at postgis.refractions.net
>>http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
> --
> Gerry Creager -- gerry.creager at tamu.edu
> Network Engineering -- AATLT, Texas A&M University
> Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
> Page: 979.228.0173
> Office: 903A Eller Bldg, TAMU, College Station, TX 77843
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users
>
>
>
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at postgis.refractions.net
> http://postgis.refractions.net/mailman/listinfo/postgis-users

--
Gerry Creager -- gerry.creager at tamu.edu
Network Engineering -- AATLT, Texas A&M University
Cell: 979.229.5301 Office: 979.458.4020 FAX: 979.847.8578
Page: 979.228.0173
Office: 903A Eller Bldg, TAMU, College Station, TX 77843

_______________________________________________
postgis-users mailing list
postgis-users at postgis.refractions.net
http://postgis.refractions.net/mailman/listinfo/postgis-users