AW: [UMN_MAPSERVER-USERS] slighty OT: new hardware for a mapserver application

Tue Aug 9 13:13:49 EDT 2005

Frank Broniewski wrote:
> If I may interfere, I have also an question about putting vector data
> into postgis.
> If I may qoute you:
> 
> 
>>...though i would suggest to 
>>merge the shapes and MySQl alpha data into one PostgreSQL/PostGIS 
>>feature table. ...
> 
> 
> Are you putting all your e.g. point shapes into one table and seperate
> them by an attribute or do you create one table for one shapefile? Is
> there an advantage of having all shapefiles of one geometry in one table
> instead of having one table per shapefile?

Umm, well no special fondness for one or the other. It depends on what 
you plan to do with the data. Being lazy by nature i would probably 
choose one table, gives me less to configure. But in the end it really 
is a question of what you need to do to the data. Is the shape file your 
primary source and every time something changes you have to re-import it 
(yuck)? Or do you also plan tp maintain the data inside PG?

> I'm curious because here at the museum  we want to put our data also
> into a postgres/postgis db and are unsure about the most clever way to
> accomplish this.
> 
> Your Idea seemed somewhat revolutionary to me, so I wondered if you
> could describe a little more in detail your way of storing shapefiles in
> the database.

Oh nothing revolutionary, pure laziness and an affection for PG/PostGIS. 
You just can do all sorts of cool spatial operations with your data 
simply using SQL once it has been stuffed into PostGIS.

Best, Arnulf.

> Greetings
>                                                       
> Frank Broniewski
> Musée National d'Histoire et d'Art
> Section Préhistoire / Projet EPC
>     Tél: +352 260 281-21
> 241, Rue de Luxembourg
> L-8077 Bertrange
> 
> 
> -----Ursprüngliche Nachricht-----
> Von: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] Im
> Auftrag von Arnulf Christl
> Gesendet: Dienstag, 9. August 2005 13:09
> An: MAPSERVER-USERS at LISTS.UMN.EDU
> Betreff: Re: [UMN_MAPSERVER-USERS] slighty OT: new hardware for a
> mapserver application
> 
> 
> Dylan Beaudette wrote:
> 
>>Thanks for the tips Ed,
>>
>>There should be about 100Gb of imagery in geotiff format, 5 to 10 Gb 
>>of vector data stored in shapefiles, and about another 5 to 10 Gb of 
>>tabular data stored in MySQL.
> 
> 
> The vectorial data will not cause any problems though i would suggest to
> 
> merge the shapes and MySQl alpha data into one PostgreSQL/PostGIS 
> feature table. We are truly happy with PG doing all the work and 
> rendering sections out of 10 million and more polygons within
> milliseconds. There are quite different opinions though regarding
> performance of shape 
> files versus databases.
> 
> 
>>I am not expecting a huge number of hits per day, but right now it is 
>>hard to say... perhaps 10 to 100 requests per day?
> 
> 
> Is that individual requests or number of users? Larger installations are
> 
> known to render several hundred thousand maps a day so that your 
> suggested numbers should not even get you start to think.
> 
> 
>>We are considering a dual processor xeon system with SCSI disks, as 
>>this closely resembles our test machine.
> 
> 
> As suggested, keep it cheap, step up later. Dual CPU is cool, loads of 
> RAM is cool when using PG, fast disks are good for large raster data 
> (see Ed, he is the expert on rasters). If you need more database vector 
> performance put it on a separate machine.
> 
> Best, Arnulf.
> 
> 
>>Thanks again,
>>
>>Dylan
>>
>>On Monday 08 August 2005 10:11 am, Ed McNierney wrote:
>>
>>
>>>Dylan -
>>>
>>>I always recommend buying cheap stuff and making sure you ask it 
>>>simple questions <g>!
>>>
>>>If you are running lots of imagery, a fast disk subsystem is a good 
>>>idea.  But it's helpful to define "large quantity" more precisely, 
>>>because one person's "large" is another's "minuscule".  RAM is cheap 
>>>and always a good idea, as it can be used for disk caching and 
>>>swapping is a big performance hit.
>>>
>>>If you can be more specific about (a) the size of your data sets, (b) 
>>>the volume of requests and image sizes supported, and (c) the formats 
>>>of your data (TIFF? PostGIS? Shapefiles?) perhaps we can be more 
>>>specific about recommendations.
>>>
>>>	- Ed
>>>
>>>Ed McNierney
>>>President and Chief Mapmaker
>>>TopoZone.com / Maps a la carte, Inc.
>>>73 Princeton Street, Suite 305
>>>North Chelmsford, MA  01863
>>>ed at topozone.com
>>>(978) 251-4242
>>>
>>>-----Original Message-----
>>>From: UMN MapServer Users List [mailto:MAPSERVER-USERS at LISTS.UMN.EDU] 
>>>On Behalf Of Dylan Beaudette
>>>Sent: Monday, August 08, 2005 12:34 PM
>>>To: MAPSERVER-USERS at LISTS.UMN.EDU
>>>Subject: [UMN_MAPSERVER-USERS] slighty OT: new hardware for a 
>>>mapserver application
>>>
>>>Greetings,
>>>
>>>We are currently in the process of ramping up the scale of a 
>>>mapserver-based application, and thus in search of some new hardware.
>>>
>>>So far we have priced out some dual processor systems, with the best 
>>>deal from Dell. The machine will be running the UNIX version of 
>>>mapserver. The large quantity of imagery and high density vector files
> 
> 
>>>will require a powerful machine in order to keep latency low.
>>>
>>>Any suggestions or stories about previous purchases (related to a 
>>>mapserver
>>>application) would be much appreciated.
>>>
>>>Thanks in advance!
>>>
>>>--
>>>Dylan Beaudette
>>>Soils and Biogeochemistry Graduate Group University of California at 
>>>Davis 530.754.7341