MapServer performance

[Ed McNierney]

Antti -

Even if you don't have control over it, you should still know what it is.
If you KNOW you're forced to live with a hideously slow disk system, then
you don't need to waste your time trying to optimize the rest of the
application.  If you know you have a really high-performing disk system for
this application, you can focus on performance improvements in other areas
and expect them to have some visible effect.

     - Ed


> From: antti siukola <siukola.antti at gmail.com>
> Date: Fri, 23 Mar 2007 19:15:31 +0200
> To: Ed McNierney <ed at topozone.com>
> Cc: <MAPSERVER-USERS at lists.umn.edu>
> Subject: Re: [UMN_MAPSERVER-USERS] MapServer performance
> 
> Ed
> 
> Of course. I understand. Unfortunately I am forced to develop these
> applications in an environment where I have no control over what
> hardware I have and can't do nothing about it. So, if there's any
> just-works kind of things I can do I would appreciate it. However
> thank you very much for your time and effort.
> 
> Cheers!
> 
> Antti S
> 
> 
> On 3/23/07, Ed McNierney <ed at topozone.com> wrote:
>> Antti -
>> 
>> Well, then that would be a good thing to learn about if you want to
>> understand your system performance.  "RAID stack" is a pretty generic
>> description, covering several different disk storage strategies, and
>> covering both IDE and SCSI drives with various interface channels.  Reading
>> from a parallel IDE 5,400 RPM RAID 1 system can be several times slower than
>> reading from a SAS 15K RPM RAID 5 array!
>> 
>> If you're going to optimize performance, find out more about your disk
>> subsystem.  Your hard disks are the slowest things in your computer.  If you
>> want to speed things up, it makes sense to see whether the slowest hardware
>> in your system is a bottleneck before you start looking elsewhere.
>> 
>>      - Ed
>> 
>> --
>> Ed McNierney
>> President and Chief Mapmaker
>> Maps a la carte, Inc. / TopoZone.com
>> 73 Princeton Street, Suite 305
>> North Chelmsford, MA  01863
>> Phone: (978) 251-4242
>> Fax: (978) 251-1396
>> ed at topozone.com
>> 
>>> From: antti siukola <siukola.antti at gmail.com>
>>> Date: Fri, 23 Mar 2007 18:14:04 +0200
>>> To: Ed McNierney <ed at topozone.com>
>>> Cc: <MAPSERVER-USERS at lists.umn.edu>
>>> Subject: Re: [UMN_MAPSERVER-USERS] MapServer performance
>>> 
>>> Hi!
>>> 
>>> The tiff's are stored on a RAID stack on the same server and the
>>> filesystem is ext3. I don't have very detailed description to give
>>> since I'm not administrating the server or have ever even seen it :)
>>> 
>>> Antti S
>>> 
>>> On 3/23/07, Ed McNierney <ed at topozone.com> wrote:
>>>> Antti -
>>>> 
>>>> 1 GB of RAM is not a lot for dealing with large raster files.  Could you
>>>> describe your hard disk subsystem in detail?  What kind of hardware are you
>>>> using to store the TIFF images?
>>>> 
>>>>      - Ed
>>>> 
>>>> --
>>>> Ed McNierney
>>>> President and Chief Mapmaker
>>>> Maps a la carte, Inc. / TopoZone.com
>>>> 73 Princeton Street, Suite 305
>>>> North Chelmsford, MA  01863
>>>> Phone: (978) 251-4242
>>>> Fax: (978) 251-1396
>>>> ed at topozone.com
>>>> 
>>>>> From: antti siukola <siukola.antti at GMAIL.COM>
>>>>> Reply-To: antti siukola <siukola.antti at GMAIL.COM>
>>>>> Date: Fri, 23 Mar 2007 11:11:18 +0200
>>>>> To: <MAPSERVER-USERS at LISTS.UMN.EDU>
>>>>> Subject: Re: [UMN_MAPSERVER-USERS] MapServer performance
>>>>> 
>>>>> Hi!
>>>>> 
>>>>> Thanks for the reply!
>>>>> 
>>>>> My tiff's are from 15M to 10G tiff tiles (about 500M each tile) and
>>>>> depending on the map theres the one 15M tiff or from 2 to 3 of the
>>>>> 500M tiles displayed (I have couple of map apps running on the
>>>>> server). It doesn't seem to matter how big the tiff's are since the
>>>>> performance is the quite the same. I also have the .shx files for my
>>>>> shape's.
>>>>> 
>>>>> The server doesn't get very much hit's since it's not an open for
>>>>> public application. I believe the average usage is about 1-10 users at
>>>>> the same time.
>>>>> 
>>>>> Hope this helps!
>>>>> 
>>>>> Antti S
>>>>> 
>>>>> On 3/21/07, Romolo Manfredini <romoloman at hotmail.com> wrote:
>>>>>> From: "antti siukola" :
>>>>>> 
>>>>>>> Oh, yes. I'm on linux with mapserver 4.8.4 and php mapscript. The
>>>>>>> server is quite new and powerful with dual xeon and 1024M of memory.
>>>>>>> Data sources are mostly SHP-files and few TABs for vector data and
>>>>>>> labels. The actual maps are .tif images. I'm using tiled tiff's and
>>>>>>> low-res images on upper scales.
>>>>>>> 
>>>>>> 
>>>>>> SHP and tiff files are typically bad beast to manage expecially if they
>>>>>> are
>>>>>> big shape and big tiff, also dependig on file system type you may have
>>>>>> some
>>>>>> performance issues related to locking and caching. (1 GB ram with large
>>>>>> files is not so much)
>>>>>> Also, do you have spatial index on your files (.shx) ?
>>>>>> Spatial index improve performances a lot.
>>>>>> May you tell me the size of your files and how many of them are involved
>>>>>> in
>>>>>> a single map ?
>>>>>> 
>>>>>> Best Regards,
>>>>>> Romolo
>>>>>> 
>>>>>> 
>>>> 
>>>> 
>> 
>>