[mapserver-users] WMS Orthophoto performance

[Andy Colson]

> 2010/8/2 Andy Colson <andy at squeakycode.net <mailto:andy at squeakycode.net>>
>
>     On 8/2/2010 5:06 AM, Cédric MOULLET wrote:
>
>         Hello,
>         We've got a 150 GB ECW dataset (aerial imagery) and the LAYER
>         configuration uses a TILEINDEX.
>         We can observe a WMS request time of about 3 seconds (on a
>         server with
>         this configuration: 7.5 GB RAM, 4 core, 64 bits / MapServer 5.4.3).
>         I'd like to know if 3 seconds is considered as a normal response
>         time
>         for this kind of WMS request.
>         What could be done in order to improve this performance (I'm
>         aware of
>         http://www.mapserver.org/optimization/raster.html) ?
>         Thanks !
>         Cédric
>
>
>     Wow, that's a big file.  Is that all in one file?  I dont really
>     know the answer, I've never used anything over 4 gig, but I do have
>     a few questions:
>
>     1) I assume you are IO bound, do you have this on a raid, or
>     something with fast IO?  Can you check to see if you are CPU bound
>     or IO bound?
>
>     2) if it is one big file, how would you feel about trying it broken
>     up into smaller files?
>
>     -Andy
>

On 8/2/2010 12:16 PM, Cédric MOULLET wrote:
 > Hi Andy,
 > Thanks for your answer.
 > The dataset is composed of several ECW files, that's why we use 
TILEINDEX.
 > The data are hosted on EBS (http://aws.amazon.com/ebs/). It's supposed
 > to have normal I/O performance, but probably not fantastic performance.
 > Of what I have observed, I would say that the CPU is probably the
 > bottleneck.
 > My goal is to have a 1s response time. Do I dream too much ?
 > Cédric
 >


I have had problems where we had lots and lots of little tiny files.  It 
was much quicker to make a single gigabyte file.  But, that again was IO 
problems.

If you really are CPU bound, then I'm not sure what you can do.  Can you 
get your client program to request multiple tiles at once instead of one 
big one?  mapserver wont use more than one cpu per request, so you'll 
need to make multiple requests to use multiple cpu's.

 > Of what I have observed, I would say that the CPU is probably the
 > bottleneck.

Are you guessing?  How did you investigate it?  Can you run 'vmstat 3' 
for a while and post it?  It's hard to fix unless you know what's broke.

Also, in the mapfile, you can enable debugging for that one layer, one 
of the debug levels includes timing information (its been a while since 
I've done it, but the doc's describe it pretty well).

Also, what size of image is being returned?  Maybe its TCP/IP 
bandwidth/latency problems?  You could compare client side timing with 
the mapfile debug timing... they should be pretty close.  If the mapfile 
shows a quicker time than your webclient then its not mapserver at all.

-Andy