[Mon May 3 11:35:28 2010].1406 msDrawMap(): Drawing Label Cache, 0.000s<br>[Mon May 3 11:35:28 2010].132164 msDrawMap() total time: 0.136s<br>[Mon May 3 11:35:28 2010].511199 msSaveImage() total time: 0.379s<br>[Mon May 3 11:35:28 2010].511596 mapserv request processing time (msLoadMap not<br>
incl.): 0.533s<br>[Mon May 3 11:35:28 2010].511607 msFreeMap(): freeing map at 0x128c9cc0.<br>[Mon May 3 11:35:31 2010].519338 msDrawMap(): Layer 1 ((null)), 0.173s<br>[Mon May 3 11:35:31 2010].519495 msDrawMap(): Drawing Label Cache, 0.000s<br>
[Mon May 3 11:35:31 2010].519507 msDrawMap() total time: 0.179s<br>[Mon May 3 11:35:32 2010].564206 msSaveImage() total time: 1.045s<br>[Mon May 3 11:35:32 2010].564672 mapserv request processing time (msLoadMap not<br>
incl.): 1.233s<br>[Mon May 3 11:35:32 2010].564683 msFreeMap(): freeing map at 0x15534cb0.<br><br>Seems pretty clear the problem is msSaveImage(). Not really sure what to do to fix this.<br><br><div class="gmail_quote">
On Mon, May 3, 2010 at 2:05 PM, Fawcett, David (MPCA) <span dir="ltr"><<a href="mailto:David.Fawcett@state.mn.us">David.Fawcett@state.mn.us</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin: 0pt 0pt 0pt 0.8ex; border-left: 1px solid rgb(204, 204, 204); padding-left: 1ex;">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Adam, </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">It sounds like you have already enabled debug mode, if now, you
should be able to get the rendering time from the output.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">David.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<div style="border-width: 1pt medium medium; border-style: solid none none; border-color: rgb(181, 196, 223) -moz-use-text-color -moz-use-text-color; padding: 3pt 0in 0in;">
<p class="MsoNormal"><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> Adam Eskreis
[mailto:<a href="mailto:aeskreis@gmail.com" target="_blank">aeskreis@gmail.com</a>] <br>
<b>Sent:</b> Monday, May 03, 2010 1:02 PM<br>
<b>To:</b> Fawcett, David (MPCA)<br>
<b>Cc:</b> Lime, Steve D (DNR); <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><div><div></div><div class="h5"><br>
<b>Subject:</b> Re: [mapserver-users] Re: WMS and tiles cutting off</div></div></span></p>
</div><div><div></div><div class="h5">
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="margin-bottom: 12pt;">Thank you for the response
David.<br>
<br>
I have optimized the query all I can. I have optimized the query to the
point that even with all categories selected, the query has a run time of 52
ms. I can only assume that the issue lies with the generation of the
images. I will try to run it without the "IN" part of the
query, but I can't imagine that it'st he problem. I think the problem is
that there are a large number of symbols that need to be generated, and it's
taking mapserver a long time to load them in. Also, yes I have indexes on
all columns that I am calling in, and yes, they are all using the same SRS.
I am still trying a few more things, but I don't think it's the database query
that's causing the problem.<br>
<br>
</p>
<div>
<p class="MsoNormal">On Fri, Apr 30, 2010 at 9:37 AM, Fawcett, David (MPCA) <<a href="mailto:David.Fawcett@state.mn.us" target="_blank">David.Fawcett@state.mn.us</a>>
wrote:</p>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Adam, </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">It sounds like you have boiled the issue
down to too much of a time lag between the time the query is executed and the
image appears in the client. </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">Is it taking a long time for MapServer
to draw the image, or is it the delivery of the image to the requesting machine
and subsequent placement in the client? </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">If it is taking MapServer a long time,
maybe there are some more data optimizations that you can do. Is there a
better way to structure your data, can you do any pre-processing? Is it
all in the same SRS, got attribute and spatial indexes? I have used the
SQL IN for some apps with reasonable performance, but in the last 6 months or
so, there were some posts on this list comparing the use of IN to a layer (or
class?) filter. The latter solution was significantly faster.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">If it is taking a long time to get to
the client, then that is likely an infrastructure issue on either end.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">If you want some more help digging in
more deeply, post some more stats and mapfile snippets.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);">David.</span></p>
<p class="MsoNormal"><span style="font-size: 11pt; color: rgb(31, 73, 125);"> </span></p>
<div style="border-width: 1pt medium medium; border-style: solid none none; padding: 3pt 0in 0in; border-color: -moz-use-text-color;">
<p class="MsoNormal"><b><span style="font-size: 10pt;">From:</span></b><span style="font-size: 10pt;"> <a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>
[mailto:<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>]
<b>On Behalf Of </b>Adam Eskreis<br>
<b>Sent:</b> Thursday, April 29, 2010 4:24 PM<br>
<b>To:</b> Lime, Steve D (DNR)<br>
<b>Cc:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a></span></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size: 10pt;"><br>
<b>Subject:</b> Re: [mapserver-users] Re: WMS and tiles cutting off</span></p>
</div>
</div>
</div>
<div>
<div>
<p class="MsoNormal"> </p>
<p class="MsoNormal" style="margin-bottom: 12pt;">I tried
doing ratio:1 unfortunately it doesn't seem like enough of a performance
increase. Separate layers for each category is an option, however, there
are over a hundred categories, and any combination of categories can be on at
any given time, so this could prove to have performance issues of it's
own. However, I thank you guys Steve and Kaipi for your advice. I
think I am just going to go back to my old method, which was using php
mapscript to generate the tiles, then each time the user selected new
categories, purge the cache and remake the mapfile. It's not the ideal
solution but it seems to be the only one.<br>
<br>
Thanks again,<br>
<br>
-Adam</p>
<div>
<p class="MsoNormal">On
Thu, Apr 29, 2010 at 3:50 PM, Lime, Steve D (DNR) <<a href="mailto:Steve.Lime@state.mn.us" target="_blank">Steve.Lime@state.mn.us</a>>
wrote:</p>
<p class="MsoNormal">Note
that by default I believe openlayers will request an image that is *much*
larger than what is visible. Setting the layers (OL) ratio:1 makes it the same
size as the view port and that can speed up rendering considerably.<br>
<br>
Alternatively you could create a separate set of tiles for each category I
suppose and switch OL layers as necessary...</p>
<div>
<p class="MsoNormal"><br>
Steve<br>
<br>
________________________________________<br>
From: <a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>
[<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>]
On Behalf Of Adam Eskreis [<a href="mailto:aeskreis@gmail.com" target="_blank">aeskreis@gmail.com</a>]</p>
</div>
<p class="MsoNormal">Sent:
Thursday, April 29, 2010 2:25 PM</p>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;">To: <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
Subject: Re: [mapserver-users] Re: WMS and tiles cutting off</p>
</div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;">Ok, I
just gave an untiled layer a try, unfortunately though, due to the large amount
of symbols being displayed on the screen, there is about a 2 second delay every
time the map is panned. My client will most certainly find this
unacceptable. I have tried using smaller images, but I cannot sacrifice
any more quality. I believe my database query to be as optimized as it
can be (has about 50 ms runtime, I honestly don't think I can get it any lower
than that).<br>
<br>
If these are the only options, is there something else besides WMS that you all
would recommend using to solve this issue?<br>
<br>
Thanks,<br>
-Adam</p>
</div>
<div>
<p class="MsoNormal">On
Thu, Apr 29, 2010 at 2:54 PM, Lime, Steve D (DNR) <<a href="mailto:Steve.Lime@state.mn.us" target="_blank">Steve.Lime@state.mn.us</a><mailto:<a href="mailto:Steve.Lime@state.mn.us" target="_blank">Steve.Lime@state.mn.us</a>>>
wrote:<br>
So use two layers. One for the background, use tilecache for that. Then use an
untiled wms/mapserver layer for the overlayed points. - Steve<br>
________________________________________</p>
</div>
<p class="MsoNormal">From:
<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a><mailto:<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>>
[<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a><mailto:<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>>]
On Behalf Of Adam Eskreis [<a href="mailto:aeskreis@gmail.com" target="_blank">aeskreis@gmail.com</a><mailto:<a href="mailto:aeskreis@gmail.com" target="_blank">aeskreis@gmail.com</a>>]</p>
<div>
<p class="MsoNormal">Sent:
Thursday, April 29, 2010 1:24 PM</p>
</div>
<p class="MsoNormal">To:
<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><mailto:<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a>></p>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;">Subject:
Re: [mapserver-users] Re: WMS and tiles cutting off<br>
<br>
I have thought about using tilecache with metatiling, however there is a caveat
about this map that i forgot to mention in my main post, and it has to do with
that "categories" parameter. Basically, each point/symbol on
the map has an associated category in the database backend. I am using
mapserver/WMS parameters to change the categories which are called. Each
time the user selects a category, only the points in that category are shown.
My query in the mapfile looks something like thi:<br>
<br>
SELECT point_geom,gid,symbol,angle FROM mapdata WHERE category IN(%CATEGORIES%)";<br>
<br>
And each time the user changes the categories, I use openlayers to merge new
parameters, and input the new category values that should be used. Is
tilecache still useable if I have a system like this set up?<br>
<br>
Thanks,<br>
-Adam</p>
</div>
<div>
<p class="MsoNormal">On
Thu, Apr 29, 2010 at 6:14 AM, kaipi <<a href="mailto:mapcollect@gmx.net" target="_blank">mapcollect@gmx.net</a><mailto:<a href="mailto:mapcollect@gmx.net" target="_blank">mapcollect@gmx.net</a>><mailto:<a href="mailto:mapcollect@gmx.net" target="_blank">mapcollect@gmx.net</a><mailto:<a href="mailto:mapcollect@gmx.net" target="_blank">mapcollect@gmx.net</a>>>>
wrote:<br>
<br>
Hi,<br>
<br>
you can use <a href="http://tilecache.org/" target="_blank">http://tilecache.org/</a>
Tilecache with the option metatiling<br>
enabled. With metatiling enabled Tilecache will fetch larger tiles (eg.<br>
4096x4096) from Mapserver and will cut them into smaller tiles (eg.<br>
256x256). It will also increase performance because Mapserver does not need<br>
to render tiles when they are already in cache.<br>
<br>
kaipi<br>
--<br>
View this message in context: <a href="http://osgeo-org.1803224.n2.nabble.com/WMS-and-tiles-cutting-off-tp4978418p4979280.html" target="_blank">http://osgeo-org.1803224.n2.nabble.com/WMS-and-tiles-cutting-off-tp4978418p4979280.html</a><br>
Sent from the Mapserver - User mailing list archive at Nabble.com.<br>
_______________________________________________<br>
mapserver-users mailing list</p>
</div>
<p class="MsoNormal"><a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><mailto:<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a>><mailto:<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><mailto:<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a>>></p>
<div>
<div>
<p class="MsoNormal" style="margin-bottom: 12pt;"><a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a></p>
</div>
</div>
</div>
<p class="MsoNormal"> </p>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"> </p>
</div></div></div>
</div>
</blockquote></div><br>