mapserver initialization delays

Scott Melby smelby at TURNKEY-TECHNOLOGY.COM
Mon Jun 12 16:26:37 EDT 2006


Stephen -

Thanks for the response.  Although I agree that there is definitely 
something that is getting "evicted" from memory, it does not appear to 
me that the amount of pertinent data that is cached in memory is the 
issue.  The reason I say this, is that I can perform hundreds of 
queries, all with differing bounding boxes and differing dataFilter 
parameters (no result set overlap), and only the first query after a 
long period is slow.  So... unless the initial request is loading ALL 
data into memory (regardless of the FILTER and BBOX params), it seems 
that it would incur delays on some requests other than the original.  
Note: When I am using FILTER and BBOX params the problem is a bit less 
severe (40+ secs for initial draw, 3+ secs for subsequent is pretty 
typical).

Regardless of the cause of the delays, I really can't have my users 
hitting a 40+ second delay for an interactive query on the site.  Are 
there known strategies for dealing with large datasets?  The response 
time I get for all requests other than the first are completely 
reasonable.  So... my attempts thus far have been aimed at attempting to 
keep things from getting into this state via background hits.  This, 
however, has been unsuccessful for some unknown reason.  I can see that 
the web server is getting my requests (1 per minute) and I can also see 
that mapserver is returning a valid .PNG file for each request.

Thanks again
Scott

Stephen Woodbridge wrote:

> Scott,
>
> What you are likely seeing is the fact the the first long draw is that 
> postgres is finding all you data and it is getting cached in memory. 
> So the subsequent requests are all fetching the data from the cache. 
> If you wait more and more of the data gets bumped from the cache.
>
> There is very little mapserver specific startup costs. You might have 
> read about database connection costs and pooling, but that is not what 
> you are seeing.
>
> -Steve W.
>
> Scott Melby wrote:
>
>> Hello -
>>
>> I am pretty new to MapServer, so I apologize if I am re-hashing old 
>> material.  However, I am having a significant problem and am unable 
>> to find a "reasonable" solution documented anywhere.  I am running 
>> mapserver 4.6, and am serving a single layer that contains about 
>> 350,000 points from a PostGIS database.  The layer has a GiST index 
>> on the geometry column.  The following request 
>> http://www.geonomy.com/cgi-bin/mapserv?map=/var/www/geonomy/geonomy.map&layer=geonomy_features&mode=map&dataFilter=geometry%20is%20not%20null 
>> will draw all points in my table into a 256x256 RGB PNG file.  The 
>> issue I am having is that MapServer can take up to 3 minutes to draw 
>> the image the first time it is called(after any delay of 30+ minutes 
>> with no calls).  Once that initial draw completes, subsequent hits 
>> draw in about 9 seconds.  In my research I have found a page that 
>> indicates that this is due to MapServer initialization.  But the 
>> advice given there was to send a request that causes all map features 
>> to be drawn after setting up the server.... which has not solved my 
>> problems.  The delay does not appear to be tied to a particular 
>> client session... because hitting that same URL from another browser 
>> draws quickly if done within a few minutes of the initial query.  
>> However, if I wait long enough (seems to be about 20-30 minutes) the 
>> next request will once again take 2-3 minutes instead of 9 seconds.  
>> Note: I don't believe that the issue is due to efficiencies of 
>> drawing the same image or accessing the same data repeatedly, because 
>> I have found that altering the dataFilter parameter (which is passed 
>> to a FILTER on the layer in the map file) causes the image to change, 
>> but does not significantly alter the performance (filters with less 
>> points draw more quickly, but still take a relatively long time for 
>> initial query if delay is long enough before it is called).  I have 
>> looked at the mapserver demo_init.html file and have added code to my 
>> servlet that periodically performs the following get 
>> http://www.geonomy.com/cgi-bin/mapserv?map=/var/www/geonomy/geonomy.map&layer=geonomy_features&program=/cgi-bin/mapserv&map_web_imagepath=/tmp&map_web_imageurl=/tmp 
>> request every N minutes (intervals of 1 minute - 10 minutes)... but, 
>> this does not change the behavior.
>> I have also tried performing the following get at 1 minute intervals 
>> to try to avoid problems that could be caused by inactivity (such as 
>> the GiST index having to be reloaded from disk, or mapserver dropping 
>> its connection to PostGIS)..  In this case the GET request is being 
>> sent from a Java using a HttpURLConnection that is set to not use 
>> caches.  Note: I have also tried adding a BBOX to this request to 
>> make sure that the GiST index is being used.  However, this does not 
>> fix the problem either.
>>
>> http://www.geonomy.com/cgi-bin/mapserv?map=/var/www/geonomy/geonomy.map&REQUEST=GetMap&SERVICE=WMS&VERSION=1.1.1&LAYERS=geonomy_features&FORMAT=PNG&BGCOLOR=0xFFFFFF&TRANSPARENT=TRUE&SRS=EPSG:4326&BBOX=-115,30,-108,37&WIDTH=256&HEIGHT=256&reaspect=false&dataFilter=geometry%20is%20not%20null 
>>
>>
>> Any help is greatly appreciated.
>>
>> Thanks in advance
>> Scott
>>



More information about the mapserver-users mailing list