[mapserver-users] mapcache - transparency, surrounding area color, performance when assembling images (pink tiles)

Tue Jun 12 14:31:46 PDT 2012

Thomas,

I loaded up the demo mapcache.xml.  Even on the demo resampling was slow.  The only change I made to the html file was change the mapextent from: 
new OpenLayers.Bounds(-180.000000,-90.000000,180.000000,90.000000) 
to: 
new OpenLayers.Bounds(-179.235454,-89.456458,179.354812,89.545112)

So I don’t think the issue was with my mapcache.xml file.  

Mark Volz
GIS Specialist

-----Original Message-----
From: thomas bonfort [mailto:thomas.bonfort at gmail.com] 
Sent: Tuesday, June 12, 2012 1:48 PM
To: Mark Volz
Cc: MapserverList OSGEO
Subject: Re: [mapserver-users] mapcache - transparency, surrounding area color, performance when assembling images (pink tiles)

Something's definitely wrong with your setup :(
given your machine specs, typical timings should more likely be:
- for tiled requests, around .2-.5 ms/req
- for off-grid 256x256 requests, around 10-20 ms/req

what timings to you get if you request tiles / maps using the supplied
default mapcache.xml ?

On Tue, Jun 12, 2012 at 8:31 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
> Thomas
>
> Here are a couple examples of url calls made to mapcache.  In general most of them are formatted as such:
> http://10.0.8.116/mapcache?LAYER=&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&STYLES=&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&FORMAT=image%2Fjpeg&LAYERS=LyonAirPhotos&SRS=EPSG%3A103749&BBOX=509776,195448,511824,197496&WIDTH=256&HEIGHT=256
>
> Scale 1:6912 (same grid as mapcache):
> BBOX=524112,183160,526160,185208&WIDTH=256&HEIGHT=256
>
> Scale 1:7000 (slightly off from mapcache grid):
> BBOX=502444.44444451,182259.25925936,504518.51851858,184333.33333344&WIDTH=256&HEIGHT=256.
> When mapcache had to assemble tiles, all 4 cores of the server ran at 100%
>
> I also tried single tile mode to see if that made any difference.  It did.  Mapcache produced single tile images in about .8 to 1.5 seconds no matter if I am using the same grid as mapcache or not.  This speed obviously is not nearly as fast as tile mode when using the same grid as mapcache.  That being said, it seems like something is stopping mapcache from scaling well when it needs to resample multiple tiles at the same time.
>
> Mark Volz
> GIS Specialist
>
>
> -----Original Message-----
> From: thomas bonfort [mailto:thomas.bonfort at gmail.com]
> Sent: Tuesday, June 12, 2012 11:34 AM
> To: Mark Volz
> Cc: MapserverList OSGEO
> Subject: Re: [mapserver-users] mapcache - transparency, surrounding area color, performance when assembling images (pink tiles)
>
> Hi Mark,
>
> On Tue, Jun 12, 2012 at 6:28 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
>> Thomas,
>>
>> Our server is behind a firewall, but anyways here is some of the information that I can share which might help.
>>
>> Server: New Quad Core Xeon, 20 GB ram, 1Tb storage free, RAID 5
>> Tiles: jpeg (source, cache format, and format used in assembling
>> tiles)
>> Grid:  County Coordinates, 256*256, extent matches the extent of the
>> client application
>>
>> Zoom Level, Resolution, Approximate Scale (Res*12*72) 0, 1024,
>> 1:884736  (region) 1, 512 , 1:442368 2, 256 , 1:221184  (county wide)
>> 3, 128 , 1:110592 4, 64 , 1:55296 5, 32 , 1:27648 6, 16 , 1:13824 7, 8
>> , 1:6912 8, 4 , 1:3456 9, 2 , 1:1728 10, 1 , 1:864 11, 0.5 , 1:432
>> (0.5 feet matches the size of the imagery)
>>
>> Unscientific speed results for various assemble types:
>>
>
> Even if the server is behind a firewall, can you provide the urls for each of these test cases, I basically only need the width, height and bbox parameters (provided they are all requesting the same LAYERS=
> combination)
>
>> Source wms server: 400-800 ms
>> 1:6912 (resolution matching cache, area cached) 4-50 ms
>> 1:6912 (resolution matching cache, area not cached) Cache hits are
>> around .470-1.4 seconds
>> 1:7000 (forward to wms server) 500-800 ms
>> 1:7000 (assemble from cache on an area already cached) 1-12 seconds
>> (panning small amounts is quicker, zooming in our out takes a while)
>> 1:7000 (assemble from cache on an area not already cached) 2-14
>> seconds
>>
>> Thanks
>>
>> Mark Volz
>> GIS Specialist
>>
>> -----Original Message-----
>> From: thomas bonfort [mailto:thomas.bonfort at gmail.com]
>> Sent: Tuesday, June 12, 2012 5:27 AM
>> To: Mark Volz
>> Cc: MapserverList OSGEO
>> Subject: Re: [mapserver-users] mapcache - transparency, surrounding
>> area color, performance when assembling images (pink tiles)
>>
>> (please stay on list, you answers can help others facing the same
>> issue)
>>
>> On Mon, Jun 11, 2012 at 9:26 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
>>> Thomas,
>>>
>>> It turns out that most of my problems were when <full_wms>assemble</full_wms> was set...  apparently it defaults to jpeg when mapcache has to assemble, which was causing problems with my png caches.  I experimented with changing the default to PNG.
>>>
>>> I think my resolutions are set (somewhat) appropriately, for testing I am using resolutions of 1024 512 256 128 64 32 16 8 4 2 1.0 and .5.  when I am browsing at one of these resolutions mapcache is very fast.  It just seems that mapcache is slow when I am using a resolution that is slightly off of one of these.  Anyways, I just turned  forward on <full_wms>forward</full_wms>, which seems to be okay.
>>
>> Slightly off is expected to be a bit slower, but not very noticeably so.  Please to try again with some higher resolutions configured (the disk space overhead for those will be negligeable), mapcache assembling should be at least as fast as a normal wms server if it is allowed to hit a correctly configured grid resolution. For reference, the classical mercator grid has 7 higher resolutions than what you have configured, which will help for requests covering approximately state/country scaled getmaps.
>> Can you also provide the full url for a getmap request that exhibits this behavior, along with timings comparing "assemble" and "forward"
>> modes.
>>
>> regards,
>> thomas
>>
>>
>>
>>>
>>> Thank you for all your help!
>>>
>>>
>>> Mark Volz
>>> GIS Specialist
>>>
>>> -----Original Message-----
>>> From: thomas bonfort [mailto:thomas.bonfort at gmail.com]
>>> Sent: Monday, June 11, 2012 9:47 AM
>>> To: Mark Volz
>>> Cc: mapserver-users at lists.osgeo.org
>>> Subject: Re: [mapserver-users] mapcache - transparency, surrounding
>>> area color, performance when assembling images (pink tiles)
>>>
>>> On Mon, Jun 11, 2012 at 4:32 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
>>>> Hello,
>>>>
>>>> First,  I am having trouble figuring out how to get transparency to
>>>> work in mapcache when using a Mapserver/WMS source.  Does anyone
>>>> have an example of how to store a transparent mapcache with a Mapserver wms source?
>>>
>>> make the tileset reference a png format, not a jpeg one.
>>>
>>>>
>>>> Next,  The color of my cache is fine, however the surrounding area
>>>> outside of the extent of the cache is black.  Can that be changed to
>>>> white, or transparent?
>>>
>>> same as previously, configure the wms service create png not jpeg
>>> <service type="wms" enabled="true">
>>>      ...
>>>      <format>PNG</format>
>>> </service>
>>>
>>>
>>>>
>>>> Third,  Mapcache seems really slow when it needs to assemble images.
>>>> I am getting plenty of pink screens, and the response time is
>>>> usually greater than 5 seconds.  Mapcache is much faster when I use
>>>> <full_wms>forward</full_wms>, so I know the problem is not the source wms.
>>>> Does anyone have an idea as to why the performance is poor when
>>>> assembling images?
>>> I suspect you have defined a <grid> who's resolutions aren't of the same order of magnitude as the resolution of the getmap requests, i.e.
>>> when creating the result image, mapcache has to decompress and resample a very high number of tile images. Try adding a couple of higher resolution entries to the beginning of your <resolutions> array.
>>>
>>> --
>>> thomas
>>>
>>>>
>>>> Thank You
>>>>
>>>> Mark Volz
>>>> GIS Specialist
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mapserver-users mailing list
>>>> mapserver-users at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>>>