[mapserver-users] mapcache - transparency, surrounding area color, performance when assembling images (pink tiles)

Tue Jun 12 09:44:34 PDT 2012

On Tue, Jun 12, 2012 at 6:33 PM, thomas bonfort
<thomas.bonfort at gmail.com> wrote:
> Hi Mark,
>
> On Tue, Jun 12, 2012 at 6:28 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
>> Thomas,
>>
>> Our server is behind a firewall, but anyways here is some of the information that I can share which might help.
>>
>> Server: New Quad Core Xeon, 20 GB ram, 1Tb storage free, RAID 5
>> Tiles: jpeg (source, cache format, and format used in assembling tiles)
>> Grid:  County Coordinates, 256*256, extent matches the extent of the client application
>>
>> Zoom Level, Resolution, Approximate Scale (Res*12*72)
>> 0, 1024, 1:884736  (region)
>> 1, 512 , 1:442368
>> 2, 256 , 1:221184  (county wide)
>> 3, 128 , 1:110592
>> 4, 64 , 1:55296
>> 5, 32 , 1:27648
>> 6, 16 , 1:13824
>> 7, 8 , 1:6912
>> 8, 4 , 1:3456
>> 9, 2 , 1:1728
>> 10, 1 , 1:864
>> 11, 0.5 , 1:432  (0.5 feet matches the size of the imagery)
>>
>> Unscientific speed results for various assemble types:
>>
>
> Even if the server is behind a firewall, can you provide the urls for
> each of these test cases, I basically only need the width, height and
> bbox parameters (provided they are all requesting the same LAYERS=
> combination)
>
>> Source wms server: 400-800 ms
>> 1:6912 (resolution matching cache, area cached) 4-50 ms
>> 1:6912 (resolution matching cache, area not cached) Cache hits are around .470-1.4 seconds
>> 1:7000 (forward to wms server) 500-800 ms
>> 1:7000 (assemble from cache on an area already cached) 1-12 seconds (panning small amounts is quicker, zooming in our out takes a while)

For reference, on a single core of a Core i5 cpu, when requesting a
~1200x700 image compositing 2 layers (i.e. around 40-50 tiles to
decompress+reassemble+recompress), I'm getting 6 req/s, which seems
very far off from what you are getting with a much higher spec'd
machine.

>> 1:7000 (assemble from cache on an area not already cached) 2-14 seconds
>>
>> Thanks
>>
>> Mark Volz
>> GIS Specialist
>>
>> -----Original Message-----
>> From: thomas bonfort [mailto:thomas.bonfort at gmail.com]
>> Sent: Tuesday, June 12, 2012 5:27 AM
>> To: Mark Volz
>> Cc: MapserverList OSGEO
>> Subject: Re: [mapserver-users] mapcache - transparency, surrounding area color, performance when assembling images (pink tiles)
>>
>> (please stay on list, you answers can help others facing the same issue)
>>
>> On Mon, Jun 11, 2012 at 9:26 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
>>> Thomas,
>>>
>>> It turns out that most of my problems were when <full_wms>assemble</full_wms> was set...  apparently it defaults to jpeg when mapcache has to assemble, which was causing problems with my png caches.  I experimented with changing the default to PNG.
>>>
>>> I think my resolutions are set (somewhat) appropriately, for testing I am using resolutions of 1024 512 256 128 64 32 16 8 4 2 1.0 and .5.  when I am browsing at one of these resolutions mapcache is very fast.  It just seems that mapcache is slow when I am using a resolution that is slightly off of one of these.  Anyways, I just turned  forward on <full_wms>forward</full_wms>, which seems to be okay.
>>
>> Slightly off is expected to be a bit slower, but not very noticeably so.  Please to try again with some higher resolutions configured (the disk space overhead for those will be negligeable), mapcache assembling should be at least as fast as a normal wms server if it is allowed to hit a correctly configured grid resolution. For reference, the classical mercator grid has 7 higher resolutions than what you have configured, which will help for requests covering approximately state/country scaled getmaps.
>> Can you also provide the full url for a getmap request that exhibits this behavior, along with timings comparing "assemble" and "forward"
>> modes.
>>
>> regards,
>> thomas
>>
>>
>>
>>>
>>> Thank you for all your help!
>>>
>>>
>>> Mark Volz
>>> GIS Specialist
>>>
>>> -----Original Message-----
>>> From: thomas bonfort [mailto:thomas.bonfort at gmail.com]
>>> Sent: Monday, June 11, 2012 9:47 AM
>>> To: Mark Volz
>>> Cc: mapserver-users at lists.osgeo.org
>>> Subject: Re: [mapserver-users] mapcache - transparency, surrounding
>>> area color, performance when assembling images (pink tiles)
>>>
>>> On Mon, Jun 11, 2012 at 4:32 PM, Mark Volz <MarkVolz at co.lyon.mn.us> wrote:
>>>> Hello,
>>>>
>>>> First,  I am having trouble figuring out how to get transparency to
>>>> work in mapcache when using a Mapserver/WMS source.  Does anyone have
>>>> an example of how to store a transparent mapcache with a Mapserver wms source?
>>>
>>> make the tileset reference a png format, not a jpeg one.
>>>
>>>>
>>>> Next,  The color of my cache is fine, however the surrounding area
>>>> outside of the extent of the cache is black.  Can that be changed to
>>>> white, or transparent?
>>>
>>> same as previously, configure the wms service create png not jpeg
>>> <service type="wms" enabled="true">
>>>      ...
>>>      <format>PNG</format>
>>> </service>
>>>
>>>
>>>>
>>>> Third,  Mapcache seems really slow when it needs to assemble images.
>>>> I am getting plenty of pink screens, and the response time is usually
>>>> greater than 5 seconds.  Mapcache is much faster when I use
>>>> <full_wms>forward</full_wms>, so I know the problem is not the source wms.
>>>> Does anyone have an idea as to why the performance is poor when
>>>> assembling images?
>>> I suspect you have defined a <grid> who's resolutions aren't of the same order of magnitude as the resolution of the getmap requests, i.e.
>>> when creating the result image, mapcache has to decompress and resample a very high number of tile images. Try adding a couple of higher resolution entries to the beginning of your <resolutions> array.
>>>
>>> --
>>> thomas
>>>
>>>>
>>>> Thank You
>>>>
>>>> Mark Volz
>>>> GIS Specialist
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> mapserver-users mailing list
>>>> mapserver-users at lists.osgeo.org
>>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>>>