[mapserver-users] mamcache vertical assembly

[Stephen Woodbridge]

On 2/5/2019 8:52 PM, Richard Greenwood wrote:
> Stephen,
>
> I was watching this thread hoping for a reply to your original 
> question because it's an interesting problem. If you have time to 
> provide a more detailed explanation of your solution I'd be interested.

So what I had originally were 7 layers of NOAA ENC (Electronic Nautical 
Charts) being pulled from the NOAA WMS service and these were getting 
cached locally in mapcache. Since my app was always turning on/off all 
seven layers as a set, I wanted to build a new layer that merged all 
seven into a single layer. Part of the reason for this was to reduce the 
number of layer in openlayers for performance improvements. This would 
be just a vertical stacking of the the seven layers into one. Because of 
the way NOAA service is structured each layer has a different end point 
so it was not possible to have their server do the stacking.

I eventually found this post:
https://lists.osgeo.org/pipermail/mapserver-users/2012-April/072045.html
which suggests, this might no be the best approach.

In addition, while looking at this issue I realized the NOAA service 
changed it layer definitions for some of the data and my mapcache stuff 
was broken as a result. In the end, I decided to create a mapserver 
mapfile that did WMS client requests to the NOAA server and broke up the 
seven layers into 21 (7 each area, line and point) smaller layers that 
allowed for more correct stacking of the data and setting appropriate 
MIN/MAX scaledenom  values as recommended by the getCapabilities for the 
NOAA services. Now I have one mapcache layer that uses the new mosaic 
mapfile and this does the vertical stacking which is slow given that 
mapserver has to make about 21 WMS requests to serve each metatile 
request from mapcache. But the end result is good and tiles are fast 
once they are populated.

Also, I will note that the NOAA ENC service updates the chart data on a 
regular basis, so I expire my tiles in the cache after 7 days so they 
get regenerated possible using new data. I think because of this, the 
strategy of using mapserver to do the stacking and mapcache to handle 
the caching is a better division of labor and once the app is converted 
to use the new mosaic layer, I'll remove the 7 old layers to free up 
disk space.

Hope this helps someone with a similar issue.

Best regards,
   -Steve W

>
> Best regards,
> Rich
>
>
> On Tue, Feb 5, 2019 at 4:38 PM Stephen Woodbridge 
> <stephenwoodbridge37 at gmail.com <mailto:stephenwoodbridge37 at gmail.com>> 
> wrote:
>
>     I found response to question by Thomas on this subject that
>     suggested it might cause a deadlock if you ran out of connections.
>
>     Anyway I decided to build a new map file to do the vertical
>     assembly and then cache that through mapcache.
>
>     -Steve
>
>     Sent from my iPhone
>
>     On Jan 27, 2019, at 11:35 PM, Lime, Steve D (MNIT)
>     <steve.lime at state.mn.us <mailto:steve.lime at state.mn.us>> wrote:
>
>>     I’ll try to derive something from the source and report back.
>>     ------------------------------------------------------------------------
>>     *From:* mapserver-users <mapserver-users-bounces at lists.osgeo.org
>>     <mailto:mapserver-users-bounces at lists.osgeo.org>> on behalf of
>>     Stephen Woodbridge <stephenwoodbridge37 at gmail.com
>>     <mailto:stephenwoodbridge37 at gmail.com>>
>>     *Sent:* Sunday, January 27, 2019 7:08:38 PM
>>     *To:* mapserver-users
>>     *Subject:* [mapserver-users] mamcache vertical assembly
>>     Hi,
>>
>>     The mapcache docs has a page for vertical assembly but it is blank :(
>>
>>     I have 7 layers in mapcache and I would like to vertically assemble
>>     these into a composite layer. In the past I have been loading all
>>     7 in
>>     openlayers but it makes more sense to just composite them into a new
>>     layer. There are some scale limits that apply to the various layers.
>>
>>     Does anyone have some info on how to do this in mapcache directly
>>     rather
>>     than create a mapfile and compositing them in mapserver and then
>>     making
>>     that a new layer in mapcache.
>>
>>     Thanks,
>>        -Steve W
>>
>>     ---
>>     This email has been checked for viruses by Avast antivirus software.
>>     https://www.avast.com/antivirus
>>
>>     _______________________________________________
>>     mapserver-users mailing list
>>     mapserver-users at lists.osgeo.org
>>     <mailto:mapserver-users at lists.osgeo.org>
>>     https://lists.osgeo.org/mailman/listinfo/mapserver-users
>     _______________________________________________
>     mapserver-users mailing list
>     mapserver-users at lists.osgeo.org
>     <mailto:mapserver-users at lists.osgeo.org>
>     https://lists.osgeo.org/mailman/listinfo/mapserver-users
>
>
>
> -- 
> Richard W. Greenwood, PLS
> www.greenwoodmap.com <http://www.greenwoodmap.com>