[mapserver-users] [mapserver-dev] Setup for MVT (was: Time for a 7.2.2 release?)

Tue Feb 12 11:34:51 PST 2019

BTW my MVT demo does have a sample mapache config included now. Note that the MVT creation does do simplification so for example you'll see islands in lakes disappear as they become degenerate and points removed from lines, etc.... That's the fun part and hat you need for scanline conversion is a little different from what Open/Web GL needs.

-----Original Message-----
From: Stephen Woodbridge [mailto:stephenwoodbridge37 at gmail.com] 
Sent: Tuesday, February 12, 2019 12:45 PM
To: Lime, Steve D (MNIT) <steve.lime at state.mn.us>; mapserver-users <mapserver-users at lists.osgeo.org>
Subject: Re: [mapserver-dev] Setup for MVT (was: Time for a 7.2.2 release?)

Hi Steve,

Thanks!, I put this back on the list so other can benefit from your 
thoughts.

Tiles at various scales is not intuitive to me either. If you don't 
simplify the geometries, then all you are getting it more features in 
each tile as you zoom out. But your point about degenerate geometries 
play right into this and you can't really simplify them except via 
clustering which makes more sense on the client side.

I was thinking of using this for serving point features to an OL5 
application. Since my case was so simple, I ended up writing a trivial 
PHP handler to do this using a bbox strategy, but I want to revisit that 
using mapcache and MVT because I think it would be good to learn and I 
think it would be more efficient on both the client and server, not that 
I need to worry that much about that.

-Steve

On 2/12/2019 11:56 AM, Lime, Steve D (MNIT) wrote:
> I like the OL example as well. Years ago I created an initial run at that format using the CGI-based options against a common MapFile. It worked ok for that use case because all the differences between examples were configured completely on the client.
>
> 1) You actually don't need to define an output format - there's one set by default if the support is compiled in. You can override of course.
> 2) Configuration for MVT mirrors that of WMS (strangely) and WFS at least in terms of determining which feature attributes are exposed, so OWS_INCLUDE_ITEMS and OWS_[item name]_TYPE are about all you care about. This applies to either MapCache or mode=tile requests. GeoJSON and KML drivers behave similarly.
> 3) If using MapCache the requests are made using WMS so that protocol needs to be enabled in the MapFIle.
> 4) The other option is using the CGI and mode=tile. You can set the format either in the mapfile via IMAGETYPE or via URL using map_imagetype=mvt
> 5) The MVT format is layer-based so it maps nicely to MapServer. You can certainly mix geometry types across layers. I don't know that the format supports mixed geometry types within layers - MapServer doesn't so it's a moot point anyway.
> 6) MapServer is generating binary protobuf files so there isn't really a way to show sample outout.
> 7) The rendering process does respect MIN/MAXSCALEDENOM values
> 8) Features are converted from map coordinates to tile coordinates much the same way as you would for drawing in an images. Features are clipped and then sampled based on the tile extent and resolution. I think it's helpful to think of MVT features as almost pre-processed features suitable for drawing-only.
> 9) One difference from the vector->raster conversion and the vector->mvt conversion is that I was finding that the Mapbox GL client was pretty sensitive to degenerate geometries - specifically triangles that trend towards a point (area < 1) or a line (triangle height along the longest size < 1 pixel). The geometries are valid according to a few MVT utilities out there but Mapbox GL stops rendering the geometry as soon as it reaches a part like this. I've got a feeling this is related to Web GL itself. Anyway, MapServer 7.2.2 has some additional filtering in place to remove this portions of a geometry. I suspect we'll run into other issues like this as we gain experience.
>
> One thing I don't have good handle on is strategies to optimize tiles based on scale. For example, it's certainly possible to send full-resolution features at a certain zoom level and it would seem to me that you wouldn't want to keep sending them since the client already has as good a data as it can get. This is all probably heavily dependent on the behavior of the client and until folks get a chance to use this functionality we won't be able to provide much guidance.
>
> --Steve
>
> -----Original Message-----
> From: Stephen Woodbridge [mailto:stephenwoodbridge37 at gmail.com]
> Sent: Thursday, February 07, 2019 9:06 PM
> To: Lime, Steve D (MNIT) <steve.lime at state.mn.us>; mapserver-dev at lists.osgeo.org
> Subject: Re: [mapserver-dev] Time for a 7.2.2 release?
>
> Steve L,
>
> I like what Openlayers has in the way of examples but that would be a
> major undertaking so I think it is beyond the scope of things. I have
> found the  Mapserver OGC pages to be very help as an example. For this
> specific case of MVT tiles, a simple mapfile with the appropriate bits
> commented. And for mapcache, and a sample mapcache.xml referencing the
> sample mapfile should do it. I would think this could all be done on a
> single page and added to the documentation.
>
> For the mapfile, I presume you need to setup an output format, and then
> there is probably some metadata that needs to be defined to tell what
> properties should be included with the data. Is this on each layer? What
> happens if multiple layers are requested with different geometry types?
> you get back a FeatureCollection?  What does a request look like, is
> this a standard WMS or WFS request? I'm guessing a WMS because that is
> what mapcache expects.  A sample geojson output might be helpful. I'm
> just tossing out ideas here that I think might be helpful to someone not
> familiar with it and trying to make use of it.
>
> If you want to send me some info on/off list, I will try to organize it
> and put together markdown page and send that back to you.
>
> -Steve W
>
> On 2/6/2019 5:45 PM, Lime, Steve D (MNIT) wrote:
>> I'll update the RFC. At a minimum it's missing the voting history.
>>
>> As for other documentation we’d need to create a new page I guess that would be more task-based (e.g. I want to serve vector tiles using MapServer and MapCache). We don't do much of that and perhaps we should. There really aren't any new configuration options, heck the MVT output format is built in so as long as the support is compiled in there isn't much to it. Steve, maybe you could put together an outline of what your ideal "how to" might contain?
>>
>> --Steve
>>
>> -----Original Message-----
>> From: mapserver-dev [mailto:mapserver-dev-bounces at lists.osgeo.org] On Behalf Of Stephen Woodbridge
>> Sent: Wednesday, February 06, 2019 9:31 AM
>> To: mapserver-dev at lists.osgeo.org
>> Subject: Re: [mapserver-dev] Time for a 7.2.2 release?
>>
>> It would be nice if RFC 119 on MVT Tiles was updated to indicate it is
>> release. I'm not sure if there is any docs, I did go looking and the
>> only thing I found was references to SteveL's github stuff, but it would
>> be nice is there was an example showing how to setup mapserver and
>> mapcache serving MVT Tiles.
>>
>> Thanks,
>>      -Steve W
>>
>> On 2/6/2019 9:47 AM, Lime, Steve D (MNIT) wrote:
>>> I had mentioned a mid-January release a while back so we’re a tad
>>> late. There a couple of MVT-related bug fixes, among other things,
>>> that would be nice to release. Anything folks were hoping to see included?
>>>
>>> Otherwise I’ll offer a +1 for a 7.2.2 release in the next week. Jeff
>>> has graciously offered to help...
>>>
>>> —Steve
>>>
>>> _______________________________________________
>>> mapserver-dev mailing list
>>> mapserver-dev at lists.osgeo.org
>>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C496918c1609c48fb50c308d6911a2738%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636855938851058102&sdata=3mhmfn%2Fz%2B9gDWoJ%2Brm8Ln8a8lAvavC549FCS6K421lg%3D&reserved=0
>> ---
>> This email has been checked for viruses by Avast antivirus software.
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.avast.com%2Fantivirus&data=02%7C01%7Csteve.lime%40state.mn.us%7C496918c1609c48fb50c308d6911a2738%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636855938851058102&sdata=k4r%2BcFoBqk11Y%2B20foTd3fbGoNX68sKCTRFXrQ5ufx0%3D&reserved=0
>>
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C496918c1609c48fb50c308d6911a2738%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636855938851068104&sdata=nFdif%2FmbfivEu9E%2B29W6kPQsJaQ0EZ%2BAL4ebI1OvpLc%3D&reserved=0
>
> ---
> This email has been checked for viruses by Avast antivirus software.
> https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.avast.com%2Fantivirus&data=02%7C01%7Csteve.lime%40state.mn.us%7C496918c1609c48fb50c308d6911a2738%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636855938851068104&sdata=dbwm5xtyBZm2D4MRBTM2fI2zBA9JIFKjWaKdX2pYhYs%3D&reserved=0
>

---
This email has been checked for viruses by Avast antivirus software.
https://na01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.avast.com%2Fantivirus&data=02%7C01%7Csteve.lime%40state.mn.us%7C496918c1609c48fb50c308d6911a2738%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636855938851068104&sdata=dbwm5xtyBZm2D4MRBTM2fI2zBA9JIFKjWaKdX2pYhYs%3D&reserved=0