[MapServer-dev] Variables in mapfiles

Tue Dec 26 17:58:36 PST 2023

+1

..Tom

On Tue, Dec 26, 2023 at 8:02 PM Michael Smith via MapServer-dev <
mapserver-dev at lists.osgeo.org> wrote:

> I like it. Having more runtime config is always useful.
>
> Michael Smith
>
> On Dec 26, 2023, at 4:57 PM, Steve Lime via MapServer-dev <
> mapserver-dev at lists.osgeo.org> wrote:
>
> 
> As a follow up. My previous message was inaccurate and kind of dumb on my
> part. You only have to set a default value with no validation block for
> things to basically function as a variable with runtime subs. MapServer
> will not try to do a substitution if a validation expression is not set and
> will automatically fall back to the default value. So, you just need:
>
>   "variable_default" "some value"
>
> in a validation block and then you'd reference "%variable%" in supported
> locations. Passing variable= in the request will have no effect.
>
> If we allowed "some value" to reference an environment variable (maybe
> with a leading $ in the default value) then I could see where that might be
> a pretty useful way to externalize some configuration information. We might
> consider adding a VALIDATION block to the config file as well.
>
> --Steve
>
> On Fri, Nov 17, 2023 at 3:37 PM Lime, Steve D (MNIT) via MapServer-dev <
> mapserver-dev at lists.osgeo.org> wrote:
>
>> Hi Jukka: Yeah, this is a potential area for improvement. We’ve be using
>> a couple of strategies where I work. One being includes plus runtime subs
>> so you can embed replacement strings as necessary, similar to your
>> examples. The environment file gets included in a validation block and you
>> set both a default and validation patterns for each value:
>>
>>
>>
>>   “dbhost_default” “some.database.host”
>>
>>   “dbhost”                “^some\.database\.host$”
>>
>>
>>
>> So dbhost can only be the one value. I feels cleaner just write things
>> the way you suggest. However, it feels wrong to have to set a value twice –
>> and is potentially error prone in a bad way. Plus, there’s no connection to
>> the environment. I’m not a huge fan.
>>
>>
>>
>> The other approach is using a pre-processor to essentially compile a
>> mapfile into a final version. We’re using YAML + mappyfile to create an
>> environment-specific version of the mapfile. The YAML definition can
>> consist of constants plus references to environment variables. There is
>> loads of potential here but deployment in different environments requires a
>> compilation step plus a working Python environment.
>>
>>
>>
>> So, I can see value in a VARIABLES block or reference to a file. Could
>> also do that at the config file level instead. Beyond loading the hash
>> somehow, one would need a method to apply the variables in a limited
>> fashion, probably while the mapfile is being parsed. So, load the DATA
>> value and then apply variables or something like that.
>>
>>
>>
>> This warrants an RFC I think…
>>
>>
>>
>> --Steve
>>
>>
>>
>> *From:* MapServer-dev <mapserver-dev-bounces at lists.osgeo.org> *On Behalf
>> Of *Rahkonen Jukka via MapServer-dev
>> *Sent:* Thursday, November 9, 2023 5:08 PM
>> *To:* 'mapserver-dev at lists.osgeo.org' <mapserver-dev at lists.osgeo.org>
>> *Subject:* [MapServer-dev] Variables in mapfiles
>>
>>
>>
>> Hi,
>>
>>
>>
>> When I was reading the old bug reports in the OSGeo code sprint this 20
>> years old one https://github.com/MapServer/MapServer/issues/408 from
>> year 2003 did not feel rotten at all. And the last comment
>> https://github.com/MapServer/MapServer/issues/408#issuecomment-1065080140
>> has received 5 thumbs up within 1 year which shows great user activity by
>> our standards.
>>
>>
>>
>> I think that environmental variables may be too strong tool for this
>> purpose. For example, are those who write mapfiles allowed to set env
>> variables in managed environments? Maybe something similar to SYMBOLSET and
>> INCLUDE would be easier to use.
>>
>> As an example, I found a pretty good bug report that includes a mapfile
>> and data, but for testing the mapfile on my Windows machine I need to make
>> a few edits.
>>
>>
>>
>> CONFIG "PROJ_LIB" "/usr/local/share/proj"
>>
>>                           I am on Windows and my proj is in some other
>> place
>>
>> WMS_ONLINERESOURCE http://odroid1:5001/
>>
>> wms_service_onlineresource http://odroid1:5001/
>>
>>                           I don’t understand the difference between those
>> two, but anyway I run Mapserver at http://localhost:8060/...
>>
>> DATA "./bug-report.xml"
>>
>>                           I prefer to keep data in different place than
>> mapfiles
>>
>> DATA "./data-in-31287"
>>
>>                           Same for this shapefile data
>>
>>
>>
>> What if we could write non-fixed paths and other items into the mapfile
>> like
>>
>>
>>
>> CONFIG "PROJ_LIB" "%proj_lib%"
>>
>> WMS_ONLINERESOURCE "%onlineresource_test%"
>>
>> wms_service_onlineresource "%onlineresource_test%"
>>
>> DATA "%datapath_test%/bug-report.xml"
>>
>> DATA "%datapath_test%/data-in-31287"
>>
>> ….
>>
>> MAPVARIABLES "map_variables.txt"
>>
>> END # mapfile
>>
>>
>>
>> The contents of "map_variables.txt"
>>
>>
>>
>> //Some magic
>>
>> proj_lib                                      [value]
>>
>> onlineresource_test                [value]
>>
>> onlineresource_qa                  [value]
>>
>> onlineresourse_production   [value]
>>
>> datapath_test                           [value]
>>
>> datapath_qa                             [value]
>>
>> datapath_production             [value]
>>
>> symbolset_topomap               [value]
>>
>> symbolset_citymap                 [value]
>>
>> etc.
>>
>>
>>
>> A difference to INCLUDE is that while the whole include file is slipped
>> inside the mapfile the variable file could be a dictionary. That would
>> allow forwarding the mapfile from test to qa to production easily. And when
>> the IP address of test servers change an edit in one place would be enough.
>> It would still be possible to use separate "map_variables_testing.txt" for
>> testing if it feels better.
>>
>>
>>
>> I admit that there are alternative scenarios via INCLUDE or config file
>> or environmental variables.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> MapServer-dev mailing list
>> MapServer-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
> _______________________________________________
> MapServer-dev mailing list
> MapServer-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
> _______________________________________________
> MapServer-dev mailing list
> MapServer-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20231226/e8c069f6/attachment.htm>