[MapServer-dev] Variables in mapfiles

[Lime, Steve D (MNIT)]

Hi Jukka: Yeah, this is a potential area for improvement. We've be using a couple of strategies where I work. One being includes plus runtime subs so you can embed replacement strings as necessary, similar to your examples. The environment file gets included in a validation block and you set both a default and validation patterns for each value:

  "dbhost_default" "some.database.host"
  "dbhost"                "^some\.database\.host$"

So dbhost can only be the one value. I feels cleaner just write things the way you suggest. However, it feels wrong to have to set a value twice - and is potentially error prone in a bad way. Plus, there's no connection to the environment. I'm not a huge fan.

The other approach is using a pre-processor to essentially compile a mapfile into a final version. We're using YAML + mappyfile to create an environment-specific version of the mapfile. The YAML definition can consist of constants plus references to environment variables. There is loads of potential here but deployment in different environments requires a compilation step plus a working Python environment.

So, I can see value in a VARIABLES block or reference to a file. Could also do that at the config file level instead. Beyond loading the hash somehow, one would need a method to apply the variables in a limited fashion, probably while the mapfile is being parsed. So, load the DATA value and then apply variables or something like that.

This warrants an RFC I think...

--Steve

From: MapServer-dev <mapserver-dev-bounces at lists.osgeo.org> On Behalf Of Rahkonen Jukka via MapServer-dev
Sent: Thursday, November 9, 2023 5:08 PM
To: 'mapserver-dev at lists.osgeo.org' <mapserver-dev at lists.osgeo.org>
Subject: [MapServer-dev] Variables in mapfiles

Hi,

When I was reading the old bug reports in the OSGeo code sprint this 20 years old one https://github.com/MapServer/MapServer/issues/408 from year 2003 did not feel rotten at all. And the last comment https://github.com/MapServer/MapServer/issues/408#issuecomment-1065080140 has received 5 thumbs up within 1 year which shows great user activity by our standards.

I think that environmental variables may be too strong tool for this purpose. For example, are those who write mapfiles allowed to set env variables in managed environments? Maybe something similar to SYMBOLSET and INCLUDE would be easier to use.
As an example, I found a pretty good bug report that includes a mapfile and data, but for testing the mapfile on my Windows machine I need to make a few edits.

CONFIG "PROJ_LIB" "/usr/local/share/proj"
                          I am on Windows and my proj is in some other place
WMS_ONLINERESOURCE http://odroid1:5001/
wms_service_onlineresource http://odroid1:5001/
                          I don't understand the difference between those two, but anyway I run Mapserver at http://localhost:8060/...
DATA "./bug-report.xml"
                          I prefer to keep data in different place than mapfiles
DATA "./data-in-31287"
                          Same for this shapefile data

What if we could write non-fixed paths and other items into the mapfile like

CONFIG "PROJ_LIB" "%proj_lib%"
WMS_ONLINERESOURCE "%onlineresource_test%"
wms_service_onlineresource "%onlineresource_test%"
DATA "%datapath_test%/bug-report.xml"
DATA "%datapath_test%/data-in-31287"
....
MAPVARIABLES "map_variables.txt"
END # mapfile

The contents of "map_variables.txt"

//Some magic
proj_lib                                      [value]
onlineresource_test                [value]
onlineresource_qa                  [value]
onlineresourse_production   [value]
datapath_test                           [value]
datapath_qa                             [value]
datapath_production             [value]
symbolset_topomap               [value]
symbolset_citymap                 [value]
etc.

A difference to INCLUDE is that while the whole include file is slipped inside the mapfile the variable file could be a dictionary. That would allow forwarding the mapfile from test to qa to production easily. And when the IP address of test servers change an edit in one place would be enough. It would still be possible to use separate "map_variables_testing.txt" for testing if it feels better.

I admit that there are alternative scenarios via INCLUDE or config file or environmental variables.

-Jukka Rahkonen-















-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20231117/1f509fb7/attachment.htm>