[MapServer-dev] Variables in mapfiles

Fri Dec 29 08:02:01 PST 2023

Default values in validation blocks behave like variables when there is no
corresponding regex entry. So you'd get the best of both...

On Wed, Dec 27, 2023 at 7:42 PM Richard Greenwood <
richard.greenwood at gmail.com> wrote:

> I may be missing the point, but it seems like it would be useful to have
> variables in a MAP file. As I understand it, the CONFIG file is
> server-wide. If there are multiple MAP files for multiple projects on the
> same server they would all be sharing the same variables. Obviously the
> variables could be uniquely named for each project. But what about an
> include file that was used in multiple projects in which you wanted a
> variable that was set in the MAP file that included it? For example, a
> layer definition in an include file that was used in multiple projects in
> which you wanted to have a CONNECTION or DATA variable that was specific to
> the project.
>
> Rich
>
> On Wed, Dec 27, 2023 at 2:52 AM Seth G via MapServer-dev <
> mapserver-dev at lists.osgeo.org> wrote:
>
>> Hi Steve,
>>
>> I like the idea of a VARIABLES block in the CONFIG file. I'm not sure a
>> VALIDATION block would also be required - if a user can edit a CONFIG file
>> they'd have to assume responsibility for any values set.
>> As a first step of processing the Mapfile any $varName in a VARIABLE
>> block default variable value would be replaced by the values in the CONFIG
>> file. This should require no changes to the Mapfile syntax.
>> I'd prefer to avoid using environment variables for values as these are
>> very much platform and web server dependent.
>>
>> Seth
>>
>> --
>> web:https://geographika.net & https://mapserverstudio.net
>> twitter: @geographika
>>
>> On Tue, Dec 26, 2023, at 10:57 PM, Steve Lime via MapServer-dev wrote:
>>
>> As a follow up. My previous message was inaccurate and kind of dumb on my
>> part. You only have to set a default value with no validation block for
>> things to basically function as a variable with runtime subs. MapServer
>> will not try to do a substitution if a validation expression is not set and
>> will automatically fall back to the default value. So, you just need:
>>
>>   "variable_default" "some value"
>>
>> in a validation block and then you'd reference "%variable%" in supported
>> locations. Passing variable= in the request will have no effect.
>>
>> If we allowed "some value" to reference an environment variable (maybe
>> with a leading $ in the default value) then I could see where that might be
>> a pretty useful way to externalize some configuration information. We might
>> consider adding a VALIDATION block to the config file as well.
>>
>> --Steve
>>
>> On Fri, Nov 17, 2023 at 3:37 PM Lime, Steve D (MNIT) via MapServer-dev <
>> mapserver-dev at lists.osgeo.org> wrote:
>>
>> Hi Jukka: Yeah, this is a potential area for improvement. We’ve be using
>> a couple of strategies where I work. One being includes plus runtime subs
>> so you can embed replacement strings as necessary, similar to your
>> examples. The environment file gets included in a validation block and you
>> set both a default and validation patterns for each value:
>>
>>
>>
>>   “dbhost_default” “some.database.host”
>>
>>   “dbhost”                “^some\.database\.host$”
>>
>>
>>
>> So dbhost can only be the one value. I feels cleaner just write things
>> the way you suggest. However, it feels wrong to have to set a value twice –
>> and is potentially error prone in a bad way. Plus, there’s no connection to
>> the environment. I’m not a huge fan.
>>
>>
>>
>> The other approach is using a pre-processor to essentially compile a
>> mapfile into a final version. We’re using YAML + mappyfile to create an
>> environment-specific version of the mapfile. The YAML definition can
>> consist of constants plus references to environment variables. There is
>> loads of potential here but deployment in different environments requires a
>> compilation step plus a working Python environment.
>>
>>
>>
>> So, I can see value in a VARIABLES block or reference to a file. Could
>> also do that at the config file level instead. Beyond loading the hash
>> somehow, one would need a method to apply the variables in a limited
>> fashion, probably while the mapfile is being parsed. So, load the DATA
>> value and then apply variables or something like that.
>>
>>
>>
>> This warrants an RFC I think…
>>
>>
>>
>> --Steve
>>
>>
>>
>> *From:* MapServer-dev <mapserver-dev-bounces at lists.osgeo.org> *On Behalf
>> Of *Rahkonen Jukka via MapServer-dev
>> *Sent:* Thursday, November 9, 2023 5:08 PM
>> *To:* 'mapserver-dev at lists.osgeo.org' <mapserver-dev at lists.osgeo.org>
>> *Subject:* [MapServer-dev] Variables in mapfiles
>>
>>
>>
>> Hi,
>>
>>
>>
>> When I was reading the old bug reports in the OSGeo code sprint this 20
>> years old one https://github.com/MapServer/MapServer/issues/408 from
>> year 2003 did not feel rotten at all. And the last comment
>> https://github.com/MapServer/MapServer/issues/408#issuecomment-1065080140
>> has received 5 thumbs up within 1 year which shows great user activity by
>> our standards.
>>
>>
>>
>> I think that environmental variables may be too strong tool for this
>> purpose. For example, are those who write mapfiles allowed to set env
>> variables in managed environments? Maybe something similar to SYMBOLSET and
>> INCLUDE would be easier to use.
>>
>> As an example, I found a pretty good bug report that includes a mapfile
>> and data, but for testing the mapfile on my Windows machine I need to make
>> a few edits.
>>
>>
>>
>> CONFIG "PROJ_LIB" "/usr/local/share/proj"
>>
>>                           I am on Windows and my proj is in some other
>> place
>>
>> WMS_ONLINERESOURCE http://odroid1:5001/
>>
>> wms_service_onlineresource http://odroid1:5001/
>>
>>                           I don’t understand the difference between those
>> two, but anyway I run Mapserver at http://localhost:8060/...
>>
>> DATA "./bug-report.xml"
>>
>>                           I prefer to keep data in different place than
>> mapfiles
>>
>> DATA "./data-in-31287"
>>
>>                           Same for this shapefile data
>>
>>
>>
>> What if we could write non-fixed paths and other items into the mapfile
>> like
>>
>>
>>
>> CONFIG "PROJ_LIB" "%proj_lib%"
>>
>> WMS_ONLINERESOURCE "%onlineresource_test%"
>>
>> wms_service_onlineresource "%onlineresource_test%"
>>
>> DATA "%datapath_test%/bug-report.xml"
>>
>> DATA "%datapath_test%/data-in-31287"
>>
>> ….
>>
>> MAPVARIABLES "map_variables.txt"
>>
>> END # mapfile
>>
>>
>>
>> The contents of "map_variables.txt"
>>
>>
>>
>> //Some magic
>>
>> proj_lib                                      [value]
>>
>> onlineresource_test                [value]
>>
>> onlineresource_qa                  [value]
>>
>> onlineresourse_production   [value]
>>
>> datapath_test                           [value]
>>
>> datapath_qa                             [value]
>>
>> datapath_production             [value]
>>
>> symbolset_topomap               [value]
>>
>> symbolset_citymap                 [value]
>>
>> etc.
>>
>>
>>
>> A difference to INCLUDE is that while the whole include file is slipped
>> inside the mapfile the variable file could be a dictionary. That would
>> allow forwarding the mapfile from test to qa to production easily. And when
>> the IP address of test servers change an edit in one place would be enough.
>> It would still be possible to use separate "map_variables_testing.txt" for
>> testing if it feels better.
>>
>>
>>
>> I admit that there are alternative scenarios via INCLUDE or config file
>> or environmental variables.
>>
>>
>>
>> -Jukka Rahkonen-
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> MapServer-dev mailing list
>> MapServer-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
>> _______________________________________________
>> MapServer-dev mailing list
>> MapServer-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
>>
>> _______________________________________________
>> MapServer-dev mailing list
>> MapServer-dev at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
>>
>
>
> --
> Richard W. Greenwood
> www.greenwoodmap.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20231229/a5fd62eb/attachment.htm>