[mapserver-users] New limitations with mapserver 5 CGI

[Steve Lime]

We aren't limited solely to the lexer. DATA and TEMPLATE, the two most widely used CGI
overrides have a second control: DATAPATTERN and TEMPLATEPATTERN. When in a particular
parsing state we check the appropriate pattern (which NULL, or unmatchable, by default) to
decide if we should accept the override or not. The parameters at issue are those that aren't
held to any validation by the lexer. Colors, sizes and such all have to be of a certain format
or data type. Others like filters, data paths, connection strings etc... aren't subject to 
validation. We could impose validation like DATAPATTERN or TEMPLATEPATTERN on those
parameters specifically. I wouldn't want to add a boat load of new parameters but we could
do something like this in layer metadata:

METADATA
  data_validation_pattern 'my pattern'
  filter_validation_pattern 'another pattern'
  ...
END

We already use this scheme for validation of runtime substitutions and query strings for attribute
tables. This way the user could control which of these parameters could be overridden at runtime.
All parameters that are subject to validation as part of mapfile parsing would be web accessible
by default. Other parameters that we never want to allow override of can still be controlled within
the lexer.

The METADATA code would need special provisions to never allow setting keys ending in 
"_validation_pattern" from a URL source.

DATAPATTERN and TEMPLATEPATTERN would become deprecated.

Steve

>>> On 7/10/2008 at 10:19 PM, in message
<30fe546d0807102019y52beb686oe3ba5ba311cbee49 at mail.gmail.com>, "Paul Ramsey"
<pramsey at cleverelephant.ca> wrote:
> I'd like to see backwards feature compatibility available, and I think
> the CGI override controller is one of the most powerful
> (under-appreciated) features of Mapserver.
> 
> Since the control of what is over-rideable is in the lexer though,
> we're sort of in a narrow box. We could do a compile-time option,
> which would allow us to default to strict but allow (relatively) easy
> conversion to flexible. That wouldn't really help our Windows brethren
> though, since they mostly rely on pre-compiled versions, and binary
> maintainers don't maintain lots of different builds.
> 
> Having a map-level run-time directive to control the availability of
> URL over-rides would be the ideal, IMO.  Default, NONE.  Other
> options, STRICT and FULL.  However, I don't see how to get there from
> here with the lexer requirement.
> 
> P.
> 
> On Thu, Jul 10, 2008 at 6:05 PM, Graeme Watmuff
> <gwatmuff at geographicweb.com.au> wrote:
>> Hi Steve,
>>
>> Thanks for replying and thanks for the coding pointers. You are right
>> about the mapserv 4.10 tileindex syntax. Like filter, it is simply
>> ignored by v5 rather than throwing an error. My apologies.
>>
>> Would it make sense to create a 'secure' version for the paranoid and a
>> another version with full functionality? I realize this would create
>> more maintenance work for you, but not all mapserver users are going to
>> be able to climb inside the code themselves to modify functionality
>> (fortunately we can).
>>
>> Regarding security, I presume you are concerned about mapserver exposing
>> sensitive data to unintended users. If so, should you be taking that
>> responsibility on yourself by limiting mapserver or should the onus be
>> upon mapserver users to manage their own data more responsibly? I am a
>> great believer in personal responsibility. If the security risk is made
>> clear to users, they can make their own decisions as to the data they
>> potentially expose to the world.
>>
>> We have greatly appreciated and benefited from the tremendous work you
>> and your associates have done in bringing mapserver to the world and we
>> have always looked forward to the added features of each new version.
>> But upgrading now comes at some degree of inconvenience for us and maybe
>> others unless someone wants to publish the workaround to resurrect lost
>> functionality for all to simply acquire.
>>
>> I am not convinced that limiting the previous functionality is the right
>> direction for mapserver unless you fear say a litigious threat from
>> security breaches that people may bring upon themselves (hardly your
>> responsibility). Those with seriously sensitive data are not going to
>> want expose the CGI to all the world via a browser-sent URL anyway. I
>> would expect them to send URLs from say, a java web app embedded in a
>> secured tomcat container.
>>
>> Just my thoughts.
>>
>> Regards and best wishes
>>
>> Graeme
>>
>> On Thu, 2008-07-10 at 17:13 -0500, Steve Lime wrote:
>>> Hi Graeme: Comments inline... I definitely underestimated the impacts of 
> those
>>> changes. Anyway...
>>>
>>> >>> On 7/9/2008 at 11:23 PM, in message
>>> <1215663787.2513.112.camel at bongolx.ringo.net>, Graeme Watmuff
>>> <gwatmuff at geographicweb.com.au> wrote:
>>> > It has become apparent through some frustrating moments of trial and
>>> > error that many of the mapfile variables that were changeable via URL
>>> > with mapserver 4.10.x CGI seem no longer changeable with v5.x or 5.2.x.
>>> > There is a hint from the docs that this is because of security concerns?
>>> > Our java apps don't allow web users to directly interact with the CGI,
>>> > so security doesn't seem so important in our case.
>>>
>>> Allowing unaltered, direct modification of things like FILTERs or 
> CONNECTIONS
>>> is a potential security risk although I've never run across a documented 
> case. Filters
>>> are risky because they are passed to the underlying data driver which 
> generally
>>> trust the contents.
>>>
>>> > Some examples:
>>> >
>>> > map_web_metadata='text1' 'text2'
>>> > map_layerName_filter=<expression>
>>> > map_layerName_tileindex=<tileindex.shp>
>>> > map_layerName_data=<datasource>
>>> >
>>> > allow changes to the mapfile via URL with mapserver 4.10.x
>>> >
>>> > Following the new syntax specified for v5 + I had expected
>>> >
>>> > map_web=metadata+"'text1' 'text2'" or maybe
>>> > map_web_metadata[0]='text1'+'text2',
>>> > map_layer[layerName]=filter+<expression>,
>>> > map_layer[layerName]=tileindex+<tileindex.shp>,
>>> > map_layer[layerName]=data+<datasource>
>>> >
>>> > to all perform in the time-honoured way.
>>>
>>> And they don't... If they did you could combine modifications to a layer in 
> a single
>>> variable:
>>>
>>>   
> map_layer[layerName]=filter+<expression>+tileindex+<tileindex.shp>+data+<datasou
> rce>
>>>
>>> > Only 'data' could be changed with this new mapserver 5 syntax. I
>>> > happened to discover through the change logs that changing tileindex was
>>> > reinstated for mapserver 5 CGI, but using the old mapserver 4.10 syntax
>>> > (map_layerName_tileindex)=<tileindex.shp> - not the new mapserver 5
>>> > syntax style (map_layer[layerName]=tileindex+<tileindex.shp>).
>>>
>>> Looking at the code, TILEINDEX is not changable via a URL with any syntax. 
> The
>>> change log references a bug fix to 4.10. The old syntax doesn't work at all.
>>>
>>> > mapserv 5 CGI throws an error (Parsing error near (filter)) when the new
>>> > syntax is used for filter and ignores the old 4.10 filter syntax.
>>>
>>> This is expected. It will fail with an error as opposed to the silent 
> failure of versions
>>> past.
>>>
>>> > No URL syntax for changing the web object's metadata content appears
>>> > acceptable to the mapserver 5.
>>>
>>> Correct.
>>>
>>> > I would dearly like to see changing the layer filter via URL restored
>>> > also to mapserver 5. And the web object's metadata setting functionality
>>> > too.
>>>
>>> It's possible to customize this behavior by editing maplexer.l and 
> recompiling
>>> the code. Simply find the keyword you're interested in supporting and change
>>> the states at which it is valid. So, for TILEINDEX:
>>>
>>> <INITIAL>tileindex
>>>
>>> becomes:
>>>
>>> <INITIAL,URL_STRING>tileindex
>>>
>>> Same goes for the other parameters you'd want to activate.
>>>
>>> > Can anyone indicate what mapfile properties are or are not going to be
>>> > reinstated in mapserver 5 compared to mapserver 4 for change via URL?
>>> > Obviously this has development ramifications for us when we try to
>>> > upgrade mapserver.
>>>
>>> If I (or anyone) can think of a way to make this tunable at runtime I'd love
>>> to talk more about it. The set of parameters supported in 5.2 has only 
> changed
>>> a little bit since 5.0 based on user input. I can't see exposing more 
> without
>>> good reason. FILTER in particular is worrisome and there is a work around 
> with
>>> the runtime substitution (e.g. FILTER %someval%). In that example you can
>>> pass someval via URL but you have to define a validation pattern (a regex) 
> to
>>> check it against.
>>>
>>> > Alternatively, am I missing something with new mapserver 5 syntax?
>>>
>>> I'm curious if updating maplexer.l to fit your particular needs is workable.
>>>
>>> > Graeme Watmuff
>>>
>>> Steve
>>>
>> --
>> Graeme Watmuff
>> Geographic Web Solutions Pty Ltd
>> 17 Tay Road
>> Woodforde
>> South Australia  5072
>> ph:  + 61 8 83364463
>>
>> _______________________________________________
>> mapserver-users mailing list
>> mapserver-users at lists.osgeo.org 
>> http://lists.osgeo.org/mailman/listinfo/mapserver-users 
>>