[mapserver-users] New limitations with mapserver 5 CGI

[Steve Lime]

FYI I've created ticket http://trac.osgeo.org/mapserver/ticket/2699 to track this...

Steve

>>> On 7/11/2008 at 11:41 AM, in message <487746E3.5157.008F.0 at dnr.state.mn.us>,
"Steve Lime" <Steve.Lime at dnr.state.mn.us> wrote:
> We aren't limited solely to the lexer. DATA and TEMPLATE, the two most widely 
> used CGI
> overrides have a second control: DATAPATTERN and TEMPLATEPATTERN. When in a 
> particular
> parsing state we check the appropriate pattern (which NULL, or unmatchable, 
> by default) to
> decide if we should accept the override or not. The parameters at issue are 
> those that aren't
> held to any validation by the lexer. Colors, sizes and such all have to be 
> of a certain format
> or data type. Others like filters, data paths, connection strings etc... 
> aren't subject to 
> validation. We could impose validation like DATAPATTERN or TEMPLATEPATTERN 
> on those
> parameters specifically. I wouldn't want to add a boat load of new 
> parameters but we could
> do something like this in layer metadata:
> 
> METADATA
>   data_validation_pattern 'my pattern'
>   filter_validation_pattern 'another pattern'
>   ...
> END
> 
> We already use this scheme for validation of runtime substitutions and query 
> strings for attribute
> tables. This way the user could control which of these parameters could be 
> overridden at runtime.
> All parameters that are subject to validation as part of mapfile parsing 
> would be web accessible
> by default. Other parameters that we never want to allow override of can 
> still be controlled within
> the lexer.
> 
> The METADATA code would need special provisions to never allow setting keys 
> ending in 
> "_validation_pattern" from a URL source.
> 
> DATAPATTERN and TEMPLATEPATTERN would become deprecated.
> 
> Steve
> 
>>>> On 7/10/2008 at 10:19 PM, in message
> <30fe546d0807102019y52beb686oe3ba5ba311cbee49 at mail.gmail.com>, "Paul Ramsey"
> <pramsey at cleverelephant.ca> wrote:
>> I'd like to see backwards feature compatibility available, and I think
>> the CGI override controller is one of the most powerful
>> (under-appreciated) features of Mapserver.
>> 
>> Since the control of what is over-rideable is in the lexer though,
>> we're sort of in a narrow box. We could do a compile-time option,
>> which would allow us to default to strict but allow (relatively) easy
>> conversion to flexible. That wouldn't really help our Windows brethren
>> though, since they mostly rely on pre-compiled versions, and binary
>> maintainers don't maintain lots of different builds.
>> 
>> Having a map-level run-time directive to control the availability of
>> URL over-rides would be the ideal, IMO.  Default, NONE.  Other
>> options, STRICT and FULL.  However, I don't see how to get there from
>> here with the lexer requirement.
>> 
>> P.
>> 
>> On Thu, Jul 10, 2008 at 6:05 PM, Graeme Watmuff
>> <gwatmuff at geographicweb.com.au> wrote:
>>> Hi Steve,
>>>
>>> Thanks for replying and thanks for the coding pointers. You are right
>>> about the mapserv 4.10 tileindex syntax. Like filter, it is simply
>>> ignored by v5 rather than throwing an error. My apologies.
>>>
>>> Would it make sense to create a 'secure' version for the paranoid and a
>>> another version with full functionality? I realize this would create
>>> more maintenance work for you, but not all mapserver users are going to
>>> be able to climb inside the code themselves to modify functionality
>>> (fortunately we can).
>>>
>>> Regarding security, I presume you are concerned about mapserver exposing
>>> sensitive data to unintended users. If so, should you be taking that
>>> responsibility on yourself by limiting mapserver or should the onus be
>>> upon mapserver users to manage their own data more responsibly? I am a
>>> great believer in personal responsibility. If the security risk is made
>>> clear to users, they can make their own decisions as to the data they
>>> potentially expose to the world.
>>>
>>> We have greatly appreciated and benefited from the tremendous work you
>>> and your associates have done in bringing mapserver to the world and we
>>> have always looked forward to the added features of each new version.
>>> But upgrading now comes at some degree of inconvenience for us and maybe
>>> others unless someone wants to publish the workaround to resurrect lost
>>> functionality for all to simply acquire.
>>>
>>> I am not convinced that limiting the previous functionality is the right
>>> direction for mapserver unless you fear say a litigious threat from
>>> security breaches that people may bring upon themselves (hardly your
>>> responsibility). Those with seriously sensitive data are not going to
>>> want expose the CGI to all the world via a browser-sent URL anyway. I
>>> would expect them to send URLs from say, a java web app embedded in a
>>> secured tomcat container.
>>>
>>> Just my thoughts.
>>>
>>> Regards and best wishes
>>>
>>> Graeme
>>>
>>> On Thu, 2008-07-10 at 17:13 -0500, Steve Lime wrote:
>>>> Hi Graeme: Comments inline... I definitely underestimated the impacts of 
>> those
>>>> changes. Anyway...
>>>>
>>>> >>> On 7/9/2008 at 11:23 PM, in message
>>>> <1215663787.2513.112.camel at bongolx.ringo.net>, Graeme Watmuff
>>>> <gwatmuff at geographicweb.com.au> wrote:
>>>> > It has become apparent through some frustrating moments of trial and
>>>> > error that many of the mapfile variables that were changeable via URL
>>>> > with mapserver 4.10.x CGI seem no longer changeable with v5.x or 5.2.x.
>>>> > There is a hint from the docs that this is because of security concerns?
>>>> > Our java apps don't allow web users to directly interact with the CGI,
>>>> > so security doesn't seem so important in our case.
>>>>
>>>> Allowing unaltered, direct modification of things like FILTERs or 
>> CONNECTIONS
>>>> is a potential security risk although I've never run across a documented 
>> case. Filters
>>>> are risky because they are passed to the underlying data driver which 
>> generally
>>>> trust the contents.
>>>>
>>>> > Some examples:
>>>> >
>>>> > map_web_metadata='text1' 'text2'
>>>> > map_layerName_filter=<expression>
>>>> > map_layerName_tileindex=<tileindex.shp>
>>>> > map_layerName_data=<datasource>
>>>> >
>>>> > allow changes to the mapfile via URL with mapserver 4.10.x
>>>> >
>>>> > Following the new syntax specified for v5 + I had expected
>>>> >
>>>> > map_web=metadata+"'text1' 'text2'" or maybe
>>>> > map_web_metadata[0]='text1'+'text2',
>>>> > map_layer[layerName]=filter+<expression>,
>>>> > map_layer[layerName]=tileindex+<tileindex.shp>,
>>>> > map_layer[layerName]=data+<datasource>
>>>> >
>>>> > to all perform in the time-honoured way.
>>>>
>>>> And they don't... If they did you could combine modifications to a layer in 
>> a single
>>>> variable:
>>>>
>>>>   
>> map_layer[layerName]=filter+<expression>+tileindex+<tileindex.shp>+data+<datasou
>> rce>
>>>>
>>>> > Only 'data' could be changed with this new mapserver 5 syntax. I
>>>> > happened to discover through the change logs that changing tileindex was
>>>> > reinstated for mapserver 5 CGI, but using the old mapserver 4.10 syntax
>>>> > (map_layerName_tileindex)=<tileindex.shp> - not the new mapserver 5
>>>> > syntax style (map_layer[layerName]=tileindex+<tileindex.shp>).
>>>>
>>>> Looking at the code, TILEINDEX is not changable via a URL with any syntax. 
>> The
>>>> change log references a bug fix to 4.10. The old syntax doesn't work at all.
>>>>
>>>> > mapserv 5 CGI throws an error (Parsing error near (filter)) when the new
>>>> > syntax is used for filter and ignores the old 4.10 filter syntax.
>>>>
>>>> This is expected. It will fail with an error as opposed to the silent 
>> failure of versions
>>>> past.
>>>>
>>>> > No URL syntax for changing the web object's metadata content appears
>>>> > acceptable to the mapserver 5.
>>>>
>>>> Correct.
>>>>
>>>> > I would dearly like to see changing the layer filter via URL restored
>>>> > also to mapserver 5. And the web object's metadata setting functionality
>>>> > too.
>>>>
>>>> It's possible to customize this behavior by editing maplexer.l and 
>> recompiling
>>>> the code. Simply find the keyword you're interested in supporting and change
>>>> the states at which it is valid. So, for TILEINDEX:
>>>>
>>>> <INITIAL>tileindex
>>>>
>>>> becomes:
>>>>
>>>> <INITIAL,URL_STRING>tileindex
>>>>
>>>> Same goes for the other parameters you'd want to activate.
>>>>
>>>> > Can anyone indicate what mapfile properties are or are not going to be
>>>> > reinstated in mapserver 5 compared to mapserver 4 for change via URL?
>>>> > Obviously this has development ramifications for us when we try to
>>>> > upgrade mapserver.
>>>>
>>>> If I (or anyone) can think of a way to make this tunable at runtime I'd love
>>>> to talk more about it. The set of parameters supported in 5.2 has only 
>> changed
>>>> a little bit since 5.0 based on user input. I can't see exposing more 
>> without
>>>> good reason. FILTER in particular is worrisome and there is a work around 
>> with
>>>> the runtime substitution (e.g. FILTER %someval%). In that example you can
>>>> pass someval via URL but you have to define a validation pattern (a regex) 
>> to
>>>> check it against.
>>>>
>>>> > Alternatively, am I missing something with new mapserver 5 syntax?
>>>>
>>>> I'm curious if updating maplexer.l to fit your particular needs is workable.
>>>>
>>>> > Graeme Watmuff
>>>>
>>>> Steve
>>>>
>>> --
>>> Graeme Watmuff
>>> Geographic Web Solutions Pty Ltd
>>> 17 Tay Road
>>> Woodforde
>>> South Australia  5072
>>> ph:  + 61 8 83364463
>>>
>>> _______________________________________________
>>> mapserver-users mailing list
>>> mapserver-users at lists.osgeo.org 
>>> http://lists.osgeo.org/mailman/listinfo/mapserver-users 
>>>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org 
> http://lists.osgeo.org/mailman/listinfo/mapserver-users