[mapserver-users] [mapserver-dev] FW: WMS request fails when spaces are encoded as plus symbol in query part of URL
Rahkonen Jukka (Tike)
jukka.rahkonen at mmmtike.fi
Thu Aug 21 08:04:21 PDT 2014
Hi,
What if I have mapfiles in directories "my mapfiles" and in "my+mapfiles" (made a test and they seem to be OK for Windows) and I use WMS urls http://host.com/cgi-bin/mapserv.exe?map=my mapfiles/demo.map and urls http://host.com/cgi-bin/mapserv.exe?map=my+mapfiles/demo.map?
If user sends "map=my+mapfiles" then the listening party should interpret that + means space and search mapfiles from "my mapfiles". Theoretical example but somehow I feel that using per cent encoding in every place would be more safe though not as compact.
How about the "ä" case and "%C3%A4" vs. %E4? Mapserver generates %E4's now and the ArcGIS WMS I should cascade will not accept it even if the problem with the space character is solved. Following RFC3986 would solve this problem.
-Jukka Rahkonen-
Lime, Steve D wrote:
> The bit of reading I've done indicates that +'s for spaces are still allowed by the
> rfc3986.txt. The + character is still a valid delimiter in the query portion of the
> string. It's silly for ESRI to not support either - I mean the trend is towards more
> compact URLs. MapServer CGI processing handles %20 or +...
>
> Adding a switch to toggle + vs %20 encoding would be painful.
>
> Steve
>
> -----Original Message-----
> From: mapserver-users-bounces at lists.osgeo.org [mailto:mapserver-users-
> bounces at lists.osgeo.org] On Behalf Of Rahkonen Jukka (Tike)
> Sent: Thursday, August 21, 2014 8:28 AM
> To: Mapserver-Users (mapserver-users at lists.osgeo.org)
> Subject: Re: [mapserver-users] [mapserver-dev] FW: WMS request fails when
> spaces are encoded as plus symbol in query part of URL
>
> Hi,
>
> A little bit more information. Here is a GetMap request that returns an image
> from ArcGIS server. Space is changed into "%20" and letter "ä" is changed into
> "%C3%A4" . This, according to
> http://www.albionresearch.com/misc/urlencode.php, seems to implement
> RFC3986.
>
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/MapServer/WMSServer?REQUES
> T=GetMap&SERVICE=WMS&VERSION=1.1.1&WIDTH=459&HEIGHT=346&LAYER
> S=Suomen%20maaper%C3%A4kartta%201:1%20000%20000&TRANSPARENT=T
> RUE&FORMAT=image%2Fpng&BBOX=434844.4854575534,7035986.957235612,
> 452739.30027509434,7144417.839725949&SRS=EPSG:3067&STYLES=
>
> The mode where space is encoded as "+" seems to be something that is called
> "RFC2396 mode". In this mode letter "ä" will become "%E4". The following test
> where I have only changed "%C3%A4" into "%E4" proves that ArcGIS server
> does not support RFC2396 mode.
>
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/MapServer/WMSServer?REQUES
> T=GetMap&SERVICE=WMS&VERSION=1.1.1&WIDTH=459&HEIGHT=346&LAYER
> S=Suomen%20maaper%E4kartta%201:1%20000%20000&TRANSPARENT=TRUE
> &FORMAT=image%2Fpng&BBOX=434844.4854575534,7035986.957235612,452
> 739.30027509434,7144417.839725949&SRS=EPSG:3067&STYLES=
>
> If http://www.ietf.org/rfc/rfc3986.txt is newer than
> http://www.ietf.org/rfc/rfc2396.txt I can imagine that there are servers which
> support either the old, or the new, or both, or neither. Should ther be a
> RFC2396/RFC3986 switch for selecting the URL-encoding method to use? Or is
> RFC2396 deprecated and ESRI is doing the right thing by supporting only
> RFC3986 and Mapserver should do the same?
>
>
> -Jukka Rahkonen-
>
>
> Rahkonen Jukka wrote:
> >
> > Hi,
> >
> > Here is an ArcGIS WMS to test with.
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/GTKWMS/MapServer/WMSServe
> > r?request=GetCapabilities&service=WMS
> >
> > Do it quickly because I have asked the service provider to change the
> > layer names, and guess what - because I could not make our Mapserver
> > to cascade the service. You will find layer names like <Name>Maaperä
> > 1:20 000; 1:50 000</Name>
> >
> > I would really much like to know also how the non-ascii characters
> > should be handled in the mapfile when cascading such layers.
> >
> > -Jukka Rahkonen-
> >
> >
> >
> > Jeff McKenna wrote:
> >
> > > spaces are encoded as plus symbol in query part of URL
> > >
> > > I agree it's a change that should slip in now for 7.0, but I hope
> > > those with later ArcServer versions can test it in the beta (or we
> > > have one volunteer to let us use their Arc WMS instance for testing).
> > >
> > > -jeff
> > >
> > >
> > >
> > > On 2014-08-20 1:52 PM, Lime, Steve D (MNIT) wrote:
> > > > Devs: Is this a change that we should make for 7.0? Seems
> > > > reasonable but I'm not sure what the side effects might be.
> > > >
> > > > Steve
> > > >
> > > > *From:*Moen, Paul T. [mailto:pmoen at nd.gov]
> > > > *Sent:* Wednesday, August 20, 2014 9:44 AM
> > > > *To:* Lime, Steve D (MNIT); mapserver-users at lists.osgeo.org
> > > > *Subject:* Re: [mapserver-users] WMS request fails when spaces are
> > > > encoded as plus symbol in query part of URL
> > > >
> > > > Steve,
> > > >
> > > > You are right about the outcome of 1 and 2. 1 encodes the % and 2
> > > > throws the following error.
> > > >
> > > > msBuildWMSLayerURLBase(): One of wms_onlineresource,
> > > > wms_server_version, wms_name metadata is missing in layer USGS DRG
> > > > 250k Topo Maps. Please either provide a valid CONNECTION URL, or
> > > > provide those values in the layer's metadata.\n\n
> > > >
> > > > I found the function as you said and
> > > >
> > > > I removed the
> > > >
> > > > if (*i == ' ')
> > > >
> > > > *j = '+';
> > > >
> > > > else
> > > >
> > > > from the function then recompiled.
> > > >
> > > > char *msEncodeUrlExcept(const char *data, const char except)
> > > >
> > > > {
> > > >
> > > > char *hex = "0123456789ABCDEF";
> > > >
> > > > const char *i;
> > > >
> > > > char *j, *code;
> > > >
> > > > int inc;
> > > >
> > > > unsigned char ch;
> > > >
> > > > for (inc=0, i=data; *i!='\0'; i++)
> > > >
> > > > if (msEncodeChar(*i))
> > > >
> > > > inc += 2;
> > > >
> > > > code = (char*)msSmallMalloc(strlen(data)+inc+1);
> > > >
> > > > for (j=code, i=data; *i!='\0'; i++, j++) {
> > > >
> > > > if ( except != '\0' && *i == except ) {
> > > >
> > > > *j = except;
> > > >
> > > > } else if (msEncodeChar(*i)) {
> > > >
> > > > ch = *i;
> > > >
> > > > *j++ = '%';
> > > >
> > > > *j++ = hex[ch/16];
> > > >
> > > > *j = hex[ch%16];
> > > >
> > > > } else
> > > >
> > > > *j = *i;
> > > >
> > > > }
> > > >
> > > > *j = '\0';
> > > >
> > > > return code;
> > > >
> > > > }
> > > >
> > > > Everything works again after a mapserver recompile, install and
> > > > finally a restart of apache. Thanks so much for the path to the solution.
> > > >
> > > > Paul Moen
> > > > pmoen at nd.gov <mailto:pmoen at nd.gov>
> > > > 701-328-2434
> > > >
> > > > *From: *<Lime>, "Steve D (MNIT)" <Steve.Lime at state.mn.us
> > > > <mailto:Steve.Lime at state.mn.us>>
> > > > *Date: *Wednesday, August 20, 2014 at 12:41 AM
> > > > *To: *Paul Moen <pmoen at nd.gov <mailto:pmoen at nd.gov>>,
> > > > "mapserver-users at lists.osgeo.org
> > > > <mailto:mapserver-users at lists.osgeo.org>"
> > > > <mapserver-users at lists.osgeo.org
> > > > <mailto:mapserver-users at lists.osgeo.org>>
> > > > *Subject: *RE: [mapserver-users] WMS request fails when spaces are
> > > > encoded as plus symbol in query part of URL
> > > >
> > > > Hmmm... Nice backwards compatibility ESRI. The +'s seem to still
> > > > be quite legal (http://tools.ietf.org/html/rfc3986 and other
> > > > references) in the query string (but not the path). I don't have
> > > > access to test and I'm guessing these ideas won't work but they
> > > > might be
> > worth a quick try:
> > > >
> > > > 1) Try encoding the wms_name in the metadata: "wms_name"
> > > > "Topomap%20DRG%20250k"
> > > >
> > > > 2) Don't set the wms_name in the metadata but add it to the
> > > > connection: CONNECTION
> > > >
> > > "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSS
> > > er
> > > ver?L
> > > AYERS=Topomap%20DRG%20250k&"
> > > >
> > > > I'm betting MapServer will encode the %'s in 1 and throw an error in 2.
> > > > Otherwise you could hack the MapServer source. Function is called
> > > > msEncodeUrlExcept() in mapstring.c you can see the section to
> > > > change starting at line 1138 in git master
> > > > (https://github.com/mapserver/mapserver/blob/master/mapstring.c)
> > > > <https://github.com/mapserver/mapserver/blob/master/mapstring.c).>
> > > >
> > > > Maybe it's worth making this change as part of 7.0?
> > > >
> > > > Steve
> > > >
> > > > ------------------------------------------------------------------
> > > > --
> > > > --
> > > > --
> > > >
> > > > *From:*mapserver-users-bounces at lists.osgeo.org
> > > > <mailto:mapserver-users-bounces at lists.osgeo.org>
> > > > [mapserver-users-bounces at lists.osgeo.org
> > > > <mailto:mapserver-users-bounces at lists.osgeo.org>] on behalf of
> > > > Moen, Paul T. [pmoen at nd.gov <mailto:pmoen at nd.gov>]
> > > > *Sent:* Tuesday, August 19, 2014 4:56 PM
> > > > *To:* mapserver-users at lists.osgeo.org
> > > > <mailto:mapserver-users at lists.osgeo.org>
> > > > *Subject:* [mapserver-users] WMS request fails when spaces are
> > > > encoded as plus symbol in query part of URL
> > > >
> > > > Hi all,
> > > >
> > > > We are consuming WMS services hosted by ESRI ArcGIS Server 10.0.
> > > > The server was upgraded to ArcGIS Server 10.2.2 and we no longer
> > > > can access layer names with spaces.
> > > >
> > > > This is the server's capabilities.
> > > >
> > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMS
> > > > Se rv er?request=GetCapabilities&service=WMS
> > > >
> > > > This is the layer I am using.
> > > >
> > > > LAYER
> > > >
> > > > # DEBUG 5
> > > >
> > > > CONNECTIONTYPE WMS
> > > >
> > > > CONNECTION
> > > >
> > >
> >
> "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?"
> > > >
> > > > METADATA
> > > >
> > > > "wms_name" "Topomap DRG 250k"
> > > >
> > > > "wms_format" "image/png"
> > > >
> > > > "wms_server_version" "1.1.1"
> > > >
> > > > "wms_srs" "EPSG:2266"
> > > >
> > > > END
> > > >
> > > > TYPE RASTER
> > > >
> > > > STATUS OFF
> > > >
> > > > NAME "USGS DRG 250k Topo Maps"
> > > >
> > > > END
> > > >
> > > > Mapserver encodes this request as follows. Notice that the layer
> > > > name, which is 'Topomap DRG 250k', is encoded with the spaces
> > > > becoming the
> > > '+'
> > > > character.
> > > >
> > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMS
> > > > Se
> > > > rv
> > > >
> > >
> >
> er?LAYERS=Topomap+DRG+250k&REQUEST=map&WMTVER=1.0.0&SERVICE=W
> > > MS&FORMAT
> > > >
> > >
> >
> =image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BBOX=19
> > > 06240.1
> > > >
> > >
> >
> 5992838,596656.333359479,2119730.85012732,789150.692893686&TRANSPAR
> > > ENT
> > > > =TRUE&EXCEPTIONS=INIMAGE
> > > >
> > > > This returns the error in an image that says "Parameter 'layers'
> > > > contains unacceptable layer names."
> > > >
> > > > When I replace the + in the layer name with %20,
> > > > LAYERS=Topomap+DRG+250k become LAYERS=Topomap%20DRG%20250k
> > > and the request becomes the following.
> > > >
> > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMS
> > > > Se
> > > > rv
> > > >
> > >
> >
> er?LAYERS=Topomap%20DRG%20250k&REQUEST=map&WMTVER=1.0.0&SERVI
> > > CE=WMS&FO
> > > >
> > >
> >
> RMAT=image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BB
> > > OX=19062
> > > >
> > >
> >
> 40.15992838,596656.333359479,2119730.85012732,789150.692893686&TRAN
> > > SPA
> > > > RENT=TRUE&EXCEPTIONS=INIMAGE
> > > >
> > > > That request works. Obviously, ESRI is no longer decoding + as a
> > > > space and therefore does not recognize the layer name with spaces.
> > > >
> > > > Is there any way I can have mapserver encode all characters using
> > > > percent-encoding, or at least encode spaces as %20 instead of a +
> character.
> > > >
> > > > Thanks,
> > > >
> > > > Paul Moen
> > > > pmoen at nd.gov <mailto:pmoen at nd.gov>
> > > > 701-328-2434
> > > >
> > > >
> > > _______________________________________________
> > > mapserver-dev mailing list
> > > mapserver-dev at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
More information about the mapserver-users
mailing list