[mapserver-users] [mapserver-dev] FW: WMS request fails when spaces are encoded as plus symbol in query part of URL
Rahkonen Jukka (Tike)
jukka.rahkonen at mmmtike.fi
Tue Aug 26 08:08:40 PDT 2014
Hi,
With a help from the local ESRI support this case is now filed into ESRI bug tracker and developers together with something called UAG are now considering what to do to with it.
-Jukka Rahkonen-
________________________________________
Hi,
I made some more testing. It seems that the changed behaviour in ArcGIS Server 10.2.2 affects also other software:
- QGIS 2.0 *can* read layers with spaces in the layer names
- OpenJUMP 1.7 *can't* because it changes spaces to + signs in GetMaps
- Geoserver 2.6 RC1 *can't* cascade layers and the reason is probably the same.
Does anybody know where the ESRI bug tracker is? I would like to file a ticket.
-Jukka-
Rahkonen Jukka wrote:
>
> Hi,
>
> What if I have mapfiles in directories "my mapfiles" and in "my+mapfiles" (made
> a test and they seem to be OK for Windows) and I use WMS urls
> http://host.com/cgi-bin/mapserv.exe?map=my mapfiles/demo.map and urls
> http://host.com/cgi-bin/mapserv.exe?map=my+mapfiles/demo.map?
>
> If user sends "map=my+mapfiles" then the listening party should interpret that +
> means space and search mapfiles from "my mapfiles". Theoretical example but
> somehow I feel that using per cent encoding in every place would be more safe
> though not as compact.
>
> How about the "ä" case and "%C3%A4" vs. %E4? Mapserver generates %E4's
> now and the ArcGIS WMS I should cascade will not accept it even if the problem
> with the space character is solved. Following RFC3986 would solve this problem.
>
> -Jukka Rahkonen-
>
>
> Lime, Steve D wrote:
>
>
> > The bit of reading I've done indicates that +'s for spaces are still
> > allowed by the rfc3986.txt. The + character is still a valid delimiter
> > in the query portion of the string. It's silly for ESRI to not support
> > either - I mean the trend is towards more compact URLs. MapServer CGI
> processing handles %20 or +...
> >
> > Adding a switch to toggle + vs %20 encoding would be painful.
> >
> > Steve
> >
> > -----Original Message-----
> > From: mapserver-users-bounces at lists.osgeo.org [mailto:mapserver-users-
> > bounces at lists.osgeo.org] On Behalf Of Rahkonen Jukka (Tike)
> > Sent: Thursday, August 21, 2014 8:28 AM
> > To: Mapserver-Users (mapserver-users at lists.osgeo.org)
> > Subject: Re: [mapserver-users] [mapserver-dev] FW: WMS request fails
> > when spaces are encoded as plus symbol in query part of URL
> >
> > Hi,
> >
> > A little bit more information. Here is a GetMap request that returns
> > an image from ArcGIS server. Space is changed into "%20" and letter
> > "ä" is changed into "%C3%A4" . This, according to
> > http://www.albionresearch.com/misc/urlencode.php, seems to implement
> > RFC3986.
> >
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/MapServer/WMSServer?REQUE
> > S
> T=GetMap&SERVICE=WMS&VERSION=1.1.1&WIDTH=459&HEIGHT=346&LAYER
> >
> S=Suomen%20maaper%C3%A4kartta%201:1%20000%20000&TRANSPARENT=T
> >
> RUE&FORMAT=image%2Fpng&BBOX=434844.4854575534,7035986.957235612,
> > 452739.30027509434,7144417.839725949&SRS=EPSG:3067&STYLES=
> >
> > The mode where space is encoded as "+" seems to be something that is
> > called
> > "RFC2396 mode". In this mode letter "ä" will become "%E4". The
> > following test where I have only changed "%C3%A4" into "%E4" proves
> > that ArcGIS server does not support RFC2396 mode.
> >
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/MapServer/WMSServer?REQUE
> > S
> T=GetMap&SERVICE=WMS&VERSION=1.1.1&WIDTH=459&HEIGHT=346&LAYER
> >
> S=Suomen%20maaper%E4kartta%201:1%20000%20000&TRANSPARENT=TRUE
> >
> &FORMAT=image%2Fpng&BBOX=434844.4854575534,7035986.957235612,452
> > 739.30027509434,7144417.839725949&SRS=EPSG:3067&STYLES=
> >
> > If http://www.ietf.org/rfc/rfc3986.txt is newer than
> > http://www.ietf.org/rfc/rfc2396.txt I can imagine that there are
> > servers which support either the old, or the new, or both, or neither.
> > Should ther be a
> > RFC2396/RFC3986 switch for selecting the URL-encoding method to use?
> > Or is
> > RFC2396 deprecated and ESRI is doing the right thing by supporting
> > only
> > RFC3986 and Mapserver should do the same?
> >
> >
> > -Jukka Rahkonen-
> >
> >
> > Rahkonen Jukka wrote:
> > >
> > > Hi,
> > >
> > > Here is an ArcGIS WMS to test with.
> > >
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/GTKWMS/MapServer/WMSServe
> > > r?request=GetCapabilities&service=WMS
> > >
> > > Do it quickly because I have asked the service provider to change
> > > the layer names, and guess what - because I could not make our
> > > Mapserver to cascade the service. You will find layer names like
> > > <Name>Maaperä
> > > 1:20 000; 1:50 000</Name>
> > >
> > > I would really much like to know also how the non-ascii characters
> > > should be handled in the mapfile when cascading such layers.
> > >
> > > -Jukka Rahkonen-
> > >
> > >
> > >
> > > Jeff McKenna wrote:
> > >
> > > > spaces are encoded as plus symbol in query part of URL
> > > >
> > > > I agree it's a change that should slip in now for 7.0, but I hope
> > > > those with later ArcServer versions can test it in the beta (or we
> > > > have one volunteer to let us use their Arc WMS instance for testing).
> > > >
> > > > -jeff
> > > >
> > > >
> > > >
> > > > On 2014-08-20 1:52 PM, Lime, Steve D (MNIT) wrote:
> > > > > Devs: Is this a change that we should make for 7.0? Seems
> > > > > reasonable but I'm not sure what the side effects might be.
> > > > >
> > > > > Steve
> > > > >
> > > > > *From:*Moen, Paul T. [mailto:pmoen at nd.gov]
> > > > > *Sent:* Wednesday, August 20, 2014 9:44 AM
> > > > > *To:* Lime, Steve D (MNIT); mapserver-users at lists.osgeo.org
> > > > > *Subject:* Re: [mapserver-users] WMS request fails when spaces
> > > > > are encoded as plus symbol in query part of URL
> > > > >
> > > > > Steve,
> > > > >
> > > > > You are right about the outcome of 1 and 2. 1 encodes the % and
> > > > > 2 throws the following error.
> > > > >
> > > > > msBuildWMSLayerURLBase(): One of wms_onlineresource,
> > > > > wms_server_version, wms_name metadata is missing in layer USGS
> > > > > DRG 250k Topo Maps. Please either provide a valid CONNECTION
> > > > > URL, or provide those values in the layer's metadata.\n\n
> > > > >
> > > > > I found the function as you said and
> > > > >
> > > > > I removed the
> > > > >
> > > > > if (*i == ' ')
> > > > >
> > > > > *j = '+';
> > > > >
> > > > > else
> > > > >
> > > > > from the function then recompiled.
> > > > >
> > > > > char *msEncodeUrlExcept(const char *data, const char except)
> > > > >
> > > > > {
> > > > >
> > > > > char *hex = "0123456789ABCDEF";
> > > > >
> > > > > const char *i;
> > > > >
> > > > > char *j, *code;
> > > > >
> > > > > int inc;
> > > > >
> > > > > unsigned char ch;
> > > > >
> > > > > for (inc=0, i=data; *i!='\0'; i++)
> > > > >
> > > > > if (msEncodeChar(*i))
> > > > >
> > > > > inc += 2;
> > > > >
> > > > > code = (char*)msSmallMalloc(strlen(data)+inc+1);
> > > > >
> > > > > for (j=code, i=data; *i!='\0'; i++, j++) {
> > > > >
> > > > > if ( except != '\0' && *i == except ) {
> > > > >
> > > > > *j = except;
> > > > >
> > > > > } else if (msEncodeChar(*i)) {
> > > > >
> > > > > ch = *i;
> > > > >
> > > > > *j++ = '%';
> > > > >
> > > > > *j++ = hex[ch/16];
> > > > >
> > > > > *j = hex[ch%16];
> > > > >
> > > > > } else
> > > > >
> > > > > *j = *i;
> > > > >
> > > > > }
> > > > >
> > > > > *j = '\0';
> > > > >
> > > > > return code;
> > > > >
> > > > > }
> > > > >
> > > > > Everything works again after a mapserver recompile, install and
> > > > > finally a restart of apache. Thanks so much for the path to the solution.
> > > > >
> > > > > Paul Moen
> > > > > pmoen at nd.gov <mailto:pmoen at nd.gov>
> > > > > 701-328-2434
> > > > >
> > > > > *From: *<Lime>, "Steve D (MNIT)" <Steve.Lime at state.mn.us
> > > > > <mailto:Steve.Lime at state.mn.us>>
> > > > > *Date: *Wednesday, August 20, 2014 at 12:41 AM
> > > > > *To: *Paul Moen <pmoen at nd.gov <mailto:pmoen at nd.gov>>,
> > > > > "mapserver-users at lists.osgeo.org
> > > > > <mailto:mapserver-users at lists.osgeo.org>"
> > > > > <mapserver-users at lists.osgeo.org
> > > > > <mailto:mapserver-users at lists.osgeo.org>>
> > > > > *Subject: *RE: [mapserver-users] WMS request fails when spaces
> > > > > are encoded as plus symbol in query part of URL
> > > > >
> > > > > Hmmm... Nice backwards compatibility ESRI. The +'s seem to still
> > > > > be quite legal (http://tools.ietf.org/html/rfc3986 and other
> > > > > references) in the query string (but not the path). I don't have
> > > > > access to test and I'm guessing these ideas won't work but they
> > > > > might be
> > > worth a quick try:
> > > > >
> > > > > 1) Try encoding the wms_name in the metadata: "wms_name"
> > > > > "Topomap%20DRG%20250k"
> > > > >
> > > > > 2) Don't set the wms_name in the metadata but add it to the
> > > > > connection: CONNECTION
> > > > >
> > > > "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WM
> > > > SS
> > > > er
> > > > ver?L
> > > > AYERS=Topomap%20DRG%20250k&"
> > > > >
> > > > > I'm betting MapServer will encode the %'s in 1 and throw an error in 2.
> > > > > Otherwise you could hack the MapServer source. Function is
> > > > > called
> > > > > msEncodeUrlExcept() in mapstring.c you can see the section to
> > > > > change starting at line 1138 in git master
> > > > > (https://github.com/mapserver/mapserver/blob/master/mapstring.c)
> > > > > <https://github.com/mapserver/mapserver/blob/master/mapstring.c)
> > > > > .>
> > > > >
> > > > > Maybe it's worth making this change as part of 7.0?
> > > > >
> > > > > Steve
> > > > >
> > > > > ----------------------------------------------------------------
> > > > > --
> > > > > --
> > > > > --
> > > > > --
> > > > >
> > > > > *From:*mapserver-users-bounces at lists.osgeo.org
> > > > > <mailto:mapserver-users-bounces at lists.osgeo.org>
> > > > > [mapserver-users-bounces at lists.osgeo.org
> > > > > <mailto:mapserver-users-bounces at lists.osgeo.org>] on behalf of
> > > > > Moen, Paul T. [pmoen at nd.gov <mailto:pmoen at nd.gov>]
> > > > > *Sent:* Tuesday, August 19, 2014 4:56 PM
> > > > > *To:* mapserver-users at lists.osgeo.org
> > > > > <mailto:mapserver-users at lists.osgeo.org>
> > > > > *Subject:* [mapserver-users] WMS request fails when spaces are
> > > > > encoded as plus symbol in query part of URL
> > > > >
> > > > > Hi all,
> > > > >
> > > > > We are consuming WMS services hosted by ESRI ArcGIS Server 10.0.
> > > > > The server was upgraded to ArcGIS Server 10.2.2 and we no longer
> > > > > can access layer names with spaces.
> > > > >
> > > > > This is the server's capabilities.
> > > > >
> > > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/W
> > > > > MS Se rv er?request=GetCapabilities&service=WMS
> > > > >
> > > > > This is the layer I am using.
> > > > >
> > > > > LAYER
> > > > >
> > > > > # DEBUG 5
> > > > >
> > > > > CONNECTIONTYPE WMS
> > > > >
> > > > > CONNECTION
> > > > >
> > > >
> > >
> >
> "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?"
> > > > >
> > > > > METADATA
> > > > >
> > > > > "wms_name" "Topomap DRG 250k"
> > > > >
> > > > > "wms_format" "image/png"
> > > > >
> > > > > "wms_server_version" "1.1.1"
> > > > >
> > > > > "wms_srs" "EPSG:2266"
> > > > >
> > > > > END
> > > > >
> > > > > TYPE RASTER
> > > > >
> > > > > STATUS OFF
> > > > >
> > > > > NAME "USGS DRG 250k Topo Maps"
> > > > >
> > > > > END
> > > > >
> > > > > Mapserver encodes this request as follows. Notice that the
> > > > > layer name, which is 'Topomap DRG 250k', is encoded with the
> > > > > spaces becoming the
> > > > '+'
> > > > > character.
> > > > >
> > > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/W
> > > > > MS
> > > > > Se
> > > > > rv
> > > > >
> > > >
> > >
> >
> er?LAYERS=Topomap+DRG+250k&REQUEST=map&WMTVER=1.0.0&SERVICE=W
> > > > MS&FORMAT
> > > > >
> > > >
> > >
> >
> =image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BBOX=19
> > > > 06240.1
> > > > >
> > > >
> > >
> >
> 5992838,596656.333359479,2119730.85012732,789150.692893686&TRANSPAR
> > > > ENT
> > > > > =TRUE&EXCEPTIONS=INIMAGE
> > > > >
> > > > > This returns the error in an image that says "Parameter 'layers'
> > > > > contains unacceptable layer names."
> > > > >
> > > > > When I replace the + in the layer name with %20,
> > > > > LAYERS=Topomap+DRG+250k become
> LAYERS=Topomap%20DRG%20250k
> > > > and the request becomes the following.
> > > > >
> > > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/W
> > > > > MS
> > > > > Se
> > > > > rv
> > > > >
> > > >
> > >
> >
> er?LAYERS=Topomap%20DRG%20250k&REQUEST=map&WMTVER=1.0.0&SERVI
> > > > CE=WMS&FO
> > > > >
> > > >
> > >
> >
> RMAT=image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BB
> > > > OX=19062
> > > > >
> > > >
> > >
> >
> 40.15992838,596656.333359479,2119730.85012732,789150.692893686&TRAN
> > > > SPA
> > > > > RENT=TRUE&EXCEPTIONS=INIMAGE
> > > > >
> > > > > That request works. Obviously, ESRI is no longer decoding + as
> > > > > a space and therefore does not recognize the layer name with spaces.
> > > > >
> > > > > Is there any way I can have mapserver encode all characters
> > > > > using percent-encoding, or at least encode spaces as %20 instead
> > > > > of a +
> > character.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Paul Moen
> > > > > pmoen at nd.gov <mailto:pmoen at nd.gov>
> > > > > 701-328-2434
> > > > >
> > > > >
> > > > _______________________________________________
> > > > mapserver-dev mailing list
> > > > mapserver-dev at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
More information about the mapserver-users
mailing list