[mapserver-users] [mapserver-dev] FW: WMS request fails when spaces are encoded as plus symbol in query part of URL
Rahkonen Jukka (Tike)
jukka.rahkonen at mmmtike.fi
Thu Aug 21 13:33:51 PDT 2014
Hi,
I made some more testing. It seems that the changed behaviour in ArcGIS Server 10.2.2 affects also other software:
- QGIS 2.0 *can* read layers with spaces in the layer names
- OpenJUMP 1.7 *can't* because it changes spaces to + signs in GetMaps
- Geoserver 2.6 RC1 *can't* cascade layers and the reason is probably the same.
Does anybody know where the ESRI bug tracker is? I would like to file a ticket.
-Jukka-
Rahkonen Jukka wrote:
>
> Hi,
>
> What if I have mapfiles in directories "my mapfiles" and in "my+mapfiles" (made
> a test and they seem to be OK for Windows) and I use WMS urls
> http://host.com/cgi-bin/mapserv.exe?map=my mapfiles/demo.map and urls
> http://host.com/cgi-bin/mapserv.exe?map=my+mapfiles/demo.map?
>
> If user sends "map=my+mapfiles" then the listening party should interpret that +
> means space and search mapfiles from "my mapfiles". Theoretical example but
> somehow I feel that using per cent encoding in every place would be more safe
> though not as compact.
>
> How about the "ä" case and "%C3%A4" vs. %E4? Mapserver generates %E4's
> now and the ArcGIS WMS I should cascade will not accept it even if the problem
> with the space character is solved. Following RFC3986 would solve this problem.
>
> -Jukka Rahkonen-
>
>
> Lime, Steve D wrote:
>
>
> > The bit of reading I've done indicates that +'s for spaces are still
> > allowed by the rfc3986.txt. The + character is still a valid delimiter
> > in the query portion of the string. It's silly for ESRI to not support
> > either - I mean the trend is towards more compact URLs. MapServer CGI
> processing handles %20 or +...
> >
> > Adding a switch to toggle + vs %20 encoding would be painful.
> >
> > Steve
> >
> > -----Original Message-----
> > From: mapserver-users-bounces at lists.osgeo.org [mailto:mapserver-users-
> > bounces at lists.osgeo.org] On Behalf Of Rahkonen Jukka (Tike)
> > Sent: Thursday, August 21, 2014 8:28 AM
> > To: Mapserver-Users (mapserver-users at lists.osgeo.org)
> > Subject: Re: [mapserver-users] [mapserver-dev] FW: WMS request fails
> > when spaces are encoded as plus symbol in query part of URL
> >
> > Hi,
> >
> > A little bit more information. Here is a GetMap request that returns
> > an image from ArcGIS server. Space is changed into "%20" and letter
> > "ä" is changed into "%C3%A4" . This, according to
> > http://www.albionresearch.com/misc/urlencode.php, seems to implement
> > RFC3986.
> >
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/MapServer/WMSServer?REQUE
> > S
> T=GetMap&SERVICE=WMS&VERSION=1.1.1&WIDTH=459&HEIGHT=346&LAYER
> >
> S=Suomen%20maaper%C3%A4kartta%201:1%20000%20000&TRANSPARENT=T
> >
> RUE&FORMAT=image%2Fpng&BBOX=434844.4854575534,7035986.957235612,
> > 452739.30027509434,7144417.839725949&SRS=EPSG:3067&STYLES=
> >
> > The mode where space is encoded as "+" seems to be something that is
> > called
> > "RFC2396 mode". In this mode letter "ä" will become "%E4". The
> > following test where I have only changed "%C3%A4" into "%E4" proves
> > that ArcGIS server does not support RFC2396 mode.
> >
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/MapServer/WMSServer?REQUE
> > S
> T=GetMap&SERVICE=WMS&VERSION=1.1.1&WIDTH=459&HEIGHT=346&LAYER
> >
> S=Suomen%20maaper%E4kartta%201:1%20000%20000&TRANSPARENT=TRUE
> >
> &FORMAT=image%2Fpng&BBOX=434844.4854575534,7035986.957235612,452
> > 739.30027509434,7144417.839725949&SRS=EPSG:3067&STYLES=
> >
> > If http://www.ietf.org/rfc/rfc3986.txt is newer than
> > http://www.ietf.org/rfc/rfc2396.txt I can imagine that there are
> > servers which support either the old, or the new, or both, or neither.
> > Should ther be a
> > RFC2396/RFC3986 switch for selecting the URL-encoding method to use?
> > Or is
> > RFC2396 deprecated and ESRI is doing the right thing by supporting
> > only
> > RFC3986 and Mapserver should do the same?
> >
> >
> > -Jukka Rahkonen-
> >
> >
> > Rahkonen Jukka wrote:
> > >
> > > Hi,
> > >
> > > Here is an ArcGIS WMS to test with.
> > >
> >
> http://gtkdata.gtk.fi/arcgis/services/GTKWMS/GTKWMS/MapServer/WMSServe
> > > r?request=GetCapabilities&service=WMS
> > >
> > > Do it quickly because I have asked the service provider to change
> > > the layer names, and guess what - because I could not make our
> > > Mapserver to cascade the service. You will find layer names like
> > > <Name>Maaperä
> > > 1:20 000; 1:50 000</Name>
> > >
> > > I would really much like to know also how the non-ascii characters
> > > should be handled in the mapfile when cascading such layers.
> > >
> > > -Jukka Rahkonen-
> > >
> > >
> > >
> > > Jeff McKenna wrote:
> > >
> > > > spaces are encoded as plus symbol in query part of URL
> > > >
> > > > I agree it's a change that should slip in now for 7.0, but I hope
> > > > those with later ArcServer versions can test it in the beta (or we
> > > > have one volunteer to let us use their Arc WMS instance for testing).
> > > >
> > > > -jeff
> > > >
> > > >
> > > >
> > > > On 2014-08-20 1:52 PM, Lime, Steve D (MNIT) wrote:
> > > > > Devs: Is this a change that we should make for 7.0? Seems
> > > > > reasonable but I'm not sure what the side effects might be.
> > > > >
> > > > > Steve
> > > > >
> > > > > *From:*Moen, Paul T. [mailto:pmoen at nd.gov]
> > > > > *Sent:* Wednesday, August 20, 2014 9:44 AM
> > > > > *To:* Lime, Steve D (MNIT); mapserver-users at lists.osgeo.org
> > > > > *Subject:* Re: [mapserver-users] WMS request fails when spaces
> > > > > are encoded as plus symbol in query part of URL
> > > > >
> > > > > Steve,
> > > > >
> > > > > You are right about the outcome of 1 and 2. 1 encodes the % and
> > > > > 2 throws the following error.
> > > > >
> > > > > msBuildWMSLayerURLBase(): One of wms_onlineresource,
> > > > > wms_server_version, wms_name metadata is missing in layer USGS
> > > > > DRG 250k Topo Maps. Please either provide a valid CONNECTION
> > > > > URL, or provide those values in the layer's metadata.\n\n
> > > > >
> > > > > I found the function as you said and
> > > > >
> > > > > I removed the
> > > > >
> > > > > if (*i == ' ')
> > > > >
> > > > > *j = '+';
> > > > >
> > > > > else
> > > > >
> > > > > from the function then recompiled.
> > > > >
> > > > > char *msEncodeUrlExcept(const char *data, const char except)
> > > > >
> > > > > {
> > > > >
> > > > > char *hex = "0123456789ABCDEF";
> > > > >
> > > > > const char *i;
> > > > >
> > > > > char *j, *code;
> > > > >
> > > > > int inc;
> > > > >
> > > > > unsigned char ch;
> > > > >
> > > > > for (inc=0, i=data; *i!='\0'; i++)
> > > > >
> > > > > if (msEncodeChar(*i))
> > > > >
> > > > > inc += 2;
> > > > >
> > > > > code = (char*)msSmallMalloc(strlen(data)+inc+1);
> > > > >
> > > > > for (j=code, i=data; *i!='\0'; i++, j++) {
> > > > >
> > > > > if ( except != '\0' && *i == except ) {
> > > > >
> > > > > *j = except;
> > > > >
> > > > > } else if (msEncodeChar(*i)) {
> > > > >
> > > > > ch = *i;
> > > > >
> > > > > *j++ = '%';
> > > > >
> > > > > *j++ = hex[ch/16];
> > > > >
> > > > > *j = hex[ch%16];
> > > > >
> > > > > } else
> > > > >
> > > > > *j = *i;
> > > > >
> > > > > }
> > > > >
> > > > > *j = '\0';
> > > > >
> > > > > return code;
> > > > >
> > > > > }
> > > > >
> > > > > Everything works again after a mapserver recompile, install and
> > > > > finally a restart of apache. Thanks so much for the path to the solution.
> > > > >
> > > > > Paul Moen
> > > > > pmoen at nd.gov <mailto:pmoen at nd.gov>
> > > > > 701-328-2434
> > > > >
> > > > > *From: *<Lime>, "Steve D (MNIT)" <Steve.Lime at state.mn.us
> > > > > <mailto:Steve.Lime at state.mn.us>>
> > > > > *Date: *Wednesday, August 20, 2014 at 12:41 AM
> > > > > *To: *Paul Moen <pmoen at nd.gov <mailto:pmoen at nd.gov>>,
> > > > > "mapserver-users at lists.osgeo.org
> > > > > <mailto:mapserver-users at lists.osgeo.org>"
> > > > > <mapserver-users at lists.osgeo.org
> > > > > <mailto:mapserver-users at lists.osgeo.org>>
> > > > > *Subject: *RE: [mapserver-users] WMS request fails when spaces
> > > > > are encoded as plus symbol in query part of URL
> > > > >
> > > > > Hmmm... Nice backwards compatibility ESRI. The +'s seem to still
> > > > > be quite legal (http://tools.ietf.org/html/rfc3986 and other
> > > > > references) in the query string (but not the path). I don't have
> > > > > access to test and I'm guessing these ideas won't work but they
> > > > > might be
> > > worth a quick try:
> > > > >
> > > > > 1) Try encoding the wms_name in the metadata: "wms_name"
> > > > > "Topomap%20DRG%20250k"
> > > > >
> > > > > 2) Don't set the wms_name in the metadata but add it to the
> > > > > connection: CONNECTION
> > > > >
> > > > "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WM
> > > > SS
> > > > er
> > > > ver?L
> > > > AYERS=Topomap%20DRG%20250k&"
> > > > >
> > > > > I'm betting MapServer will encode the %'s in 1 and throw an error in 2.
> > > > > Otherwise you could hack the MapServer source. Function is
> > > > > called
> > > > > msEncodeUrlExcept() in mapstring.c you can see the section to
> > > > > change starting at line 1138 in git master
> > > > > (https://github.com/mapserver/mapserver/blob/master/mapstring.c)
> > > > > <https://github.com/mapserver/mapserver/blob/master/mapstring.c)
> > > > > .>
> > > > >
> > > > > Maybe it's worth making this change as part of 7.0?
> > > > >
> > > > > Steve
> > > > >
> > > > > ----------------------------------------------------------------
> > > > > --
> > > > > --
> > > > > --
> > > > > --
> > > > >
> > > > > *From:*mapserver-users-bounces at lists.osgeo.org
> > > > > <mailto:mapserver-users-bounces at lists.osgeo.org>
> > > > > [mapserver-users-bounces at lists.osgeo.org
> > > > > <mailto:mapserver-users-bounces at lists.osgeo.org>] on behalf of
> > > > > Moen, Paul T. [pmoen at nd.gov <mailto:pmoen at nd.gov>]
> > > > > *Sent:* Tuesday, August 19, 2014 4:56 PM
> > > > > *To:* mapserver-users at lists.osgeo.org
> > > > > <mailto:mapserver-users at lists.osgeo.org>
> > > > > *Subject:* [mapserver-users] WMS request fails when spaces are
> > > > > encoded as plus symbol in query part of URL
> > > > >
> > > > > Hi all,
> > > > >
> > > > > We are consuming WMS services hosted by ESRI ArcGIS Server 10.0.
> > > > > The server was upgraded to ArcGIS Server 10.2.2 and we no longer
> > > > > can access layer names with spaces.
> > > > >
> > > > > This is the server's capabilities.
> > > > >
> > > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/W
> > > > > MS Se rv er?request=GetCapabilities&service=WMS
> > > > >
> > > > > This is the layer I am using.
> > > > >
> > > > > LAYER
> > > > >
> > > > > # DEBUG 5
> > > > >
> > > > > CONNECTIONTYPE WMS
> > > > >
> > > > > CONNECTION
> > > > >
> > > >
> > >
> >
> "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?"
> > > > >
> > > > > METADATA
> > > > >
> > > > > "wms_name" "Topomap DRG 250k"
> > > > >
> > > > > "wms_format" "image/png"
> > > > >
> > > > > "wms_server_version" "1.1.1"
> > > > >
> > > > > "wms_srs" "EPSG:2266"
> > > > >
> > > > > END
> > > > >
> > > > > TYPE RASTER
> > > > >
> > > > > STATUS OFF
> > > > >
> > > > > NAME "USGS DRG 250k Topo Maps"
> > > > >
> > > > > END
> > > > >
> > > > > Mapserver encodes this request as follows. Notice that the
> > > > > layer name, which is 'Topomap DRG 250k', is encoded with the
> > > > > spaces becoming the
> > > > '+'
> > > > > character.
> > > > >
> > > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/W
> > > > > MS
> > > > > Se
> > > > > rv
> > > > >
> > > >
> > >
> >
> er?LAYERS=Topomap+DRG+250k&REQUEST=map&WMTVER=1.0.0&SERVICE=W
> > > > MS&FORMAT
> > > > >
> > > >
> > >
> >
> =image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BBOX=19
> > > > 06240.1
> > > > >
> > > >
> > >
> >
> 5992838,596656.333359479,2119730.85012732,789150.692893686&TRANSPAR
> > > > ENT
> > > > > =TRUE&EXCEPTIONS=INIMAGE
> > > > >
> > > > > This returns the error in an image that says "Parameter 'layers'
> > > > > contains unacceptable layer names."
> > > > >
> > > > > When I replace the + in the layer name with %20,
> > > > > LAYERS=Topomap+DRG+250k become
> LAYERS=Topomap%20DRG%20250k
> > > > and the request becomes the following.
> > > > >
> > > > > http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/W
> > > > > MS
> > > > > Se
> > > > > rv
> > > > >
> > > >
> > >
> >
> er?LAYERS=Topomap%20DRG%20250k&REQUEST=map&WMTVER=1.0.0&SERVI
> > > > CE=WMS&FO
> > > > >
> > > >
> > >
> >
> RMAT=image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BB
> > > > OX=19062
> > > > >
> > > >
> > >
> >
> 40.15992838,596656.333359479,2119730.85012732,789150.692893686&TRAN
> > > > SPA
> > > > > RENT=TRUE&EXCEPTIONS=INIMAGE
> > > > >
> > > > > That request works. Obviously, ESRI is no longer decoding + as
> > > > > a space and therefore does not recognize the layer name with spaces.
> > > > >
> > > > > Is there any way I can have mapserver encode all characters
> > > > > using percent-encoding, or at least encode spaces as %20 instead
> > > > > of a +
> > character.
> > > > >
> > > > > Thanks,
> > > > >
> > > > > Paul Moen
> > > > > pmoen at nd.gov <mailto:pmoen at nd.gov>
> > > > > 701-328-2434
> > > > >
> > > > >
> > > > _______________________________________________
> > > > mapserver-dev mailing list
> > > > mapserver-dev at lists.osgeo.org
> > > > http://lists.osgeo.org/mailman/listinfo/mapserver-dev
> > _______________________________________________
> > mapserver-users mailing list
> > mapserver-users at lists.osgeo.org
> > http://lists.osgeo.org/mailman/listinfo/mapserver-users
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
More information about the mapserver-users
mailing list