[mapserver-users] WMS request fails when spaces are encoded as plus symbol in query part of URL

Mon Jul 6 12:12:16 PDT 2015

An update.

ESRI has decided not to follow the standard and has closed the bug, NIM104744, we submitted about not decoding a plus symbol ‘+’ to a space.  Their solution is for everyone else to encode all spaces as %20 and to ignore http://tools.ietf.org/html/rfc3986.  They have closed the bug and listed it as a known limit.

http://support.esri.com/en/bugs/nimbus/TklNMTA0NzQ0

So, in order for Mapserver to consume ESRI WMS services, with spaces in the name, the spaces have to be encoded as %20.

I see Mapserver 7.0 still encodes spaces to a ‘+’, so I thought people should know if they intend to consume any ESRI WMS layers with spaces in the name.

From: Paul Moen <pmoen at nd.gov<mailto:pmoen at nd.gov>>
Date: Wednesday, August 20, 2014 at 9:43 AM
To: "Lime, Steve D (MNIT)" <Steve.Lime at state.mn.us<mailto:Steve.Lime at state.mn.us>>, "mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>" <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>>
Subject: Re: [mapserver-users] WMS request fails when spaces are encoded as plus symbol in query part of URL

Steve,

You are right about the outcome of 1 and 2.  1 encodes the % and 2 throws the following error.

msBuildWMSLayerURLBase(): One of wms_onlineresource, wms_server_version, wms_name metadata is missing in layer USGS DRG 250k Topo Maps.  Please either provide a valid CONNECTION URL, or provide those values in the layer's metadata.\n\n

I found the function as you said and
I removed the

 if (*i == ' ')
      *j = '+';
    else
from the function then recompiled.

char *msEncodeUrlExcept(const char *data, const char except)
{
  char *hex = "0123456789ABCDEF";
  const char *i;
  char  *j, *code;
  int   inc;
  unsigned char ch;

  for (inc=0, i=data; *i!='\0'; i++)
    if (msEncodeChar(*i))
      inc += 2;

  code = (char*)msSmallMalloc(strlen(data)+inc+1);

  for (j=code, i=data; *i!='\0'; i++, j++) {
    if ( except != '\0' && *i == except ) {
      *j = except;
    } else if (msEncodeChar(*i)) {
      ch = *i;
      *j++ = '%';
      *j++ = hex[ch/16];
      *j   = hex[ch%16];
    } else
      *j = *i;
  }
  *j = '\0';

  return code;
}

Everything works again after a mapserver recompile, install and finally a restart of apache.  Thanks so much for the path to the solution.

Paul Moen
pmoen at nd.gov<mailto:pmoen at nd.gov>
701-328-2434

From: <Lime>, "Steve D (MNIT)" <Steve.Lime at state.mn.us<mailto:Steve.Lime at state.mn.us>>
Date: Wednesday, August 20, 2014 at 12:41 AM
To: Paul Moen <pmoen at nd.gov<mailto:pmoen at nd.gov>>, "mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>" <mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>>
Subject: RE: [mapserver-users] WMS request fails when spaces are encoded as plus symbol in query part of URL

Hmmm... Nice backwards compatibility ESRI. The +'s seem to still be quite legal (http://tools.ietf.org/html/rfc3986 and other references) in the query string (but not the path). I don't have access to test and I'm guessing these ideas won't work but they might be worth a quick try:

  1) Try encoding the wms_name in the metadata: "wms_name" "Topomap%20DRG%20250k"
  2) Don't set the wms_name in the metadata but add it to the connection: CONNECTION "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?LAYERS=Topomap%20DRG%20250k&"

I'm betting MapServer will encode the %'s in 1 and throw an error in 2. Otherwise you could hack the MapServer source. Function is called msEncodeUrlExcept() in mapstring.c you can see the section to change starting at line 1138 in git master (https://github.com/mapserver/mapserver/blob/master/mapstring.c)<https://github.com/mapserver/mapserver/blob/master/mapstring.c).>

Maybe it's worth making this change as part of 7.0?

Steve

________________________________
From:mapserver-users-bounces at lists.osgeo.org<mailto:mapserver-users-bounces at lists.osgeo.org> [mapserver-users-bounces at lists.osgeo.org<mailto:mapserver-users-bounces at lists.osgeo.org>] on behalf of Moen, Paul T. [pmoen at nd.gov<mailto:pmoen at nd.gov>]
Sent: Tuesday, August 19, 2014 4:56 PM
To: mapserver-users at lists.osgeo.org<mailto:mapserver-users at lists.osgeo.org>
Subject: [mapserver-users] WMS request fails when spaces are encoded as plus symbol in query part of URL

Hi all,

We are consuming WMS services hosted by ESRI ArcGIS Server 10.0.  The server was upgraded to ArcGIS Server 10.2.2 and we no longer can access layer names with spaces.

This is the server’s capabilities.
http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?request=GetCapabilities&service=WMS

This is the layer I am using.

LAYER
# DEBUG 5
CONNECTIONTYPE WMS
CONNECTION "http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?"
METADATA
"wms_name" "Topomap DRG 250k"
"wms_format" "image/png"
"wms_server_version" "1.1.1"
"wms_srs" "EPSG:2266"
END
TYPE RASTER
STATUS OFF
NAME "USGS DRG 250k Topo Maps"
END

Mapserver encodes this request as follows.  Notice that the layer name, which is ‘Topomap DRG 250k’, is encoded with the spaces becoming the ‘+’ character.

http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?LAYERS=Topomap+DRG+250k&REQUEST=map&WMTVER=1.0.0&SERVICE=WMS&FORMAT=image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BBOX=1906240.15992838,596656.333359479,2119730.85012732,789150.692893686&TRANSPARENT=TRUE&EXCEPTIONS=INIMAGE

This returns the error in an image that says “Parameter ‘layers’ contains unacceptable layer names.”

When I replace the + in the layer name with %20, LAYERS=Topomap+DRG+250k become LAYERS=Topomap%20DRG%20250k and the request becomes the following.

http://ndgishub.nd.gov/arcgis/services/All_Elevation/MapServer/WMSServer?LAYERS=Topomap%20DRG%20250k&REQUEST=map&WMTVER=1.0.0&SERVICE=WMS&FORMAT=image/png&STYLES=&HEIGHT=1146&SRS=EPSG:2266&WIDTH=1271&BBOX=1906240.15992838,596656.333359479,2119730.85012732,789150.692893686&TRANSPARENT=TRUE&EXCEPTIONS=INIMAGE

That request works.  Obviously, ESRI is no longer decoding + as a space and therefore does not recognize the layer name with spaces.

Is there any way I can have mapserver encode all characters using percent-encoding, or at least encode spaces as %20 instead of a + character.

Thanks,

Paul Moen
pmoen at nd.gov<mailto:pmoen at nd.gov>
701-328-2434

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20150706/dcfca6fb/attachment.htm>