[mapserver-users] WFS layer - XML/JSON safe chars
Ian Walberg
ian.walberg at airborne.aero
Fri Apr 22 07:09:30 PDT 2016
Folks,
I think we may be taking about different things, the wfs requests below show the problem we are seeing.
There are 2 cities in each of the sqlite databases and the only difference is on has the text 'and' and the other has the symbol '&'
Layer DATA_TEST - INSERT INTO "test" VALUES('-14.235004','-51.92528','Rio and Paulo');
http://wfs-dev.airborne.aero/cgi-bin/mapserv?map=/data/wfs/WFS_TEST.map&SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=DATA_TEST
Layer DATA_TEST2 - INSERT INTO "test" VALUES('-14.235004','-51.92528','Rio & Paulo');
http://wfs-dev.airborne.aero/cgi-bin/mapserv?map=/data/wfs/WFS_TEST.map&SERVICE=WFS&VERSION=1.1.0&REQUEST=GetFeature&TYPENAME=DATA_TEST2
So our question is do the strings in database need to be xml safe or does mapserver check that the strings are safe before serving.
Thanks
Ian
-----Original Message-----
From: Even Rouault [mailto:even.rouault at spatialys.com]
Sent: Friday, April 15, 2016 9:15 AM
To: Ian Walberg <ian.walberg at airborne.aero>
Cc: mapserver-users at lists.osgeo.org
Subject: Re: [mapserver-users] WFS layer - XML/JSON safe chars
Le vendredi 15 avril 2016 17:32:36, Ian Walberg a écrit :
> If I understand this we would not get the issue if we use a WFS
> version
> 1.0.0 request.
>
> We have just tested that and it appears that changing the version does
> not change the behavior.
>
> This is with mapserver version 7.0.1.
We are probably talking about different things.
In
https://github.com/mapserver/mapserver/commit/105877468590e99401d234e1bbac13d7645b77d0
, I have just fixed issues in GetCapabilities response generation, but your issues might be different.
I've also just tested reading from a CSV (the input datastore type shouldn't
matter) file a field with & and it is properly escaped as & in the GML output of WFS 1.0, 1.1 and 2.0
Without a mapfile + data + request you issue, it is difficult to be sure we're talking about the same thing.
>
> Thanks
>
> Ian
>
> -----Original Message-----
> From: mapserver-users [mailto:mapserver-users-bounces at lists.osgeo.org]
> On Behalf Of Even Rouault Sent: Friday, April 15, 2016 7:50 AM
> To: mapserver-users at lists.osgeo.org
> Subject: Re: [mapserver-users] WFS layer - XML/JSON safe chars
>
> Le vendredi 15 avril 2016 16:41:28, Richard Watson a écrit :
> > I am using Map Server 6.4.1. I have a map project file which has
> > the
> >
> > following title:
> > WEB
> >
> > METADATA
> >
> > [ text omitted]
> > "wms_title" "(*#(@#*$_)@#&\\\///"
> >
> > END # METADATA
> >
> > END # WEB
> >
> > When a GetCapabilties is done on the project then this is the source
> > XML which is returned:
> >
> > <Service>
> > <Name>WMS</Name>
> > <Title>(*#(@#*$_)@#&\\///</Title>
> > It seems to be escaping as required.
>
> WMS and WFS 1.0 use MapServer manual XML composition with its own XML
> escaping routines. Some other services (such as WFS 1.1 & 2.0) use
> libxml (sometimes not in an appropriate way).
>
> > On Fri, Apr 15, 2016 at 9:21 AM, Rahkonen Jukka (MML) <
> >
> > jukka.rahkonen at maanmittauslaitos.fi> wrote:
> > > Hi,
> > >
> > >
> > >
> > > I was thinking that the output of GetCapabilities is XML and &
> > > must be escaped in XML. So if I write & into the abstract and
> > > Mapserver works correctly it should write my test abstract “test ä
> > > &” escaped perhaps as
> > >
> > > test ä &
> > >
> > > That is the output from the online tool at
> > > http://www.freeformatter.com/xml-escape.html#ad-output
> > >
> > >
> > >
> > > So my conclusion was that because Mapserver does not do it right
> > > here it won’t do it right elsewhere.
> > >
> > >
> > >
> > > But now I made another test by editing the Geoserver test file
> > > “states.shp” in Spatialite as
> > >
> > > UPDATE states set STATE_NAME=’Colo&rado’ WHERE
> > > STATE_NAME=’Colorado’;
> > >
> > >
> > >
> > > Output of GetFeature from WFS 1.1.0 from Mapserver 7.0.0 includes
> > >
> > > <ms:STATE_NAME>Colo&rado</ms:STATE_NAME>
> > >
> > > So my current conclusion is that my Mapserver does not escape “&”
> > > correctly when it is used inside a mapfile and when I do
> > > GetCapabilities request. However, when data come from Spatialite
> > > database and they are consumed by GetFeature then escaping happens
> > > correctly.
> > >
> > >
> > >
> > > We are still comparing different things because you have PostGIS
> > > and I have Spatialite.
> > >
> > >
> > >
> > > -Jukka Rahkonen-
> > >
> > >
> > >
> > >
> > >
> > > Ian Walberg wrote:
> > >
> > >
> > >
> > > Jukka,
> > >
> > >
> > >
> > > I am not sure I understand the test you did.
> > >
> > >
> > >
> > > The data we are referring to is from the database field for the
> > > individual features name.
> > >
> > >
> > >
> > > Regards
> > >
> > >
> > >
> > > Ian
> > >
> > >
> > >
> > > *From:* Rahkonen Jukka (MML)
> > > [mailto:jukka.rahkonen at maanmittauslaitos.fi
> > > <jukka.rahkonen at maanmittauslaitos.fi>]
> > > *Sent:* Thursday, April 14, 2016 10:19 PM
> > > *To:* Ian Walberg <ian.walberg at airborne.aero>; MapserverList OSGEO
> > > < mapserver-users at lists.osgeo.org>
> > > *Subject:* Re: WFS layer - XML/JSON safe chars
> > >
> > >
> > >
> > > Hi Ian,
> > >
> > >
> > >
> > > Thanks for the clarification.
> > >
> > > I made a simple test by adding an abstract into layer metadata.
> > > This shows “test ä” as an abstract
> > >
> > > "wfs_abstract" "test ä”
> > >
> > >
> > >
> > > This one shows an empty abstract
> > >
> > > "wfs_abstract" "test ä &"
> > >
> > >
> > >
> > > So it looks like Mapserver doesn’t know to convert “&” into “&”
> > > for XML.
> > >
> > >
> > >
> > > -Jukka Rahkonen-
> > >
> > >
> > >
> > > Ian Walberg wrote:
> > >
> > >
> > >
> > > Jukka,
> > >
> > >
> > >
> > > Our question was for the data/text read from the database and then
> > > published as XML by the WFS service.
> > >
> > >
> > >
> > > So a pub with the name ‘Dog & bone’ created illegal xml as the &
> > > is not converted to &
> > >
> > >
> > >
> > > Regards
> > >
> > >
> > >
> > > Ian
> > >
> > >
> > >
> > > *From:* Rahkonen Jukka (MML)
> > > [mailto:jukka.rahkonen at maanmittauslaitos.fi
> > > <jukka.rahkonen at maanmittauslaitos.fi>]
> > > *Sent:* Thursday, April 14, 2016 3:10 PM
> > > *To:* Ian Walberg <ian.walberg at airborne.aero>; MapserverList OSGEO
> > > < mapserver-users at lists.osgeo.org>
> > > *Subject:* Re: WFS layer - XML/JSON safe chars
> > >
> > >
> > >
> > > Hi,
> > >
> > >
> > >
> > > You can’t have & in feature type names or attribute names in WFS
> > > because they are defined in the XML schema to be “QName”. The
> > > prefix and LocalPart are of type NCName
> > > https://en.wikipedia.org/wiki/QName
> > > and from
> > > http://stackoverflow.com/questions/1631396/what-is-an-xsncname-typ
> > > e-
> > > and-w
> > > hen-should-it-be-used
> > >
> > >
> > >
> > > The practical restrictions of NCName are that it cannot contain
> > > several symbol characters like :, @, $, %, &, /, +, ,, ;,
> > > whitespace characters or different parenthesis. Furthermore an
> > > NCName cannot begin with a number, dot or minus character although
> > > they can appear later in an NCName.
> > >
> > >
> > >
> > > -Jukka Rahkonen-
> > >
> > >
> > >
> > > *Lähettäjä:* mapserver-users [
> > > mailto:mapserver-users-bounces at lists.osgeo.org
> > > <mapserver-users-bounces at lists.osgeo.org>] *Puolesta *Ian Walberg
> > > *Lähetetty:* 14. huhtikuuta 2016 19:23
> > > *Vastaanottaja:* MapserverList OSGEO
> > > *Aihe:* [mapserver-users] WFS layer - XML/JSON safe chars
> > >
> > >
> > >
> > > Folks,
> > >
> > >
> > >
> > > Can mapserver produce XML or JSON safe characters for WFS served
> > > data or do we need to ensure the source data has the escaped values.
> > >
> > >
> > >
> > > We got tripped up today as there were some ‘&’ in our names which
> > > we had not considered.
> > >
> > >
> > >
> > > Thanks
> > >
> > >
> > >
> > > Ian
> > >
> > > _______________________________________________
> > > mapserver-users mailing list
> > > mapserver-users at lists.osgeo.org
> > > http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
> --
> Spatialys - Geospatial professional services http://www.spatialys.com
> _______________________________________________ mapserver-users
> mailing list mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
--
Spatialys - Geospatial professional services http://www.spatialys.com
More information about the mapserver-users
mailing list