[mapserver-dev] Questions on MapServer architecture and attribute conversions

Seth G sethg at geographika.co.uk
Sun Mar 17 07:43:09 PDT 2019


Hello again,

After reading through various header files again and the SWIG docs I think I understand better how it fits together. 

#ifndef SWIG..#endif guards are added throughout the header file so when it is included via %include "../../mapows.h" only the selected functions are exposed. 

I'm not sure if there is a standard way of adding the guards - it seems to be added for individual structs and groups of functions throughout the header file so I tried to fit with this convention - it does make for a large diff though to review. 

Now only gmlItemObj is exposed (as an immutable object), and gmlItemListObj (although this will only be available when USE_WMS_SVR and USE_WFS_SVR are defined. 

Exposing only the relevant parts of the header file fixes the C# issue. 

Seth


--
web:http://geographika.co.uk
twitter: @geographika


On Sun, Mar 17, 2019, at 12:48 AM, Seth G wrote:
> Hi,
> 
> I implemented a simple addition to MapScript to retrieve the gmlItemObj for a layer item (field) and added to the pull request. 
> This currently works for the Python bindings but is throwing an error for C# I'll look into. 
> 
> To use gmlItemObj I had to include mapows.h in mapscript.i. When I initially did this there were 28 errors similar to below:
> 
> mapscriptPYTHON_wrap.obj : error LNK2019: unresolved external symbol msWMSDispatch referenced in function _wrap_msWMSDispatch
> 
> The relevant commit is at: https://github.com/mapserver/mapserver/pull/5765/commits/0260296eecfb8b9bad979fc20ba5f9034ca37ffb (Ill see what happened to the whitespace in this commit and revert with just the relevant changes). 
> 
> I simply added the macro MS_DLL_EXPORT to export all these functions and was able to build and test the MapScript bindings. 
> All the other header files included in SWIG have #ifndef SWIG guards everywhere. Is anyone able to explain to me the purpose of these and provide examples of where these should be added to mapows.h?
> 
> Seth
> 
> --
> web:http://geographika.co.uk
> twitter: @geographika
> 
> 
> On Sat, Mar 16, 2019, at 11:58 AM, Lime, Steve D (MNIT) wrote:
>> I wish we could make it less GMLish and more generic. For example, it feels odd in MVT code to be relying on what feels like format-specific metadata. The various service specs can use a generic “ows” or service specific metadata namespaces. Could do the same for output... “out” as the generic namespace with “gml”, “mvt”, “ogr”, “kml” as specific namespaces.
>> 
>> gmlItemObj and gmlItemListObj could also be renamed more generically and the latter could become iteminfo as part of a layer. A cast function for type conversations would also be helpful for more standard use.
>> 
>> Just thinking out loud - I know that’s way more than you want to get into,
>> 
>> —Steve
>> 
>> 
>> *From:* mapserver-dev <mapserver-dev-bounces at lists.osgeo.org> on behalf of Seth G <sethg at geographika.co.uk>
>> *Sent:* Friday, March 15, 2019 6:58:41 PM
>> *To:* mapserver-dev at lists.osgeo.org
>> *Subject:* Re: [mapserver-dev] Questions on MapServer architecture and attribute conversions
>> 
>> Hi Steve, Even,
>> 
>> Thanks for your replies. Thinking back I think I've run into the issue of strings being used for int fields from a database, and CLASS expressions. 
>> 
>> I may make an attempt at the following based on both of your comments:
>> 
>> - add gmlItemObj (and also gmlItemListObj) as classes to MapScript
>> - allow msGMLGetItems to be called on a layerObj to populate this list
>> - allow this list to be associated/passed to a method on a shapeObj, and use the GML types to set the output attribute types
>> - if there are no field definitions supplied then attribute names will simply be indexes, and all values strings
>> 
>> Seth
>> --
>> web:http://geographika.co.uk
>> twitter: @geographika
>> 
>> On Thu, Mar 14, 2019, at 10:14 PM, Lime, Steve D (MNIT) wrote:
>> > Couple of questions here...
>> > 
>> > On your pull request. I don't think there's a need for a RFC given the 
>> > scope of the change. it's not breaking anything and seems like a nice 
>> > addition - I certainly defer to your expertise as the Python MapScript 
>> > maintainer. I guess it's never a bad idea to alert the -dev list and 
>> > take your email as that notice - so +1 from me.
>> > 
>> > On your assumptions, both are correct. 
>> > 
>> > On item types, it's funny this hasn't come up more over the years. Type 
>> > conversions seem to happen either for output using the metadata 
>> > referenced or in the expression handling based on context. You're 
>> > correct the that MapScript, most of the time you'd get the column/item 
>> > names from layer but not always since you can work with shapefiles 
>> > directly and independently of a layerObj or mapObj.
>> > 
>> > Each layer has an attribute has something called iteminfo and it holds 
>> > the index value associated with corresponding value in the items array. 
>> > So you'd look up the item someone wanted for labeling purposes (or 
>> > whatever) and would know what value to grab from a shapes values array. 
>> > Those item index values are stored all over the place (e.g. labelitem 
>> > and labelitemindex). I think with a scheme like Even describes 
>> > items/iteminto could be replaced by one property (items) that would 
>> > equate to an array of field definitions and then specific things like a 
>> > labelitem becomes a struct consisting of a string and a pointer to one 
>> > of the items. For some reason I thought Dan made a run at that at one 
>> > point but I might be mistaken.
>> > 
>> > -----Original Message-----
>> > From: mapserver-dev [mailto:mapserver-dev-bounces at lists.osgeo.org] On 
>> > Behalf Of Even Rouault
>> > Sent: Thursday, March 14, 2019 11:39 AM
>> > To: mapserver-dev at lists.osgeo.org
>> > Subject: Re: [mapserver-dev] Questions on MapServer architecture and 
>> > attribute conversions
>> > 
>> > Seth,
>> > 
>> > > 
>> > > I've created a pull request to add a __geo_interface__ to the Python
>> > > MapScript PointObj, LineObj, and ShapeObjs. This allows Python scripts to
>> > > easily share MapScript objects with other Python geospatial libraries such
>> > > as Shapely, QGIS, and ArcPy. It also completes a nice cycle as Sean Gillies
>> > > was a core developer of the MapScript Python bindings and created
>> > > __geo_interface__. This only affects the Python MapScript bindings - do I
>> > > need to create a RFC or ask for a vote to merge this for a 7.4 release?
>> > > 
>> > > I also have a few questions on the internal architecture of MapServer
>> > > classes. Could other devs confirm (or correct) the following assumptions?
>> > > 
>> > > - the shapeObj has no link back to a layerObj - they are completely
>> > > independent with no way to get a reference to a layer from a shape - all
>> > > shapeObj attribute values are stored internally as strings
>> > 
>> > Yes that's my understanding too.
>> > 
>> > > 
>> > > To get attribute names for a shapeObj I had to get retrieve them from the
>> > > relevant layerObj and store these in a new property on the shapeObj in
>> > > Python MapScript. I can then output something like the following:
>> > > 
>> > > "properties": {
>> > > "guid": "954BADBF-2891-48ED-A132-AED39C60E4C9",
>> > > "featureid": "203529",
>> > > "classid": "12"
>> > > }
>> > > 
>> > > It would be nice to be able to convert the property values to their relevant
>> > > types e.g. integers and floats. Looking through the MapServer source they
>> > > seem to only be converted to types using METADATA blocks and
>> > > "gml_FIELD_NAME_type" "datatype" and in certain cases e.g. in
>> > > mapogroutput.c Are these conversions all handled differently for different
>> > > data inputs? Is there an easy way to reuse these conversions in MapScript?
>> > 
>> > Yes this lack of proper typing on layer attributes is a bit inconvenient and 
>> > causes lot of adhoc coding in various areas of the code base: ogroutput, OGC 
>> > filtering, WFS, GML output, MVT output, etc.
>> > The fundamental issue is that layerObj::items is just an array of field names.
>> > 
>> > For brainstorming... A potential solution could take inspiration from what is 
>> > done in GDAL where:
>> > OGRLayer (~ equivalent of layerObj) -> contains a OGRFeatureDefn
>> > OGRFeature (~ equivalent of shapeObj) -> points to a OGRFeatureDefn, and has 
>> > an array of field values
>> > 
>> > A OGRFeatureDefn contains (among other things) an array of OGRFieldDefn
>> > A OGRFieldDefn is a set of attribute: field name, field type, etc...
>> > 
>> > Or if not doing a majorhaul, have indeed a pointer from shapeObj back to 
>> > layerObj (which interesting lifetime consideration, but I'm not sure MapScript 
>> > really shines on maintaining object ownership), and use information returned 
>> > by msGMLGetItems().
>> > 
>> > 
>> > Even
>> > 
>> > 
>> > -- 
>> > Spatialys - Geospatial professional services
>> >  https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com&data=02%7C01%7Csteve.lime%40state.mn.us%7C8af807e4be164b6cb28608d6a9a229ad%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636882911288492978&sdata=gq%2FwxERwIC7hIsUULQuBdz5NuWGPSjEEejynXTrU%2BWc%3D&reserved=0
>> > _______________________________________________
>> > mapserver-dev mailing list
>> > mapserver-dev at lists.osgeo.org
>> >  https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C8af807e4be164b6cb28608d6a9a229ad%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636882911288492978&sdata=K6js4DgRQJ4giCNpYqJ3oROAb7HQg3W6xHJaAbXgUxI%3D&reserved=0
>> > _______________________________________________
>> > mapserver-dev mailing list
>> > mapserver-dev at lists.osgeo.org
>> >  https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C8af807e4be164b6cb28608d6a9a229ad%7Ceb14b04624c445198f26b89c2159828c%7C0%7C1%7C636882911288502991&sdata=48CiR%2B8XMq030GchIzqmpAdse4sgV1b9Fvjk8Jzt%2Fq8%3D&reserved=0
>> _______________________________________________
>> mapserver-dev mailing list
>> mapserver-dev at lists.osgeo.org
>> https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7C8af807e4be164b6cb28608d6a9a229ad%7Ceb14b04624c445198f26b89c2159828c%7C0%7C1%7C636882911288502991&sdata=48CiR%2B8XMq030GchIzqmpAdse4sgV1b9Fvjk8Jzt%2Fq8%3D&reserved=0
> 
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-dev/attachments/20190317/9bb0452c/attachment-0001.html>


More information about the mapserver-dev mailing list