[mapserver-dev] Questions on MapServer architecture and attribute conversions

[Lime, Steve D (MNIT)]

Couple of questions here...

On your pull request. I don't think there's a need for a RFC given the scope of the change. it's not breaking anything and seems like a nice addition - I certainly defer to your expertise as the Python MapScript maintainer. I guess it's never a bad idea to alert the -dev list and take your email as that notice - so +1 from me.

On your assumptions, both are correct. 

On item types, it's funny this hasn't come up more over the years. Type conversions seem to happen either for output using the metadata referenced or in the expression handling based on context. You're correct the that MapScript, most of the time you'd get the column/item names from layer but not always since you can work with shapefiles directly and independently of a layerObj or mapObj.

Each layer has an attribute has something called iteminfo and it holds the index value associated with corresponding value in the items array. So you'd look up the item someone wanted for labeling purposes (or whatever) and would know what value to grab from a shapes values array. Those item index values are stored all over the place (e.g. labelitem and labelitemindex). I think with a scheme like Even describes items/iteminto could be replaced by one property (items) that would equate to an array of field definitions and then specific things like a labelitem becomes a struct consisting of a string and a pointer to one of the items. For some reason I thought Dan made a run at that at one point but I might be mistaken.

-----Original Message-----
From: mapserver-dev [mailto:mapserver-dev-bounces at lists.osgeo.org] On Behalf Of Even Rouault
Sent: Thursday, March 14, 2019 11:39 AM
To: mapserver-dev at lists.osgeo.org
Subject: Re: [mapserver-dev] Questions on MapServer architecture and attribute conversions

Seth,

> 
> I've created a pull request to add a __geo_interface__ to the Python
> MapScript PointObj, LineObj, and ShapeObjs. This allows Python scripts to
> easily share MapScript objects with other Python geospatial libraries such
> as Shapely, QGIS, and ArcPy. It also completes a nice cycle as Sean Gillies
> was a core developer of the MapScript Python bindings and created
> __geo_interface__.  This only affects the Python MapScript bindings - do I
> need to create a RFC or ask for a vote to merge this for a 7.4 release?
> 
> I also have a few questions on the internal architecture of MapServer
> classes. Could other devs confirm (or correct) the following assumptions?
> 
> - the shapeObj has no link back to a layerObj - they are completely
> independent with no way to get a reference to a layer from a shape - all
> shapeObj attribute values are stored internally as strings

Yes that's my understanding too.

> 
> To get attribute names for a shapeObj I had to get retrieve them  from the
> relevant layerObj and store these in a new property on the shapeObj in
> Python MapScript. I can then output something like the following:
> 
>     "properties": {
>         "guid": "954BADBF-2891-48ED-A132-AED39C60E4C9",
>         "featureid": "203529",
>         "classid": "12"
>     }
> 
> It would be nice to be able to convert the property values to their relevant
> types e.g. integers and floats. Looking through the MapServer source they
> seem to only be converted to types using METADATA blocks and
> "gml_FIELD_NAME_type" "datatype" and in certain cases e.g. in
> mapogroutput.c Are these conversions all handled differently for different
> data inputs? Is there an easy way to reuse these conversions in MapScript?

Yes this lack of proper typing on layer attributes is a bit inconvenient and 
causes lot of adhoc coding in various areas of the code base: ogroutput, OGC 
filtering, WFS, GML output, MVT output, etc.
The fundamental issue is that layerObj::items is just an array of field names.

For brainstorming... A potential solution could take inspiration from what is 
done in GDAL where:
OGRLayer (~ equivalent of layerObj) -> contains a OGRFeatureDefn
OGRFeature (~ equivalent of shapeObj) -> points to a OGRFeatureDefn, and has 
an array of field values

A OGRFeatureDefn contains (among other things) an array of OGRFieldDefn
A OGRFieldDefn is a set of attribute: field name, field type, etc...

Or if not doing a majorhaul, have indeed a pointer from shapeObj back to 
layerObj (which interesting lifetime consideration, but I'm not sure MapScript 
really shines on maintaining object ownership), and use information returned 
by msGMLGetItems().


Even


-- 
Spatialys - Geospatial professional services
https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com&data=02%7C01%7Csteve.lime%40state.mn.us%7Cbc8c4af6794f4e52259608d6a89c923d%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636881787755412437&sdata=IPFoO8gWTM72w7ROYMPbYcLwGyWSEPkGga0rllSx%2FhI%3D&reserved=0
_______________________________________________
mapserver-dev mailing list
mapserver-dev at lists.osgeo.org
https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7Cbc8c4af6794f4e52259608d6a89c923d%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636881787755412437&sdata=hYFnDNHUyhJF1VRBQXLYXKwaBjCD4q3G9QL%2B346FOZw%3D&reserved=0