[mapserver-dev] Questions on MapServer architecture and attribute conversions

[Seth G]

Hi Steve, Even,

Thanks for your replies. Thinking back I  think I've run into the issue of strings being used for int fields from a database, and CLASS expressions. 

I may make an attempt at the following based on both of your comments:

- add gmlItemObj (and also gmlItemListObj) as classes to MapScript
- allow msGMLGetItems to be called on a layerObj to populate this list
- allow this list to be associated/passed to a method on a shapeObj, and use the GML types to set the output attribute types
- if there are no field definitions supplied then attribute names will simply be indexes, and all values strings

Seth
--
web:http://geographika.co.uk
twitter: @geographika

On Thu, Mar 14, 2019, at 10:14 PM, Lime, Steve D (MNIT) wrote:
> Couple of questions here...
> 
> On your pull request. I don't think there's a need for a RFC given the 
> scope of the change. it's not breaking anything and seems like a nice 
> addition - I certainly defer to your expertise as the Python MapScript 
> maintainer. I guess it's never a bad idea to alert the -dev list and 
> take your email as that notice - so +1 from me.
> 
> On your assumptions, both are correct. 
> 
> On item types, it's funny this hasn't come up more over the years. Type 
> conversions seem to happen either for output using the metadata 
> referenced or in the expression handling based on context. You're 
> correct the that MapScript, most of the time you'd get the column/item 
> names from layer but not always since you can work with shapefiles 
> directly and independently of a layerObj or mapObj.
> 
> Each layer has an attribute has something called iteminfo and it holds 
> the index value associated with corresponding value in the items array. 
> So you'd look up the item someone wanted for labeling purposes (or 
> whatever) and would know what value to grab from a shapes values array. 
> Those item index values are stored all over the place (e.g. labelitem 
> and labelitemindex). I think with a scheme like Even describes 
> items/iteminto could be replaced by one property (items) that would 
> equate to an array of field definitions and then specific things like a 
> labelitem becomes a struct consisting of a string and a pointer to one 
> of the items. For some reason I thought Dan made a run at that at one 
> point but I might be mistaken.
> 
> -----Original Message-----
> From: mapserver-dev [mailto:mapserver-dev-bounces at lists.osgeo.org] On 
> Behalf Of Even Rouault
> Sent: Thursday, March 14, 2019 11:39 AM
> To: mapserver-dev at lists.osgeo.org
> Subject: Re: [mapserver-dev] Questions on MapServer architecture and 
> attribute conversions
> 
> Seth,
> 
> > 
> > I've created a pull request to add a __geo_interface__ to the Python
> > MapScript PointObj, LineObj, and ShapeObjs. This allows Python scripts to
> > easily share MapScript objects with other Python geospatial libraries such
> > as Shapely, QGIS, and ArcPy. It also completes a nice cycle as Sean Gillies
> > was a core developer of the MapScript Python bindings and created
> > __geo_interface__.  This only affects the Python MapScript bindings - do I
> > need to create a RFC or ask for a vote to merge this for a 7.4 release?
> > 
> > I also have a few questions on the internal architecture of MapServer
> > classes. Could other devs confirm (or correct) the following assumptions?
> > 
> > - the shapeObj has no link back to a layerObj - they are completely
> > independent with no way to get a reference to a layer from a shape - all
> > shapeObj attribute values are stored internally as strings
> 
> Yes that's my understanding too.
> 
> > 
> > To get attribute names for a shapeObj I had to get retrieve them  from the
> > relevant layerObj and store these in a new property on the shapeObj in
> > Python MapScript. I can then output something like the following:
> > 
> >     "properties": {
> >         "guid": "954BADBF-2891-48ED-A132-AED39C60E4C9",
> >         "featureid": "203529",
> >         "classid": "12"
> >     }
> > 
> > It would be nice to be able to convert the property values to their relevant
> > types e.g. integers and floats. Looking through the MapServer source they
> > seem to only be converted to types using METADATA blocks and
> > "gml_FIELD_NAME_type" "datatype" and in certain cases e.g. in
> > mapogroutput.c Are these conversions all handled differently for different
> > data inputs? Is there an easy way to reuse these conversions in MapScript?
> 
> Yes this lack of proper typing on layer attributes is a bit inconvenient and 
> causes lot of adhoc coding in various areas of the code base: ogroutput, OGC 
> filtering, WFS, GML output, MVT output, etc.
> The fundamental issue is that layerObj::items is just an array of field names.
> 
> For brainstorming... A potential solution could take inspiration from what is 
> done in GDAL where:
> OGRLayer (~ equivalent of layerObj) -> contains a OGRFeatureDefn
> OGRFeature (~ equivalent of shapeObj) -> points to a OGRFeatureDefn, and has 
> an array of field values
> 
> A OGRFeatureDefn contains (among other things) an array of OGRFieldDefn
> A OGRFieldDefn is a set of attribute: field name, field type, etc...
> 
> Or if not doing a majorhaul, have indeed a pointer from shapeObj back to 
> layerObj (which interesting lifetime consideration, but I'm not sure MapScript 
> really shines on maintaining object ownership), and use information returned 
> by msGMLGetItems().
> 
> 
> Even
> 
> 
> -- 
> Spatialys - Geospatial professional services
> https://gcc01.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.spatialys.com&data=02%7C01%7Csteve.lime%40state.mn.us%7Cbc8c4af6794f4e52259608d6a89c923d%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636881787755412437&sdata=IPFoO8gWTM72w7ROYMPbYcLwGyWSEPkGga0rllSx%2FhI%3D&reserved=0
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> https://gcc01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.osgeo.org%2Fmailman%2Flistinfo%2Fmapserver-dev&data=02%7C01%7Csteve.lime%40state.mn.us%7Cbc8c4af6794f4e52259608d6a89c923d%7Ceb14b04624c445198f26b89c2159828c%7C0%7C0%7C636881787755412437&sdata=hYFnDNHUyhJF1VRBQXLYXKwaBjCD4q3G9QL%2B346FOZw%3D&reserved=0
> _______________________________________________
> mapserver-dev mailing list
> mapserver-dev at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/mapserver-dev