[mapserver-users] GeoJSON OGR output with WFS data type issue

[Kralidis,Tom [Ontario]]

Hi: using:

MapServer: 6.2.0
GDAL/OGR: 1.9.2
PostgreSQL: 8.4.13

$ cat /etc/issue
Ubuntu 10.04.4 LTS \n \l

...we have a MapServer setup and configured to feed downstream
applications with (point) observation data via WFS and GeoJSON.

We have an issue where a null value in an integer field is being
returned as a value of 0 in GeoJSON.  This is throwing off applications
who are processing 0 values.  The layer in question is a PostGIS
connection.

Below is a trace along the OGR/MapServer workflow:

PostgreSQL/PostGIS:

FOO=# select postgis_full_version();
 
postgis_full_version
------------------------------------------------------------------------
------------------------------------------------------------------------
 POSTGIS="2.0.2SVN r10398" GEOS="3.3.5-CAPI-1.7.5" PROJ="Rel. 4.8.0, 6
March 2012" GDAL="GDAL 1.9.2, released 2012/10/08" LIBXML="2.9.0" RASTER
(1 row)

FOO=# \q data_mart_ca_live;
....
air_temp_qa                    | integer                     
...

FOO=# select air_temp_qa from data_mart_ca_live where
data_payload_id=11177725;
 air_temp_qa
-------------

(1 row)

Here we see that air_temp_qa (integer value) returns null in SQL.

OGR:

$ ogrinfo -al "PG:dbname=FOO host=localhost user=foo password=bar"
data_mart_ca_live -where "data_payload_id = '11177725'"
...
# column definition
air_temp_qa: Integer (0.0)
...
# actual value
air_temp_qa (Integer) = (null)
...
$ ogr2ogr -select air_temp_qa -f GeoJSON foo.json "PG:dbname=FOO
host=localhost user=foo password=bar" data_mart_ca_live -where
"data_payload_id = '11177725'"
$ cat foo.json

"type": "FeatureCollection",

"features": [
{ "type": "Feature", "properties": { "air_temp": null, "air_temp_qa":
null }, "geometry": { "type": "Point", "coordinates": [ -111.933333,
60.033333 ] } }

]
}

So far all is consistent in OGR with what is represented in
PostgreSQL/PostGIS.

MapServer layer definition:

LAYER
 NAME "ca-1.0-ascii"
 STATUS ON
 DEBUG ON
 CONNECTION "dbname=FOO host=localhost user=foo password=bar"
 CONNECTIONTYPE POSTGIS
 PROCESSING "CLOSE_CONNECTION=DEFER"
 DATA "the_geom from data_mart_ca_live using unique data_payload_id
using srid=4326"
 TYPE POINT
 DUMP TRUE
PROJECTION
  "init=epsg:4326"
 END
 METADATA
  "ows_authorityurl_name" "taxonomy"
  "ows_authorityurl_href" "http://localhost/taxonomy"
  "ows_identifier_authority" "taxonomy"
  "ows_identifier_value"
"/msc/observation/atmospheric/surface_weather/ca-1.0-ascii"
  "wms_layer_group" "/msc/observation/atmospheric/surface_weather"
  "ows_title" "DMS CA obs data with qa"
  "ows_abstract" "DMS CA obs data with qa"
  "ows_keywordlist" "dms,climate,test,mapserver"
  "ows_extent" "-141.089000 36.392987 -52.089000 89.784987" # early out
  "wfs_metadataurl_format" "XML"
  "gml_include_items" "all"
  "gml_types" "auto"
  "wms_timeextent" "2013-01-11T18:39:34Z/2013-04-11T18:39:34Z"
  "wms_timeitem" "instance_datetime"
 END
 INCLUDE "/home/tomk/msc-ows/trunk/services/msc/classes/dashboard.inc"
END

- Test using WMS GetFeatureInfo returns air_temp_qa as an empty XML
element (which is expected/desired).

- Test using WFS GetFeature with default OUTPUTFORMAT:

$  GET
"http://localhost/msc-ows?service=WFS&version=1.1.0&request=GetFeature&t
ypename=ca-1.0-ascii&filter=<ogc:Filter><ogc:PropertyIsEqualTo><ogc:Prop
ertyName>data_payload_id</ogc:PropertyName><ogc:Literal>11177725</ogc:Li
teral></ogc:PropertyIsEqualTo></ogc:Filter>"

... returns air_temp_qa as an empty XML element (which is
expected/desired).

- Test using WFS GetFeature with OUTPUTFORMAT=GeoJSON:

$ GET
"http://localhost/msc-ows?service=WFS&version=1.1.0&request=GetFeature&t
ypename=ca-1.0-ascii&filter=<ogc:Filter><ogc:PropertyIsEqualTo><ogc:Prop
ertyName>data_payload_id</ogc:PropertyName><ogc:Literal>11177725</ogc:Li
teral></ogc:PropertyIsEqualTo></ogc:Filter>&outputformat=GeoJSON"

...returns air_temp_qa in GeoJSON with a value of 0 (i.e. "air_temp_qa":
0).

So it looks like, somewhere in MapServer, a null integer value is being
cast to 0.  What's weird is that this is happening only for OGR driver
requested formats, not native MapServer outputs.  But in OGR proper it
works as expected.

Other notes:

- LAYER.METADATA includes "gml_types" "auto".  When commenting that out,
the value comes back as blank (which is good), but all OGC Filters
(which are perfectly valid, from a wide variety of WFS clients) fail
because it looks like, without this, the client is forced to apply a
cast or quote the ogc:Literal value given the data type.

Any suggestions are valued.

Thanks

..Tom