<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none"><!--P{margin-top:0;margin-bottom:0;} .ms-cui-menu {background-color:#ffffff;border:1px rgb(171, 171, 171) solid;font-family:"Segoe UI WPC","Segoe UI",Tahoma,"Microsoft Sans Serif",Verdana,sans-serif;font-size:10pt;color:rgb(51, 51, 51);} .ms-cui-menusection-title {display:none;} .ms-cui-ctl {vertical-align:text-top;text-decoration:none;color:rgb(51, 51, 51);} .ms-cui-ctl-on {background-color:rgb(223, 237, 250);opacity: 0.8;} .ms-cui-img-cont-float {display:inline-block;margin-top:2px} .ms-cui-smenu-inner {padding-top:0px;} .ms-owa-paste-option-icon {margin: 2px 4px 0px 4px;vertical-align:sub;padding-bottom: 2px;display:inline-block;} .ms-rtePasteFlyout-option:hover {background-color:rgb(223, 237, 250) !important;opacity:1 !important;} .ms-rtePasteFlyout-option {padding:8px 4px 8px 4px;outline:none;} .ms-cui-menusection {float:left; width:85px;height:24px;overflow:hidden}--></style>
</head>
<body>
<div style="font-size:12pt;color:#000000;background-color:#FFFFFF;font-family:Calibri,Arial,Helvetica,sans-serif;">
<p>Hi,</p>
<p><br>
</p>
<p>As I wrote, I got a motivation for my first mail because I have seen that people are quite often using GeoJSON for delivering geospatial data as data, to be saved on disk and used like shapefiles, GML etc. As a result you get stuff like this:</p>
<p><a href="http://demo.opengeo.org/geoserver/wfs?service=wfs&version=1.0.0&request=getfeature&typename=topp:states&outputformat=application/json">http://demo.opengeo.org/geoserver/wfs?service=wfs&version=1.0.0&request=getfeature&typename=topp:states&outputformat=application/json</a></p>
<p><br>
</p>
<p>You wrote and I agree with it "that XML and JSON have very different strengths and use cases ". However, people do what they want and I do feel that GeoJSON will be used for use cases where XML could be stronger like as the only supported format in some
download services.</p>
<p><br>
</p>
<p>About the nonsensical 4-field schema, it is a little bit violent but just what about everybody who is using OpenStreetMap data is doing all the time. OSM features are pushed into traditional simple feature model and a set of tags are converted to attributes
in a fixed schema. There are lots of null fields in the data and even that is in a way nonsensical, it is also practical because it makes it possible to use osm2pgsql and PostGIS and Mapnik for rendering.<br>
</p>
<p><br>
</p>
<p>I am so fixated to consume data that I was not thinking at all about how to write GeoJSON with GDAL. I was just thinking that if some data are only available as GeoJSON, how users could convert it to PostGIS etc. so that the data types of the attributes
will be the same as in the original data.</p>
<p><br>
</p>
<p>Because GeoJSON will not carry the data types as a payload I suppose that the current guess-the-datatype approach is the best starting point. Workaround by using VRT as Even suggested is good for fine tuning and cast with SQL works as well. The correct datatypes
may still be somehow uncertain but perhaps those who maintain such services will announce the structure of their data on their web pages if they feel that it is important and they for example are awaiting data updates from users. When it comes to WFS, it seems
to be an easy case because the XML schema can be reused as "GeoJSON schema".</p>
<p><br>
</p>
<p>-Jukka Rahkonen-<br>
</p>
<p><br>
</p>
<p><br>
</p>
<div style="color: #282828;">
<hr tabindex="-1" style="display: inline-block; width: 98%;">
<div dir="ltr" id="divRplyFwdMsg"><font color="#000000" face="Calibri, sans-serif" style="font-size: 11pt;"><b></b>Sean Gillies <sean@mapbox.com><br>
<b></b></font> </div>
<div>
<div dir="ltr">> Hi Even, Jukka,
<div><br>
</div>
<div>> While the OGC service architecture is heavily dependent on schemas, OGR type schemas are not *generally* useful for GeoJSON. Consider the following abbreviated feature collection:</div>
<div><br>
</div>
<div>
<div> "features": [</div>
<div> {"properties": {"a": 0, "b": "lol"}, ...},</div>
<div> {"properties": {"c": "2014-11-21", "d": "wut"}, ...}</div>
<div> ]</div>
</div>
<div><br>
</div>
<div>> It has two features and they are distinctly different types. A schema that says these features have 4 fields would be nonsensical.</div>
<div><br>
</div>
<div>> There are a bunch of different JSON schema approaches and none of them seem to have any traction.
<a href="https://github.com/json-schema/json-schema">https://github.com/json-schema/json-schema</a> for example looks to be stalled. I think the lack of traction reflects some deeper reality: that XML and JSON have very different strengths and use cases and
that attempts to XML-ize JSON by adding schemas will always eventually run out of steam. </div>
<div><br>
</div>
<div>> For OGR to write schemas into GeoJSON would be a mistake. They could be misleading and because there will never (as far as I can tell) be consensus in the JSON community on the right form of schema, anything OGR implemented would end up being a "loser".</div>
<div><br>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Fri, Nov 21, 2014 at 6:28 AM, Even Rouault <span dir="ltr">
<<a target="_blank" href="mailto:even.rouault@spatialys.com">even.rouault@spatialys.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin: 0px 0px 0px 0.8ex; border-left: 1px solid #cccccc; padding-left: 1ex;">
Jukka,<br>
<br>
Data type guessing implemented in the OGR GeoJSON driver is quite natural hopefully.<br>
A whole scan of the GeoJSON file is made and the following rules are applied :<br>
- if an attribute has integer-only content --> Integer<br>
- if an attribute has an array of integer-only content --> IntegerList<br>
- if an attribute has integer or floating point content --> Real<br>
- if an attribute has an array of integer or floating point content --> RealList<br>
- if an attribute has an array of anything else content --> StringList<br>
- otherwise --> String<br>
<br>
With RFC 50 and other pending improvements in the driver:<br>
- if an attribute has boolean-only content --> Integer(Boolean)<br>
- if an attribute has an array of boolean-only content --> IntegerList(Boolean)<br>
- if an attribute has date-only content --> Date<br>
- if an attribute has time-only content --> Time<br>
- if an attribute has datetime or date content --> DateTime<br>
<br>
I'm not sure we want to invent a .jsont format, but if you download<br>
<a target="_blank" href="http://svn.osgeo.org/gdal/trunk/gdal/swig/python/samples/ogr2vrt.py">http://svn.osgeo.org/gdal/trunk/gdal/swig/python/samples/ogr2vrt.py</a><br>
<br>
and run :<br>
<br>
python ogr2vrt.py "<a target="_blank" href="http://demo.opengeo.org/geoserver/wfs?service=wfs&version=1.0.0&request=getfeature&typename=topp:states&outputformat=json">http://demo.opengeo.org/geoserver/wfs?service=wfs&version=1.0.0&request=getfeature&typename=topp:states&outputformat=json</a>"
test.vrt<br>
<br>
This will create you a VRT with the default schema, that you can easily edit.<br>
Note: as with OGR SQL CAST, this is post processing. So if the guess done by the GeoJSON driver<br>
leads to a loss of information, you cannot recover it. Hopefully the implemented rules will not<br>
lead to information loss.<br>
<br>
A better approach would be to have the schema embedded in a JSON way in the GeoJSON file itself.<br>
That could be an evolution of the format, but I'm not sure this would be really popular,<br>
given JSON/GeoJSON is heavily used by NoSQL approaches...<br>
<br>
Hum, doing a quick search, I just found <a target="_blank" href="http://json-schema.org/">
http://json-schema.org/</a> that appears to be an IETF draft.<br>
It doesn't look that the schema is embedded in the data file itself.<br>
<br>
There's also GeoJSON-LD that might be a bit related : <a target="_blank" href="https://github.com/geojson/geojson-ld">
https://github.com/geojson/geojson-ld</a><br>
<br>
CC'ing Sean in case he has thoughts on this.<br>
<br>
Even<br>
<div class="HOEnZb">
<div class="h5"><br>
> Hi,<br>
><br>
> I wonder if GDAL could have some simple and relatively user friendly way<br>
> for defining a schema for GeoJSON data. The GeoJSON driver seems to guess<br>
> the data types of attributes with some undocumented way but users could<br>
> have better knowledge about the desired schema.<br>
><br>
> I know I can control the data type by using OGR SQL and CAST as in<br>
> ogrinfo -sql "select cast(EMPLOYED as float) from OGRGeojson" states.json<br>
> -so<br>
><br>
> However, perhaps GeoJSON is enough popular for deserving an easier way for<br>
> writing a schema. First I thought that it would be enough to copy the<br>
> "csvt" text file mechanism from the GDAL CSV driver<br>
> <a target="_blank" href="http://www.gdal.org/drv_csv.html">http://www.gdal.org/drv_csv.html</a>. However, the csvt file is a plain list of<br>
> types which will be applied to the attributes in the same order than they<br>
> appear in the text file<br>
> "Integer(5)","Real(10.7)","String(15)"<br>
><br>
> For GeoJSON it would feel more user friendly to include the attribute names<br>
> in the list somehow like<br>
> "population;Integer(5)","area;Real(10.7)","name;String(15)".<br>
><br>
> This would make it easier for users to write a valid "jsont" file. A list<br>
> with attribute names could perhaps also help GDAL as well because the<br>
> features in GeoJSON file do not necessarily have same attributes.<br>
><br>
> As an example this is the right schema for a WFS feature type which is<br>
> captured from<br>
> <a target="_blank" href="http://demo.opengeo.org/geoserver/wfs?service=wfs&version=1.0.0&request=des">
http://demo.opengeo.org/geoserver/wfs?service=wfs&version=1.0.0&request=des</a><br>
> cribefeaturetype&typename=topp:states<br>
><br>
><br>
> name="the_geom" type="gml:MultiPolygonPropertyType"/><br>
> name="STATE_NAME" type="xsd:string"/><br>
> name="STATE_FIPS" type="xsd:string"/><br>
> name="SUB_REGION" type="xsd:string"/><br>
> name="STATE_ABBR" type="xsd:string"/><br>
> name="LAND_KM" type="xsd:double"/><br>
> name="WATER_KM" type="xsd:double"/><br>
> name="PERSONS" type="xsd:double"/><br>
> name="FAMILIES" type="xsd:double"/><br>
> name="HOUSHOLD" type="xsd:double"/><br>
> name="MALE" type="xsd:double"/><br>
> name="FEMALE" type="xsd:double"/><br>
> name="WORKERS" type="xsd:double"/><br>
> name="DRVALONE" type="xsd:double"/><br>
> name="CARPOOL" type="xsd:double"/><br>
> name="PUBTRANS" type="xsd:double"/><br>
> name="EMPLOYED" type="xsd:double"/><br>
> name="UNEMPLOY" type="xsd:double"/><br>
> name="SERVICE" type="xsd:double"/><br>
> name="MANUAL" type="xsd:double"/><br>
> name="P_MALE" type="xsd:double"/><br>
> name="P_FEMALE" type="xsd:double"/><br>
> name="SAMP_POP" type="xsd:double"/><br>
><br>
><br>
> This is what GDAL is guessing:<br>
> STATE_NAME: String (0.0)<br>
> STATE_FIPS: String (0.0)<br>
> SUB_REGION: String (0.0)<br>
> STATE_ABBR: String (0.0)<br>
> LAND_KM: Real (0.0)<br>
> WATER_KM: Real (0.0)<br>
> PERSONS: Real (0.0)<br>
> FAMILIES: Integer (0.0)<br>
> HOUSHOLD: Real (0.0)<br>
> MALE: Real (0.0)<br>
> FEMALE: Real (0.0)<br>
> WORKERS: Real (0.0)<br>
> DRVALONE: Integer (0.0)<br>
> CARPOOL: Integer (0.0)<br>
> PUBTRANS: Integer (0.0)<br>
> EMPLOYED: Real (0.0)<br>
> UNEMPLOY: Integer (0.0)<br>
> SERVICE: Integer (0.0)<br>
> MANUAL: Integer (0.0)<br>
> P_MALE: Real (0.0)<br>
> P_FEMALE: Real (0.0)<br>
> SAMP_POP: Integer (0.0)<br>
> bbox: RealList (0.0)<br>
><br>
> -Jukka Rahkonen-<br>
><br>
> _______________________________________________<br>
> gdal-dev mailing list<br>
> <a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
> <a target="_blank" href="http://lists.osgeo.org/mailman/listinfo/gdal-dev">http://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
<br>
</div>
</div>
<span class="HOEnZb"><font color="#888888">--<br>
Spatialys - Geospatial professional services<br>
<a target="_blank" href="http://www.spatialys.com/">http://www.spatialys.com</a><br>
</font></span>
<div class="HOEnZb">
<div class="h5">_______________________________________________<br>
gdal-dev mailing list<br>
<a href="mailto:gdal-dev@lists.osgeo.org">gdal-dev@lists.osgeo.org</a><br>
<a target="_blank" href="http://lists.osgeo.org/mailman/listinfo/gdal-dev">http://lists.osgeo.org/mailman/listinfo/gdal-dev</a><br>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</div>
</body>
</html>