[gdal-dev] Store also native geometry into Elastic Search?

[Adam Estrada]

Even,

I think the reason to store everything by default was just to be on
the safe side and it satisfied some of the project requirements at the
time. That was back in 2011 so ElasticSearch and this driver are
probably not in sync any longer. There is a lot of work that can be
done to the driver to get up to date with all the cool stuff that is
now available in ES. I'd be interested in starting a roadmap
specifically for the ES driver starting with a read capability and the
ability to store other geometry types.

Adam


On Thu, Sep 3, 2015 at 7:42 AM, Even Rouault <even.rouault at spatialys.com> wrote:
> Le jeudi 03 septembre 2015 12:49:30, Jukka Rahkonen a écrit :
>> Hi,
>>
>> What if somebody would like to put and deliver some official spatial data
>> (cadastral data) which are not in EPSG:4326 with Elastic Search? Could it
>> be possible to translate the geometries into EPSG:4326 and store and index
>> them as geo_shape, but also keep the native geometries and save them into
>> another field? Perhaps json does not like to have many geometry fields but
>> how about saving the native geometry as WKT? Indexing WKT field feels
>> stupid but maybe ES could be mapped to skip it. Reasoning is that
>> conversion from native to EPSG:4326 and back to native is not necessary
>> accurate and coordinates may change a bit.
>
> Jukka (and little question for you Adam if you read this),
>
> Well, you can do that with a Spatialite SQL query like :
>
> -sql "select *, ST_AsText(geometry) as WKT_32631 from simple_EPSG_32631" -dialect sqlite
>
> As far as not indexing the WKT_32631 field, you would need to:
> 1) export the default mapping with -lco WRITE_MAPPING=mapping.json
> 2) add "index":"no" to the WKT_32631 definition (https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-core-types.html)
> 3) run again with -lco MAPPING=mapping.json
>
> Doing all the above steps in a fully automated way, perhaps with a dedicated PRESERVE_SRC_GEOMETRY
> layer creation option, would certainly be possible, but personnaly, it's more in the
> something-to-perhaps-consider-someday queue ;-)
>
> Perhaps we would need a few creation options to alter more easily the mapping
> of the created fields (rather than the above export/modify/import workflow). Not sure
> at this point which ones would make sense, and if we aren't careful, there's a risk we end
> up with something complicated that would be redundant with the JSon API.
> I also see that the existing write-only driver creates a mapping that explicitely stores fields,
> and as we don't disable storing the _source field, so apparently this wouldn't be necessary.
> Hum, might be wise to remove the explicit "store": "yes"
>
> @adam : do you remind the reason for adding "store": "yes" in the generated ES mapping ?
>
>>
>> Am I even right that the conversion into EPSG:4326 is necessary for using
>> the geo_shape query?
>
> That's my understanding of https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-geo-shape-type.html
> "In GeoJSON, and therefore Elasticsearch, the correct coordinate order is longitude, latitude (X, Y) within coordinate arrays"
>
> Actually I've just tried to manually insert with invalidate lon,lat and the behaviour is
> hard to decypher:
>
> e.g
>
> curl -d '{  "geometry": { "type": "linestring", "coordinates": [ [ 800.0, 0.0 ], [ 3.0, 50.0 ] ] } }' "http://localhost:9200/test/FeatureCollection?pretty"
>
> works,
>
> but
>
> curl -d '{  "geometry": { "type": "linestring", "coordinates": [ [ 1000.0, 0.0 ], [ 3.0, 50.0 ] ] } }' "http://localhost:9200/test/FeatureCollection?pretty"
>
> throws an error:
> "MapperParsingException[failed to parse [geometry]]; nested: IllegalArgumentException[This method does not support GeometryCollection arguments]; "
>
> And I assume that any spherical geospatial search will behave in undefined ways if you insert
> coordinates that are not (lon,lat)
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com