[gdal-dev] Store also native geometry into Elastic Search?

[Adam Estrada]

Even,

This would have been around version 0.16.x. Not really sure which one
exactly but I don't think any changes to the geometry support happened
until 2012-2013?

VERY cool about all the modifications you guys have made! This is very exciting!

http://gdal.org/drv_elasticsearch.html

Adam

On Thu, Sep 3, 2015 at 8:52 AM, Even Rouault <even.rouault at spatialys.com> wrote:
> Le jeudi 03 septembre 2015 14:47:16, Adam Estrada a écrit :
>> Even,
>>
>> I think the reason to store everything by default was just to be on
>> the safe side and it satisfied some of the project requirements at the
>> time. That was back in 2011 so ElasticSearch and this driver are
>> probably not in sync any longer.
>
> OK, do you remember which ES version was targetted at that time ? So I can
> potentially be able to find in the doc the behaviour and see if it has changed.
>
>> There is a lot of work that can be
>> done to the driver to get up to date with all the cool stuff that is
>> now available in ES. I'd be interested in starting a roadmap
>> specifically for the ES driver starting with a read capability and the
>> ability to store other geometry types.
>
> Actually, you should have a look at the recent additions at
> http://gdal.org/drv_elasticsearch.html ;-)
>
>>
>> Adam
>>
>> On Thu, Sep 3, 2015 at 7:42 AM, Even Rouault <even.rouault at spatialys.com>
> wrote:
>> > Le jeudi 03 septembre 2015 12:49:30, Jukka Rahkonen a écrit :
>> >> Hi,
>> >>
>> >> What if somebody would like to put and deliver some official spatial
>> >> data (cadastral data) which are not in EPSG:4326 with Elastic Search?
>> >> Could it be possible to translate the geometries into EPSG:4326 and
>> >> store and index them as geo_shape, but also keep the native geometries
>> >> and save them into another field? Perhaps json does not like to have
>> >> many geometry fields but how about saving the native geometry as WKT?
>> >> Indexing WKT field feels stupid but maybe ES could be mapped to skip
>> >> it. Reasoning is that conversion from native to EPSG:4326 and back to
>> >> native is not necessary accurate and coordinates may change a bit.
>> >
>> > Jukka (and little question for you Adam if you read this),
>> >
>> > Well, you can do that with a Spatialite SQL query like :
>> >
>> > -sql "select *, ST_AsText(geometry) as WKT_32631 from simple_EPSG_32631"
>> > -dialect sqlite
>> >
>> > As far as not indexing the WKT_32631 field, you would need to:
>> > 1) export the default mapping with -lco WRITE_MAPPING=mapping.json
>> > 2) add "index":"no" to the WKT_32631 definition
>> > (https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping
>> > -core-types.html) 3) run again with -lco MAPPING=mapping.json
>> >
>> > Doing all the above steps in a fully automated way, perhaps with a
>> > dedicated PRESERVE_SRC_GEOMETRY layer creation option, would certainly
>> > be possible, but personnaly, it's more in the
>> > something-to-perhaps-consider-someday queue ;-)
>> >
>> > Perhaps we would need a few creation options to alter more easily the
>> > mapping of the created fields (rather than the above
>> > export/modify/import workflow). Not sure at this point which ones would
>> > make sense, and if we aren't careful, there's a risk we end up with
>> > something complicated that would be redundant with the JSon API. I also
>> > see that the existing write-only driver creates a mapping that
>> > explicitely stores fields, and as we don't disable storing the _source
>> > field, so apparently this wouldn't be necessary. Hum, might be wise to
>> > remove the explicit "store": "yes"
>> >
>> > @adam : do you remind the reason for adding "store": "yes" in the
>> > generated ES mapping ?
>> >
>> >> Am I even right that the conversion into EPSG:4326 is necessary for
>> >> using the geo_shape query?
>> >
>> > That's my understanding of
>> > https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping-
>> > geo-shape-type.html "In GeoJSON, and therefore Elasticsearch, the correct
>> > coordinate order is longitude, latitude (X, Y) within coordinate arrays"
>> >
>> > Actually I've just tried to manually insert with invalidate lon,lat and
>> > the behaviour is hard to decypher:
>> >
>> > e.g
>> >
>> > curl -d '{  "geometry": { "type": "linestring", "coordinates": [ [ 800.0,
>> > 0.0 ], [ 3.0, 50.0 ] ] } }'
>> > "http://localhost:9200/test/FeatureCollection?pretty"
>> >
>> > works,
>> >
>> > but
>> >
>> > curl -d '{  "geometry": { "type": "linestring", "coordinates": [ [
>> > 1000.0, 0.0 ], [ 3.0, 50.0 ] ] } }'
>> > "http://localhost:9200/test/FeatureCollection?pretty"
>> >
>> > throws an error:
>> > "MapperParsingException[failed to parse [geometry]]; nested:
>> > IllegalArgumentException[This method does not support GeometryCollection
>> > arguments]; "
>> >
>> > And I assume that any spherical geospatial search will behave in
>> > undefined ways if you insert coordinates that are not (lon,lat)
>> >
>> > Even
>> >
>> > --
>> > Spatialys - Geospatial professional services
>> > http://www.spatialys.com
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com