[gdal-dev] SRS of the Elastic Search geometries?

[Rahkonen Jukka (MML)]

Even Rouault wrote:

>>
>> Probably that's it, the explicit SRS. I started with data that have native
>> SRS epsg:3067 and OpenJUMP JML format which has no means for holding SRS.
>> Conversion into ES gives always an empty layer, just the schema gets
>> inserted but no geometries, even is I use -s_srs and -t_srs.
>>
>> Next trial was to convert JML into GML without assigning SRS. GML has an
>> undefined SRS and conversion without -s_srs and -t_srs leads to a situation
>> where metadata sayes that ES is using epsg:4326 but the features are in
>> epsg:3067. With proper -s_srs and -t_srs the result is good.

> You can also use -a_srs for that.

I made a test and it appears to work but isn't is confusing because it makes a different effect than with most other drivers? Manual page http://www.gdal.org/ogr2ogr.html tells "-a_srs srs_def:     Assign an output SRS " . With Elastic Search if you use -a_srs epsg:3067 the output SRS will actually be epsg:4326. Let's keep this secret and teach the users always use explicit -s_srs  and t_srs.

>>
>> If I convert JML into GML and assign SRS with -a_srs epsg:3067 the result
>> is the same as above. Each GML feature has SRS defined as <gml:Polygon
>> srsName="EPSG:3067"> but perhaps it is not explicit because conversion
>> into ES gives correct result only when -s_srs and -t_srs are defined.
>
>> So, two problems:
>> - No way at all to use Jump JML as input format

> Are you sure about that ? I've just tried the following and it works:
I am sure that I made a wrong conclusion. The problem is related to JML but it is somehow special. I took the test data from a real world dataset but I attach a minimal sample JML file for reproducing the issue.

> $ ogr2ogr simple_EPSG_32631.jml simple_EPSG_32631.shp -f jml
| $ ogr2ogr  -update es: simple_EPSG_32631.jml -overwrite -s_srs EPSG:32631 -
t_srs EPSG:32631
(or with -a_srs EPSG:32631)

> The behaviour with JML and GML inputs should be the same.
My test data in JML format contains this DATETIME 
 <property name="JATTOPVM">2015-06-14T22:29:15.454+0300</property>

It leads to this ElasticSearch error which I took from the ES console:
Caused by: org.elasticsearch.index.mapper.MapperParsingException: failed to pars
e date field [2015/06/14 22:29:15.454+03], tried both date format [yyyy/MM/dd HH
:mm:ss.SSS||yyyy/MM/dd], and timestamp number with locale []

When I convert JML into GML with ogr2ogr the DATETIME comes into GML as
 <ogr:JATTOPVM>2015/06/14 22:29:15.454+03</ogr:JATTOPVM>
This is OK for Elastic Search. I do not see any difference but there must be some. JML failed for me because of this parsing error. ogr2ogr does not catch the error, not even when run with --debug on.


>> - With GML parameters -s_srs and -t_srs must be defined even if each GML
>> feature has srsName and all features in the GML file have same srsName.

> The issue with GML is complex. Basically, the GML driver is very cautious and
> doesn't report a layer SRS since there's no way to know if all features have
> the same SRS without exploring the whole file, which he will avoid if there's a
> .xsd that it can understand. If you remove the .xsd, then a full initial scan
> will be done and if the driver realizes that all features have the same SRS,
> this will be written in the .gfs file, and thus reported as the SRS layer.
> With some particular formulations of GML (like a WFS GetFeature response where
> we know that the SRS is homogenous), the driver will use the SRS of the top
> Envelope as the layer SRS.

>
>> A warning like "No explicit SRS found, use -s_srs and -t_srs" might be good
>> to have.

> Should be addressed by my latest commit:
> "ES: add warnings at layer creation if no SRS is specified, and at feature
> insertion, in the case where no layer SRS is defined when the bounding box
> isn't valid for long/lat"

>
>> I know very little about Elastic Seaach, just started experiments a week
>> ago. I have concluded that what is layer for GDAL is "index", and what is
>> feature, is "document", and that documents are somehow saved as-is

> That really depends. You may choose to store or not the _all fields or
> individual fields.

>> and
>> clever indexes are built on top of them. Therefore I was thinking that
>> perhaps everything is OK on the ES side even if the documents come out from
>> it with epsg:3067 geometries.

> The spatial indices made assumption on the coordinates. For example a geohash
> really assumes the [-90,90]x[-180,180] range. Not sure how geo_shape indexing
> works.

-Jukka-
-------------- next part --------------
A non-text attachment was scrubbed...
Name: date_error.jml
Type: application/octet-stream
Size: 2489 bytes
Desc: date_error.jml
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20150901/4f293b9e/attachment.obj>