# [postgis-users] PostGIS layer spatial tolerance

Fri Aug 28 20:16:17 PDT 2009

```Andreas,

> I come from the ESRI and Oracle world. Both ArcSDE and Oracle Spatial have user-defined spatial tolerance for each spatially enabled layer. This ensures that coordinates are exact, down to the last decimal (or integer for ArcSDE).

This is not, strictly speaking, correct with regards to the Oracle documentation to equate tolerances with coordinate precision. An Oracle tolerance describes a minimum distance between two vertices. This is different from an exact number of digits in an ordinate. So, a tolerance of 0.05 means 5cm between two vertices: if the distance between the ordinates is less than that the vertices are considered to be equal.  (Though, technically, the documentation tells users to set tolerances to be half the actual real world tolerance: so, 0.05 means 0.1m. For those who know how rounding is traditionally done in the C language, this is why tolerances are specified in this way.) What many think is the case with the tolerance, is that it describes the precision of the actual ordinate. So, a tolerance of 0.05 to an ordinate 123.45678 means the ordinate should, actually, be 123.5.

The simple reality is that you can store anything in the number that make up an ordinate of a geometry. Oracle has no automatic mechanism for applying the tolerance during transactions such that the ordinates are rounded to a stated precision. It is up to your client application or your own programming of triggers to ensure that ordinate precision remains exact.

Having said all that, in my programming of Oracle (see my free PL/SQL packages) I actually take the second view in how I handle the comparison of co-ordinates. I prefer to round precisely because I have viewed the data I have stored in the database of being accurate to a stated ordinate (numeric) precision. So, in my packages, I have programmed a function called Tolerance (with wrappers called ST_SnapToGrid) which will round the ordinates to the stated precision.

So, in the following, you will note that I can construct a geometry with any number of digits but you have to write a function yourself to round them to your data's actual precision:

SELECT GEOM.Tolerance(a.geom,0.005,0.05).Get_WKT() as geom
FROM (SELECT mdsys.SDO_Geometry('LINESTRING(1.12345 1.3445,2.43534 2.03998398)') as geom
FROM dual) a;

GEOM
---------------------------------------
LINESTRING (1.12 1.3, 2.44 2.0)

To do this is PostGIS use ST_SnapToGrid()

regards
Simon
--
SpatialDB Advice and Design, Solutions Architecture and Programming,
Oracle Database 10g Administrator Certified Associate; Oracle Database 10g SQL Certified Professional
Oracle Spatial, SQL Server, PostGIS, MySQL, ArcSDE, Manifold GIS, FME, Radius Topology and Studio Specialist.
39 Cliff View Drive, Allens Rivulet, 7150, Tasmania, Australia.