[gdal-dev] Precision Mystery (FGDB)

[Ragi Burhum]

Paul,

The one line that I think is making the difference is

<HighPrecision>true</HighPrecision>

because, if memory serves me right, it kicks in logic for using
64bit-coordinates internally (actually I remember something about the sign
and the mantissa being 53bits - can't recall exactly). ESRI migrated to what
they call high precision coordinate system in ArcGIS
9.2<http://webhelp.esri.com/arcgisdesktop/9.2/index.cfm?TopicName=Migrating_to_high_precision>.
So all the software really expects the coordinate system to be declared that
way by now. There is a good white paper on the ESRI site that describes how
ArcGIS manages its coordinate systems
internally<http://downloads.esri.com/support/whitepapers/ao_/UnderstandingCoordinateManagement_June2007.pdf>.
I hope that helps.

- Ragi



Date: Thu, 30 Jun 2011 13:26:21 -0700
> From: Paul Ramsey <pramsey at cleverelephant.ca>
> Subject: Re: [gdal-dev] Precision Mystery (FGDB)
> To: gdal-dev at lists.osgeo.org
> Message-ID: <BANLkTikNQyhsDkuNFNdEkN3S6GiHL5ihDQ at mail.gmail.com>
> Content-Type: text/plain; charset=ISO-8859-1
>
> OKAY, found it, and it's an odd one. It has nothing to do with the
> shape buffers or the transit into PostGIS... I just wrote a small
> program that uses the FGDB API almost exclusively, it only uses OGR to
> write the shape buffer.
>
> And the difference is in the declaration of the SRS... when I declare
> a UTM srs with the following extra XML
>
>                                    <XOrigin>-5120900</XOrigin>
>                                    <YOrigin>-9998100</YOrigin>
>                                    <XYScale>10000</XYScale>
>                                    <ZOrigin>-100000</ZOrigin>
>                                    <ZScale>10000</ZScale>
>                                    <MOrigin>-100000</MOrigin>
>                                    <MScale>10000</MScale>
>                                    <XYTolerance>0.001</XYTolerance>
>                                    <ZTolerance>0.001</ZTolerance>
>                                    <MTolerance>0.001</MTolerance>
>                                    <HighPrecision>true</HighPrecision>
>
> Then writing the POINT(1 2 3) and reading back with ogrinfo I get
>
> FID Column = OBJECTID
> Geometry Column = SHAPE
> OGRFeature(test):1
>  POINT (1 2 3)
>
> But if I omit those lines, I get:
>
> FID Column = OBJECTID
> Geometry Column = SHAPE
> OGRFeature(test):1
>  POINT (1.000001729731139 1.999981429457875 0.0)
>
> Weird precision changes and my Z is gone! So those things matter...
> but what do they do...
>
> P.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20110701/84a132ce/attachment.html