[Gdal-dev] OGR returns wrong floating values for shapefiles (and
integer as real, in error)
Roger.Bivand at nhh.no
Tue Oct 24 11:51:30 EDT 2006
On Tue, 24 Oct 2006, Frank Warmerdam wrote:
> Maciej Sieczka wrote:
> > Frank Warmerdam wrote:
> >> Maciej Sieczka wrote:
> >>> Hmm, this looks wrong (?) - my CAT and LCAT *do* have a zero precision.
> >>> So I'd expect them to be treated as integer. And the other 4 columns
> >>> have 15 decimal places precision, not 2. Open the dbf in a spreadsheet
> >>> to see that:
> >>> CAT,N,11,0
> >>> LCAT,N,11,0
> >>> Z,N,24,15
> >>> Z_BREACH,N,24,15
> >>> Z_BREACH1,N,24,15
> >>> LENGTH,N,24,15
> >> Because Shapelib does not have a 64bit integer type, it automatically
> >> treats integer fields that are potentially to large to fit in a 32bit
> >> integer as floating point. This isn't a great approach and causes a
> >> number of problems,
> > One of which is that after v.out.ogr->v.in.ogr one gets his previous
> > integer fields imported as double floating point.
> >> but it isn't completely without reason.
> > But is that planned to be fixed?
> It is a known problem that I hope to address eventually but there is no
> timeline for a fix.
That seems fair, given that only widths < 10 match 32-bit integers. The
problem is that we need to read wide fields as double anyway (and they've
only got 16 digits), and then check to see if there is no difference
between the value and the rounded value, having taken fuzz into account,
and keeping missing values handled properly. Not easy. So one route is to
make sure that integer fields get written with the tightest width
possible, but I'm not sure where this happens in v.out.ogr,
> Best regards,
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no
More information about the Gdal-dev