[Gdal-dev] OGR returns wrong floating values for shapefiles (and integer as real, in error)

[Roger Bivand]

On Tue, 24 Oct 2006, Frank Warmerdam wrote:

> Maciej Sieczka wrote:
> > Frank Warmerdam wrote:
> >> Maciej Sieczka wrote:
> >>> Hmm, this looks wrong (?) - my CAT and LCAT *do* have a zero precision.
> >>> So I'd expect them to be treated as integer. And the other 4 columns
> >>> have 15 decimal places precision, not 2. Open the dbf in a spreadsheet
> >>> to see that:
> >>>
> >>> CAT,N,11,0
> >>> LCAT,N,11,0
> >>> Z,N,24,15
> >>> Z_BREACH,N,24,15
> >>> Z_BREACH1,N,24,15
> >>> LENGTH,N,24,15
> > 
> >> Because Shapelib does not have a 64bit integer type, it automatically
> >> treats integer fields that are potentially to large to fit in a 32bit
> >> integer as floating point.  This isn't a great approach and causes a
> >> number of problems,
> > 
> > One of which is that after v.out.ogr->v.in.ogr one gets his previous
> > integer fields imported as double floating point.
> > 
> >> but it isn't completely without reason.
> > 
> > But is that planned to be fixed?
> 
> Maciek,
> 
> It is a known problem that I hope to address eventually but there is no
> timeline for a fix.

That seems fair, given that only widths < 10 match 32-bit integers. The
problem is that we need to read wide fields as double anyway (and they've
only got 16 digits), and then check to see if there is no difference
between the value and the rounded value, having taken fuzz into account, 
and keeping missing values handled properly. Not easy. So one route is to 
make sure that integer fields get written with the tightest width 
possible, but I'm not sure where this happens in v.out.ogr, 
OGR_L_CreateField, maybe?

Roger

> 
> Best regards,
> 

-- 
Roger Bivand
Economic Geography Section, Department of Economics, Norwegian School of
Economics and Business Administration, Helleveien 30, N-5045 Bergen,
Norway. voice: +47 55 95 93 55; fax +47 55 95 95 43
e-mail: Roger.Bivand at nhh.no