[fdo-trac] #635: FdoDataValue: error converting max + 1
FDO
trac_fdo at osgeo.org
Wed Mar 31 14:55:13 EDT 2010
#635: FdoDataValue: error converting max + 1
-----------------------------+----------------------------------------------
Reporter: brentrobinson | Owner: brentrobinson
Type: defect | Status: new
Priority: major | Milestone: 3.6.0
Component: FDO API | Version: 3.5.0
Severity: 2 | Keywords: DataValue convert precision
External_id: |
-----------------------------+----------------------------------------------
The following:
{{{
FdoPtr<FdoDoubleValue> dblVal =
FdoDoubleValue::Create(pow((double)2,(double)63));
FdoPtr<FdoInt64Value> i64Val = FdoInt64Value::Create(dbiVal, false,
false, true);
}}}
will fail with "Cannot convert value 9.22337203685478e+018 to int64
without loss of precision", if run on a Windows 64bit machine.
This should have succeeded since dblVal > FdoInt64Max and conversion was
attempted with truncate=true.
--
Ticket URL: <http://trac.osgeo.org/fdo/ticket/635>
FDO <http://fdo.osgeo.org/>
Feature Data Objects
More information about the fdo-trac
mailing list