[gdal-dev] ogr2ogr problem with DECIMAL big int columns

[Even Rouault]

Le jeudi 13 octobre 2011 10:49:53, Luca Sigfrido Percich a écrit :
> Hi all,
> 
> I'm trying to convert with ogr2ogr a shapefile having a decimal(15,0) ID
> column containig very long integer unique IDs (for example,
> 113800200319004)
> For whom it may concern, it is a TeleAtlas MultiNet file.
> 
> When converting to shapefile, I get the same column type in out.shp -
> decimal(15, 0) - but all the values are truncated to the lowest long
> integer, -2147483648, so it looks like a conversion to long integer is
> taking place somewhere:
> 
> ogr2ogr -s_srs EPSG:4326 -t_srs EPSG:3003 -f "ESRI Shapefile"
> test/out.shp in.shp

Yes the lack of support for big integers is a well known problem. A RFC on 
that subject has been prepared some time a go ( 
http://trac.osgeo.org/gdal/wiki/rfc31_ogr_64 ) but has not been adopted nor 
implemented yet.

> 
> When converting to MapInfo TAB, the column is converted to Float (8
> bytes), which could lead to loss of information when the number is
> really big and thus gets approximated when stored in floating point
> format:
> 
> ogr2ogr -s_srs EPSG:4326 -t_srs EPSG:3003 -f "MapInfo file" test/out.tab
> in.shp
> 
> Please note that MapInfo supports the Decimal column type, so a straight
> type mapping should be implemented.

Remember that ogr2ogr never does direct translation from one format to another 
one, but uses the OGR data model as a pivot representation. So even when 2 
formats support a concept, but OGR does not, you cannot expect miracles from 
ogr2ogr.

> 
> I tried the -fieldTypeToString option, which in this case could do the
> 
> trick, but the "decimal" type is not supported:
> > Unhandled type for fieldtypeasstring option : decimal

Expected as documented in http://gdal.org/ogr2ogr.html :

"""
-fieldTypeToString type1, ...:
(starting with GDAL 1.7.0) converts any field of the specified type to a field of 
type string in the destination layer. Valid types are : Integer, Real, String, 
Date, Time, DateTime, Binary, IntegerList, RealList, StringList. Special value 
All can be used to convert all fields to strings. This is an alternate way to 
using the CAST operator of OGR SQL, that may avoid typing a long SQL query. 
"""

> 
> It works fine with the "real" type, though:
> 
> ogr2ogr -fieldTypeToString real -s_srs EPSG:4326 -t_srs EPSG:3003 -f
> "MapInfo file" test/out.tab in.shp
> 
> but all the decimal and real columns are converted into character, which
> is a negative but not critical drawback.
> 
> Should I open a ticket? 

This is probably the issue that has the biggest number of tickets, so no you 
don't need to file a new one ;-)

> Or could you suggest a workaround other than
> converting all floating point columns to char?

No, that's one of the main reason that justified adding -fieldTypeToString

> 
> Thank you in advance
> 
> Sig
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/gdal-dev/attachments/20111013/386e2546/attachment-0001.html