[gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

Mon Sep 2 04:30:21 PDT 2013

Hi Peifer.

Thank you for your comments. They do explain more in depth what is happening. But as seen in your first example, the value 333.123456 is reduced to 300 while the value 0.00012345 is reduced to 0.0001. I think this is a big change in the relative values of each number.

I am curious to know if the "new" implementation in GDAL 1.10 is actually what is wanted/expected? 

I use the DECIMAL_PRECISION for rounding purposes to control how many digits I want to the right of the comma, not how many digits I want to use for the entire number.
I know I get a lot of zeros if I use DECIMAL_PRECISION=10 in GDAL 1.9.2 but I retain my precision.

In GDAL 1.10 I have to determine the number of digits to the left of the comma used by the largest number in my data set, add the number of digits I want to have to the right of the comma and use that value as DECIMAL_PRECISION. The result must be that I get a lot of (waste) digits to the right of the comma for small numbers and the expected result for large numbers. But isn't this similar to what the patch (mentioned ealier) wanted to avoid? :

> The AAIGrid float format string is "%6.20g"
> Firstly, the 6 doesn't really add anything, and when writing small integers it causes them to be space-padded, which is a waste of bytes.
> At present AAIGrid supports "-co DECIMAL_PRECISION=10" which changes the formatstring to "%.10f".
> This is a little inconsistent, and the %f causes integers to be written as "9.000000000" which is a horrible waste of bytes.

I could write my own GeoTIFF to AAIGrid exporter, but I don't believe that should be necessary :)

Kind regards, Casper

-----Original Message-----
From: gdal-dev-bounces at lists.osgeo.org [mailto:gdal-dev-bounces at lists.osgeo.org] On Behalf Of peifer
Sent: 2. september 2013 12:21
To: gdal-dev at lists.osgeo.org
Subject: Re: [gdal-dev] Change of DECIMAL_PRECISION in AAIGrid

Casper Børgesen wrote
> This is my syntax:
> 
> gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=2 
> my_source.vrt my_target.asc
> 
> I have tried the above syntax using gdal 1.9.2 and 1.10 and these are 
> examples from my results:
> 
> 1.9.2:
> ...
> 3.77 3.83 3.89 3.87 3.79 3.49 3.03 2.50 1.96 1.56 1.29 0.95 0.77 0.54 
> 0.53
> 0.41
> ...
> 
> 1.10:
> ...
> 3.8 3.8 3.9 3.9 3.8 3.5 3 2.5 2 1.6 1.3 0.95 0.77 0.54 0.53 0.41 ...
> 
> Has the meaning of DECIMAL_PRECISION changed from 1.9.2 to 1.10?
> 
> It seems like in 1.9.2 the meaning is X characters to the right of the 
> comma and in 1.10 it means X describing characters?
> 
> I haven't tried the 1.10.1 release because I can't find the binaries 
> on Tamas gisinternals.

Hmm. After a look into the source code [0] and some testing [1], it looks to me that DECIMAL_PRECISION is used as precision of a "%.<precision>g" format. 

Format letter g means: "Print a number in either scientific notation or in floating-point notation, whichever uses fewer characters."

For format letter g, the precision value defines: "Maximum number of
*significant* digits."

I see there has been this change recently:
http://trac.osgeo.org/gdal/changeset/25032/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp 

Hermann

[0]
http://trac.osgeo.org/gdal/browser/trunk/gdal/frmts/aaigrid/aaigriddataset.cpp#L1051

[1]

# Input file

$ cat in.asc
ncols 1
nrows 4
xllcorner 1
yllcorner 1
cellsize  1
0.00012345
1.12345678
22.1234567
333.123456

# Test run

$ for i in 1 2 3 4 5 ; do
  echo DECIMAL_PRECISION=$i &&
  gdal_translate -of AAIGrid -ot Float32 -co DECIMAL_PRECISION=$i in.asc out.asc &&
  cat out.asc
done

# Output

DECIMAL_PRECISION=1

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.0001
 1
 2e+01
 3e+02

DECIMAL_PRECISION=2

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.00012
 1.1
 22
 3.3e+02

DECIMAL_PRECISION=3

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.000123
 1.12
 22.1
 333

DECIMAL_PRECISION=4

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.0001234
 1.123
 22.12
 333.1

DECIMAL_PRECISION=5

ncols        1
nrows        4
xllcorner    1.000000000000
yllcorner    1.000000000000
cellsize     1.000000000000
 0.00012345
 1.1235
 22.123
 333.12

--
View this message in context: http://osgeo-org.1560.x6.nabble.com/gdal-dev-Change-of-DECIMAL-PRECISION-in-AAIGrid-tp5075524p5075585.html
Sent from the GDAL - Dev mailing list archive at Nabble.com.
_______________________________________________
gdal-dev mailing list
gdal-dev at lists.osgeo.org
http://lists.osgeo.org/mailman/listinfo/gdal-dev