[GRASS-dev] Overflow warning in r.mapcalc calculation

[Paulo van Breugel]

Hi Rainer,

Thanks a lot for your reply. The problem is indeed that the result of 
the calculation does not fit into the variable and your answer made me 
realize I had to take a closer look at what happens inside the formula. 
As it turns out, the problem lies in the following part of the formula:

round(C/0.0001)

The output of round() is an integer and c/0.0001 gives values that are 
larger then the maximum number for integer layers (which I presume is 
(2^31)-1). So I need to rethink the formula and the use of round.

Paulo



On 05/15/2013 03:51 PM, Rainer M. Krug wrote:
> Paulo van Breugel <p.vanbreugel at gmail.com> writes:
>
>> Hi,
>>
>> I am having trouble with the following computation, which gives me an overflow warning ("WARNING: Overflow occured in the
>> calculation").
> oerflow usually means that the result of a calculation does not fit into
> the variable type selected, i.e. is either smaller then the smallest
> value which can be stored or larger then the largest value.
>
> GRASS uses FCELL (float) and DCELL (double) varisbles, where double can
> store a larger range (sorry - I don't have the actual values at hand).
>
> You should be able to do the calculations by explicitely converting the
> result to double by using the double() function:
>
> UNTESTED:
>
>    r.mapcalc "A = if(B==0, double( (round(C/0.0001)-1175699902)/(3007966667-1175699902) *100.0 ), 1) " --overwrite
>
> should work, although untested.It is possible, thet the conversion to
> double needs to be made inside the calculation, depending where the
> actual overflow occurs.
>
>> r.mapcalc "A = if(B==0, (round(C/0.0001)-1175699902)/(3007966667-1175699902) *100.0, 1)" --overwrite
>>
>> whereby C is a map with values between 1 and 31000. It seems to be related to the size of the numbers (no warning if I
>> divide C by 0.001), but I am not clear what limit I am hitting here or how to avoid this.
>>
>> The warning does not stop the calculation, and the resulting map seems to be correct. However, I rather avoid this
>> warning, also because the warning message causes problems when running in batch from within R.
> Yes - overflow is a warning only, but *usually* results in wrong
> results! I fell in that trap once and it took me some time to figure out
> what was going on, as, if I remember correctly, no warning was issued
> (GRASS 6.4?).
>
> Cheers,
>
> Rainer
>
>> I am running GRASS 7.0 on Ubuntu 13.04 64 bit.
>>
>> Best
>>
>> Paulo
>>
>> _______________________________________________
>> grass-dev mailing list
>> grass-dev at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/grass-dev
>

-- 

Paulo van Breugel
Department of Geosciences and Natural Resource Management
Section of Forest, Nature and Biomass
University of Copenhagen

Rolighedsvej 23
1958 Frederiksberg C
Phone: +45 353-31646
Phone: +31 6 12189147
e-mail: pvb at life.ku.dk
e-mail: p.vanbreugel at gmail.com

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20130515/ee439529/attachment-0001.html>