[gdal-dev] gdal_calc.py gives different answers

[Ari Jolma]

I have no experience with gdal_calc.pv nor do I know about its internals 
anything beside it being implemented with numpy. So I comment this only 
from a general point of view.

05.01.2016, 09:00, Avyav Kumar kirjoitti:
> This is a rather interesting error, which is listed here 
> (https://trac.osgeo.org/gdal/ticket/5478). While getting the 
> intermediate calculations of the raster band for A and B, I noticed 
> the following -
>
> For --calc="A*20+B", the raster formed as the result of the addition 
> of A*20 and B is not equal to A*20 + B.
>
> For --calc="A*20.0+B", the raster formed as the result of the addition 
> of A*20.0 and B is, in fact, equal to A*20 + B.
>
> Logically, these should have the same value as there is no difference 
> between 20.0 and 20.

I would say that there is since 20.0 is a real number and 20 is an 
integer. Now, depending on whether A and B are real or integer valued, 
A*20 can be either real or integer valued and A*20.0 is real valued, and 
A*20+B and A*20.0+B can be either real or integer valued. Treating 
values as real or integer are quite different in programming languages 
and also when storing them with raster formats. Also, as you say below, 
integer values can easily flow over.

Best regards,

Ari

> However, as it is pointed out in the thread, running gdalinfo -stats 
> on them yields -
>
> For --calc="A*20+B", the raster formed is a valid raster.
>
> For --calc="A*20.0+B", the raster formed is invalid!
>
> Hence, even though the intermediate calculation on the second --calc 
> value is correct, there is no valid output raster formed. Is it 
> because the value A*20 + B would have overshot the boundary values for 
> 8-bit channel? Any thoughts?
>
> Regards,
>
> Avyav
>
>
> _______________________________________________
> gdal-dev mailing list
> gdal-dev at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/gdal-dev

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20160105/18883407/attachment.html>