[postgis-users] ST_Mean4ma , ST_StdDev4ma rounding down

[Severin Thaler]

Hi all,

here’s the problem.
after creating the dummy raster : http://postgis.net/docs/manual-2.1/RT_reference.html <http://postgis.net/docs/manual-2.1/RT_reference.html>
i ran:

select (st_dumpvalues(rast)).* from dummy_rast;

and get:

NOTICE:  Raster provided has no bands
NOTICE:  Raster provided has no bands
 nband |                                                    valarray                                                     
-------+-----------------------------------------------------------------------------------------------------------------
     1 | {{253,254,253,254,254},{253,254,254,253,249},{250,254,254,252,249},{251,253,254,254,253},{252,250,254,254,254}}
     2 | {{78,98,122,173,209},{96,118,180,249,254},{99,112,169,245,254},{89,99,122,176,229},{79,88,97,112,135}}
     3 | {{70,86,100,135,161},{80,108,162,227,250},{90,108,175,251,254},{77,86,109,164,203},{62,69,76,86,101}}
(3 rows)

which by itself is kinda weird and that is why i unionized first, i.e.

select (st_dumpvalues(st_union(rast))).* from dummy_rast;

and now get a more proper output:

 nband |                                                    valarray                                                     
-------+-----------------------------------------------------------------------------------------------------------------
     1 | {{253,254,253,254,254},{253,254,254,253,249},{250,254,254,252,249},{251,253,254,254,253},{252,250,254,254,254}}
     2 | {{78,98,122,173,209},{96,118,180,249,254},{99,112,169,245,254},{89,99,122,176,229},{79,88,97,112,135}}
     3 | {{70,86,100,135,161},{80,108,162,227,250},{90,108,175,251,254},{77,86,109,164,203},{62,69,76,86,101}}
(3 rows)

however, my real concern is that when i compute the average over the 3 bands:

select (st_dumpvalues(st_mapalgebra(array[row(rast, 1), row(rast, 2), row(rast, 3)]::rastbandarg[], 'st_mean4ma(double precision[], int[], text[])'::regprocedure))).* from (select st_union(rast) as rast from dummy_rast) as foo;

i get

 nband |                                                    valarray                                                     
-------+-----------------------------------------------------------------------------------------------------------------
     1 | {{133,146,158,187,208},{143,160,198,243,251},{146,158,199,249,252},{139,146,161,198,228},{131,135,142,150,163}}
(1 row)

i.e. the actual means but rounded down, similarly for the standard deviation callback function.
is this a bug because it shouldn’t be rounded down, we should have the decimals too.

thanx for feedback
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20160220/6da8c7f3/attachment.html>