[postgis-users] Aggregating rasters by adding and other confusions

Fri Mar 9 20:22:56 PST 2018

David,

I took a cursory glance at the union code we have in place.

What it seems to do is two passes

1 pass does an ST_Union using 'COUNT'  (so in your case you'd get numbers between 0 and 3, 0 being no rasters considered having any data)

2nd pass does an ST_Union using 'SUM'  

And returns a new raster  where each pixel is  SUM/COUNT

Basic code is here:

http://postgis.net/docs/doxygen/2.4/da/dde/rtpg__mapalgebra_8c_a1d94065e6cef5d5d61417b82b2cf4fb6.html#a1d94065e6cef5d5d61417b82b2cf4fb6

(note it only computes a value if the first band (which I presume to be the count band)  value > 0  and has no nodata value. )

I don't know why a count band would have a nodata value aside from when it's 0

What does COUNT return for you?  

Thanks,

Regina

From: postgis-users [mailto:postgis-users-bounces at lists.osgeo.org] On Behalf Of David M. Kaplan
Sent: Tuesday, March 06, 2018 10:29 AM
To: postgis-users at lists.osgeo.org
Subject: Re: [postgis-users] Aggregating rasters by adding and other confusions

Hi,

Yes, this does seem to be the solution. For completeness, the following did the same thing as my aggregate function:

SELECT ST_Union(rast,1,'SUM') FROM blah;

This includes replacing 0's with NULL's - ST_Union had the same behavior at my aggregate function. 

Can you explain this? For my sum, this isn't a big deal, but ST_Union(... 'MEAN') seems to be treating 0's at NULL values, thereby removing them from the mean calculation:

db=# SELECT (ST_SummaryStats(ST_Union(rast,1,'SUM'))).sum FROM blah; sum  
------
 2792
(1 row)

db=# SELECT (ST_SummaryStats(ST_Union(rast,1,'MEAN'))).sum FROM blah;
 sum  
------
 1346
(1 row)

The table "blah" has 3 rasters with no null values, so I would expect the sum of the mean to be 1/3 the sum of the sum, but clearly this is not the case.

I tried playing around with the no data value for my rasters to see if this would change things. It does change things, but not in any way I can explain:

fads=# SELECT (ST_SummaryStats(ST_Union(ST_SetBandNoDataValue(rast,1,0),1,'MEAN'))).sum FROM blah;
 sum  
------
 2025
(1 row)

fads=# SELECT (ST_SummaryStats(ST_Union(ST_SetBandNoDataValue(rast,1,-999),1,'MEAN'))).sum FROM blah;
 sum  
------
 2025
(1 row)

Setting the no data value has no effect on the sum of the sum.

Does any of this make sense?

Thanks,

David

On Mon, 2018-03-05 at 12:00 -0800, postgis-users-request at lists.osgeo.org <mailto:postgis-users-request at lists.osgeo.org>  wrote:

Date: Mon, 5 Mar 2018 12:05:40 -0500
From: "Regina Obe" <lr at pcorp.us <mailto:lr at pcorp.us> >
To: "'PostGIS Users Discussion'" <postgis-users at lists.osgeo.org <mailto:postgis-users at lists.osgeo.org> >
Subject: Re: [postgis-users] Aggregating rasters by adding and other
        confusions
Message-ID: <001f01d3b4a4$30d1fa20$9275ee60$@pcorp.us <mailto:001f01d3b4a4$30d1fa20$9275ee60$@pcorp.us> >
Content-Type: text/plain; charset="utf-8"

I think what you are looking for is ST_Union (.. SUM)  note this has union types – FIRST, MIN, MAX, COUNT, SUM, MEAN, RANGE

http://postgis.net/docs/manual-2.4/RT_ST_Union.html

From: postgis-users [mailto:postgis-users-bounces at lists.osgeo.org] On Behalf Of David M. Kaplan
Sent: Monday, March 05, 2018 10:03 AM
To: postgis-users at lists.osgeo.org <mailto:postgis-users at lists.osgeo.org> 
Subject: [postgis-users] Aggregating rasters by adding and other confusions

Hi,

I have recently started working with the postgis raster functionality. In general, I have found this really useful and have been able to do some neat things fairly simply with this raster functionality. Nevertheless, there are a few basic things that I am confused about and I was hoping someone could give me a hand.

(1) First of all, I have a table with a bunch of rasters that have the same extent, alignment, scale, etc. and I want to aggregate them together into a single raster using pixel-by-pixel addition. It seems like there should be a function to do this, but I can't find one. Is there an aggregate "ST_MapAlgebra" function? 

Given that I couldn't find one, I defined an aggregate function as follows:

CREATE OR REPLACE FUNCTION AddRasters(r1 raster, r2 raster)
       RETURNS raster AS
$BODY$
SELECT ST_MapAlgebra($1,$2,'[rast1]+[rast2]');
$BODY$ LANGUAGE 'sql' IMMUTABLE STRICT;

CREATE AGGREGATE sum (raster)
(
    sfunc = AddRasters,
    stype = raster
);

(2) This seems to work, but it has the unexpected behavior that it replaces 0 values with NULL. In my case, this is fine, but I am wondering why it does this? I can't find anything that indicates that it should be replacing zeros with NULL. Here is the metadata associated with one of my rasters (the others are similar):

# SELECT ST_BandMetadata(rast), ST_Metadata(rast), ST_SummaryStats(rast) FROM blah;
-[ RECORD 1 ]---+-------------------------------------------------------
st_bandmetadata | (16BUI,,f,)
st_metadata     | (-180,90,360,180,1,-1,0,0,4326,1)
st_summarystats | (64800,417,0.00643518518518519,0.223617719977485,0,46)

I have not defined a nodataval for these layers and the original layers have no NULL values.

(3) Is there a postgis command to turn all the NULL values back into zeros?

(4) I was also considering just defining the '+' operator for raster + raster to be pixel-by-pixel addition. Is there any reason that I wouldn't want to do this?

(5) Finally, I have been visualizing results with QGIS using the DB Manager. However, I don't see how to select a row from a raster table and incorporate just that row into the canvas. Is there a way to do this?

Thanks for the assistance,

David

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20180309/490a0bea/attachment.html>