<html xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta name=Title content=""><meta name=Keywords content=""><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 15 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
pre
{mso-style-priority:99;
mso-style-link:"HTML Preformatted Char";
margin:0in;
margin-bottom:.0001pt;
font-size:10.0pt;
font-family:Courier;}
span.EmailStyle17
{mso-style-type:personal-reply;
font-family:Calibri;
color:windowtext;}
span.HTMLPreformattedChar
{mso-style-name:"HTML Preformatted Char";
mso-style-priority:99;
mso-style-link:"HTML Preformatted";
font-family:Courier;}
span.msoIns
{mso-style-type:export-only;
mso-style-name:"";
text-decoration:underline;
color:teal;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style></head><body bgcolor=white lang=EN-US link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>Hi Rémi,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>My query was:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>UPDATE hourly_positions hp SET landform = <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'> (SELECT ST_Value(rast,hp.geom4326) FROM landforms4326_128 lf where (lf.rast && hp.geom4326) AND ST_Intersects(lf.rast,hp.geom4326))<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>You wrote: “your update is not safe because a point may intersect several tiles.” I’d like to understand that better. Is this an edge-condition problem (i.e., a point falls right on an edge between two tiles)? If so, my query would fail because the subquery would return more than one value, correct? <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>If a point is right on the edge between two tiles, is there any situation where ST_Value(rast,point) might return different values from the two tiles? I saw your reply to a similar problem in this post,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><a href="http://lists.osgeo.org/pipermail/postgis-users/2014-February/038660.html">http://lists.osgeo.org/pipermail/postgis-users/2014-February/038660.html</a>,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>where you suggested adding something like this:<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>WITH (your computing)<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>SELECT DISTINCT ON (line_id, poly_id) , poly, line<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>FROM your computing<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>ORDER BY ST_Length(line) ASC<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>…but if ST_Value() ever returns more than one value, DISTINCT will return both(all), which might just add to your problems. I notice that the questioner responded “my fix was to detect such lines and intersect them with the original polygon instead of the tiled one.” I guess I’m just confused about what is really happening with the tiles.<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>Thanks,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>John<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'>P.S. I’m not sure why, but using the bbox did speed up the query a little bit. <o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:Calibri'><o:p> </o:p></span></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal><b><span style='font-family:Calibri;color:black'>From: </span></b><span style='font-family:Calibri;color:black'>postgis-users <postgis-users-bounces@lists.osgeo.org> on behalf of Rémi Cura <remi.cura@gmail.com><br><b>Reply-To: </b>PostGIS Users Discussion <postgis-users@lists.osgeo.org><br><b>Date: </b>Saturday, February 4, 2017 at 11:23 AM<br><b>To: </b>PostGIS Users Discussion <postgis-users@lists.osgeo.org><br><b>Subject: </b>Re: [postgis-users] ST_value slow<o:p></o:p></span></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><div><p class=MsoNormal><span style='font-family:Courier'>You're welcome !<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Courier'>Actually I guess that the function is intended to work on one tile and one point.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Courier'>So using a bbox would have no sense.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Courier'>Now if you use it on a whole table instead of one tile, <o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Courier'>you have to add intersect conditions and so.<o:p></o:p></span></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt'><span style='font-family:Courier'>By the way your update is not safe because a point may intersects several tiles.<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Courier'>Cheers<o:p></o:p></span></p></div><div><p class=MsoNormal><span style='font-family:Courier'>,Rémi-C<o:p></o:p></span></p></div></div><div><p class=MsoNormal><o:p> </o:p></p><div><p class=MsoNormal>2017-02-04 20:00 GMT+01:00 J Payne <<a href="mailto:jcpayne@uw.edu" target="_blank">jcpayne@uw.edu</a>>:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-right:0in'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'>Hi Rémi,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'>Thanks for the suggestion. I tiled the raster and added a ST_Intersects, and the whole operation finished in just a couple of minutes (the previous version hadn’t completed when I left it running overnight). The command I used was this:</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'>UPDATE hourly_positions hp SET landform = </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> (SELECT ST_Value(rast,hp.geom4326) FROM landforms4326_128 lf where (lf.rast && hp.geom4326) AND ST_Intersects(lf.rast,hp.geom4326))</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'>So, I don’t mean to sound critical of PostGIS, because I love the program and I have the utmost respect for everyone who works on it, but it appears to me that ST_Value is a “dumb” function; i.e., it doesn’t know how to find pixels in a raster with any reasonable speed. Is there any technical reason that its speed couldn’t be improved? As I mentioned in my original question, this operation (finding pixel values at many individual points) is commonly needed for certain classes of problems.</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'>Thanks again for saving my bacon!</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'>John</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt;font-family:Calibri'> </span><o:p></o:p></p><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0in 0in 0in'><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><b><span style='font-family:Calibri;color:black'>From: </span></b><span style='font-family:Calibri;color:black'>postgis-users <<a href="mailto:postgis-users-bounces@lists.osgeo.org" target="_blank">postgis-users-bounces@lists.osgeo.org</a>> on behalf of Rémi Cura <<a href="mailto:remi.cura@gmail.com" target="_blank">remi.cura@gmail.com</a>><br><b>Reply-To: </b>PostGIS Users Discussion <<a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a>><br><b>Date: </b>Saturday, February 4, 2017 at 3:01 AM<br><b>To: </b>PostGIS Users Discussion <<a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a>><br><b>Subject: </b>Re: [postgis-users] ST_value slow</span><o:p></o:p></p></div><div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>Hey,</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><span style='font-family:Courier'>I suppose your raster is correctly cut into small pieces.</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>You should first try to clock one point value retrieval.</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>It should be < 50ms (or you have an index issue)</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'> </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>Then you could force the use of index by writting a function <br>update_one_point(point) $$ update XXX set XXX WHERE ST_Intersects(rast,point);$$</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'> </span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><span style='font-family:Courier'>You would then call your function on all points:</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>SELECT update_one_point(hp.geom4326)</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;margin-bottom:12.0pt'><span style='font-family:Courier'>FROM hp</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>Cheers</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'>Rémi-C</span><o:p></o:p></p></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-family:Courier'> </span><o:p></o:p></p></div></div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>2017-02-04 6:44 GMT+01:00 J Payne <<a href="mailto:jcpayne@uw.edu" target="_blank">jcpayne@uw.edu</a>>:<o:p></o:p></p><blockquote style='border:none;border-left:solid #CCCCCC 1.0pt;padding:0in 0in 0in 6.0pt;margin-left:4.8pt;margin-top:5.0pt;margin-right:0in;margin-bottom:5.0pt'><div><div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'>Hello. I have an animal track that wanders around on a landscape, and I want to extract the pixel values for each location on the track. This kind of operation is very common for animal studies. The landscape is represented by a one-row raster map (“landforms4326”), and the GPS positions are in a table called “hourly_positions”. I am using a recent version of PostGIS and a very simple query to find the pixel values:</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'>UPDATE hourly_positions hp SET landform = (SELECT ST_Value(rast,hp.geom4326) FROM landforms4326)</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'>I have about 450,000 points, and both the landscape raster and the point geometries are indexed. For some reason, the operation is *<b>incredibly</b>* slow (it’s already been running for more than 8 hours on a nearly new Macbook Pro, and still isn’t finished). Am I missing something? I can’t for the life of me imagine why it would be so slow, since PostGIS is very fast with all sorts of other operations on the same data (intersections, etc.).</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'>Thanks,</span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'> </span><o:p></o:p></p><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><span style='font-size:11.0pt'>John</span><o:p></o:p></p></div></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'><br>_______________________________________________<br>postgis-users mailing list<br><a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a><br><a href="https://lists.osgeo.org/mailman/listinfo/postgis-users" target="_blank">https://lists.osgeo.org/mailman/listinfo/postgis-users</a><o:p></o:p></p></blockquote></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'> <o:p></o:p></p></div><p class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto'>_______________________________________________ postgis-users mailing list <a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a> <a href="https://lists.osgeo.org/mailman/listinfo/postgis-users" target="_blank">https://lists.osgeo.org/mailman/listinfo/postgis-users</a><o:p></o:p></p></div></div></div></div><p class=MsoNormal><br>_______________________________________________<br>postgis-users mailing list<br><a href="mailto:postgis-users@lists.osgeo.org">postgis-users@lists.osgeo.org</a><br><a href="https://lists.osgeo.org/mailman/listinfo/postgis-users" target="_blank">https://lists.osgeo.org/mailman/listinfo/postgis-users</a><o:p></o:p></p></blockquote></div><p class=MsoNormal><o:p> </o:p></p></div><p class=MsoNormal>_______________________________________________ postgis-users mailing list postgis-users@lists.osgeo.org https://lists.osgeo.org/mailman/listinfo/postgis-users<o:p></o:p></p></div></body></html>