<div dir="ltr">Thanks for the graphs Duncan! My guess is that at the low tile sizes, the individual round-trip time would be short but the volume of disk i/o is what's coming into play. The opposite is true for large tile sizes where the disk i/o volume is decreased but it takes longer for each round-trip.<div>
<br></div><div>Having said all that, I'm not willing to suggest that there is one range appropriate for every user or deployment. Assuming you ran your scripts on a typical desktop/laptop, I'd agree that the 200 range is an adequate starting point. If on the other hand you're deploying with more hardware (lots of RAM [128GB+], large number of disk spindles (DAS, FC SANs), SSDs or RAM drives), I would expect that curve to look very different.</div>
<div><br></div><div>-bborie</div><div><br></div><div>I really need to build a performance test suite...</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">On Thu, Sep 19, 2013 at 4:55 PM, Duncan Golicher <span dir="ltr"><<a href="mailto:dgolicher@gmail.com" target="_blank">dgolicher@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Here is a simple speed test using R system.time to record the result after reloading the same raster with different tile sizes.<br>
<br><a href="https://dl.dropboxusercontent.com/u/2703650/SpeedTest.html" target="_blank">https://dl.dropboxusercontent.com/u/2703650/SpeedTest.html</a><br>
<br></div><div>Not had any time to anotate or explain the code but the test should be easily replicable (at least under Ubuntu). I provide a link to the data (which was used in the example on the geostat course). The code only runs without modification on Linux as I use system to send commands pasted toegether in R to the shell. Also needs unix odbc setting up.<br>
<br></div><div>Note that the point on raster overlay can be beaten easily for speed by the extract function in the R raster package. However the polygon overlays are now very fast and compare well with any alternative way of getting the result. Using PLR to run R functions within PostGIS is great if you want medians, quartiles etc or any other derived property.<span class="HOEnZb"><font color="#888888"><br>
</font></span></div><span class="HOEnZb"><font color="#888888"><div><br></div>Duncan<br><div><br><br></div></font></span></div><div class="HOEnZb"><div class="h5"><div class="gmail_extra"><br><br><div class="gmail_quote">
On Thu, Sep 19, 2013 at 4:32 PM, Duncan Golicher <span dir="ltr"><<a href="mailto:dgolicher@gmail.com" target="_blank">dgolicher@gmail.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div>Out of interest I quickly checked whether the conclusions still hold for PostGIS2.2.<br>The changes made in ST_Clip, and some other functions including st_value seem to have altered not just the absolute timing (much faster) but also the relative timing of operations as a function of tile size.<br>
Point on raster overlays are now slower when tile size is small (<50 pixels), whereas previously there was an almost linear increase with tile size. Bborie may be able to explain why this change has occurred. I will try to add an update to the weblog at some point in order to clarify the sitation. It is GOOD NEWS as apparently there is now a single optimum tile size for both polygon and point overlays and this does seem to lie at around 200 - 300 pixels using the same example as the weblog, although I have not run enough tests to confirm this. I'll try to find time to confirm this.<br>
<br></div><div>Duncan<br><br></div><div><br>
<br></div></div><div class="gmail_extra"><div><div><br><br><div class="gmail_quote">On Thu, Sep 19, 2013 at 10:36 AM, Pierre Racine <span dir="ltr"><<a href="mailto:Pierre.Racine@sbf.ulaval.ca" target="_blank">Pierre.Racine@sbf.ulaval.ca</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">You can also have a look at this article from Duncan Golicher if you are doing raster/vector analysis:<br>
<br>
<a href="http://duncanjg.wordpress.com/2012/10/30/tile-size-for-raster-vector-overlays-in-postgis/" target="_blank">http://duncanjg.wordpress.com/2012/10/30/tile-size-for-raster-vector-overlays-in-postgis/</a><br>
<div><div><br>
> -----Original Message-----<br>
> From: <a href="mailto:postgis-users-bounces@lists.osgeo.org" target="_blank">postgis-users-bounces@lists.osgeo.org</a> [mailto:<a href="mailto:postgis-users-" target="_blank">postgis-users-</a><br>
> <a href="mailto:bounces@lists.osgeo.org" target="_blank">bounces@lists.osgeo.org</a>] On Behalf Of Stephen Crawford<br>
> Sent: Friday, September 13, 2013 12:34 PM<br>
> To: <a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a><br>
> Subject: Re: [postgis-users] tile size<br>
><br>
> OK, thanks. I will give that a try.<br>
><br>
><br>
><br>
> On 9/13/2013 12:31 PM, Adam Eskreis wrote:<br>
><br>
><br>
> The most common tile size that I've seen in production is 256x256<br>
><br>
><br>
> On Fri, Sep 13, 2013 at 10:33 AM, Bborie Park<br>
> <<a href="mailto:dustymugs@gmail.com" target="_blank">dustymugs@gmail.com</a>> wrote:<br>
><br>
><br>
> Steve,<br>
><br>
> There really isn't. What I do recommend is that if your raster<br>
> data is not going to change over time (and you don't need to replicate the<br>
> database), load them as out-db rasters. That way, you can easily change tile<br>
> size within the database with ST_Tile.<br>
><br>
><br>
><br>
> -bborie<br>
><br>
><br>
> On Fri, Sep 13, 2013 at 6:06 AM, Stephen Crawford<br>
> <<a href="mailto:src176@psu.edu" target="_blank">src176@psu.edu</a>> wrote:<br>
><br>
><br>
> Hi All,<br>
><br>
> Is there a rule of thumb for determining the best tile<br>
> size when tiling a raster?<br>
><br>
> Thanks,<br>
> Steve<br>
><br>
><br>
> --<br>
> Stephen Crawford<br>
> Center for Environmental Informatics<br>
> The Pennsylvania State University<br>
</div></div>> <tel:<a href="tel:814.865.9905" value="+18148659905" target="_blank">814.865.9905</a>><br>
<div>><br>
><br>
><br>
> _______________________________________________<br>
> postgis-users mailing list<br>
> <a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a><br>
> <a href="http://lists.osgeo.org/cgi-" target="_blank">http://lists.osgeo.org/cgi-</a><br>
> bin/mailman/listinfo/postgis-users<br>
><br>
><br>
><br>
><br>
><br>
> _______________________________________________<br>
> postgis-users mailing list<br>
> <a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a><br>
> <a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-</a><br>
> users<br>
><br>
><br>
><br>
><br>
><br>
><br>
</div><div>> _______________________________________________<br>
> postgis-users mailing list<br>
> <a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a><br>
> <a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users</a><br>
><br>
><br>
> --<br>
> Stephen Crawford<br>
> Center for Environmental Informatics<br>
> The Pennsylvania State University<br>
> <a href="mailto:src176@psu.edu" target="_blank">src176@psu.edu</a><br>
> <a href="tel:814.865.9905" value="+18148659905" target="_blank">814.865.9905</a><br>
</div><div><div>_______________________________________________<br>
postgis-users mailing list<br>
<a href="mailto:postgis-users@lists.osgeo.org" target="_blank">postgis-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users</a><br>
</div></div></blockquote></div><br><br clear="all"><br></div></div>-- <br>Dr Duncan Golicher<br>Investigador Titular, <br>El Colegio de la Frontera Sur, Chiapas,Mexico<br>Mexico tel <a href="tel:%2B52%201%20967%20137%2094%2020" value="+5219671379420" target="_blank">+52 1 967 137 94 20</a><br>
Skype name duncangolicher<br>
<br>Publications: <a href="http://www.mendeley.com/profiles/duncan-golicher" target="_blank">http://www.mendeley.com/profiles/duncan-golicher</a><br><br>Senior lecturer, Bournemouth University, UK<br>Centre for Conservation Ecology & Environmental Change<br>
School of Applied Sciences<br>Christchurch House rm C218a<br>Bournemouth University<br>Fern Barrow<br>Poole (Dorset) BH12 5BB UK<br>Tel. <a href="tel:%2B44%20%280%291202%20961682" value="+441202961682" target="_blank">+44 (0)1202 961682</a><br>
<br>For list of publications see Researcher ID:<br><a href="http://www.researcherid.com/rid/B-4240-2009" target="_blank">http://www.researcherid.com/rid/B-4240-2009</a><br>
<br><a href="mailto:dgolicher@bournemouth.ac.uk" target="_blank">dgolicher@bournemouth.ac.uk</a><br><a href="mailto:dgoliche@ecosur.mx" target="_blank">dgoliche@ecosur.mx</a><br><br>Researcher ID:<br><a href="http://www.researcherid.com/rid/B-4240-2009" target="_blank">http://www.researcherid.com/rid/B-4240-2009</a><br>
</div>
</blockquote></div><br><br clear="all"><br>-- <br>Dr Duncan Golicher<br>Investigador Titular, <br>El Colegio de la Frontera Sur, Chiapas,Mexico<br>Mexico tel <a href="tel:%2B52%201%20967%20137%2094%2020" value="+5219671379420" target="_blank">+52 1 967 137 94 20</a><br>
Skype name duncangolicher<br><br>Publications: <a href="http://www.mendeley.com/profiles/duncan-golicher" target="_blank">http://www.mendeley.com/profiles/duncan-golicher</a><br>
<br>Senior lecturer, Bournemouth University, UK<br>Centre for Conservation Ecology & Environmental Change<br>School of Applied Sciences<br>Christchurch House rm C218a<br>Bournemouth University<br>Fern Barrow<br>Poole (Dorset) BH12 5BB UK<br>
Tel. <a href="tel:%2B44%20%280%291202%20961682" value="+441202961682" target="_blank">+44 (0)1202 961682</a><br><br>For list of publications see Researcher ID:<br><a href="http://www.researcherid.com/rid/B-4240-2009" target="_blank">http://www.researcherid.com/rid/B-4240-2009</a><br>
<br><a href="mailto:dgolicher@bournemouth.ac.uk" target="_blank">dgolicher@bournemouth.ac.uk</a><br>
<a href="mailto:dgoliche@ecosur.mx" target="_blank">dgoliche@ecosur.mx</a><br><br>Researcher ID:<br><a href="http://www.researcherid.com/rid/B-4240-2009" target="_blank">http://www.researcherid.com/rid/B-4240-2009</a><br>
</div>
</div></div><br>_______________________________________________<br>
postgis-users mailing list<br>
<a href="mailto:postgis-users@lists.osgeo.org">postgis-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users</a><br></blockquote></div><br></div>