[postgis-users] tile size
Bborie Park
dustymugs at gmail.com
Thu Sep 19 19:15:48 PDT 2013
Thanks for the graphs Duncan! My guess is that at the low tile sizes, the
individual round-trip time would be short but the volume of disk i/o is
what's coming into play. The opposite is true for large tile sizes where
the disk i/o volume is decreased but it takes longer for each round-trip.
Having said all that, I'm not willing to suggest that there is one range
appropriate for every user or deployment. Assuming you ran your scripts on
a typical desktop/laptop, I'd agree that the 200 range is an adequate
starting point. If on the other hand you're deploying with more hardware
(lots of RAM [128GB+], large number of disk spindles (DAS, FC SANs), SSDs
or RAM drives), I would expect that curve to look very different.
-bborie
I really need to build a performance test suite...
On Thu, Sep 19, 2013 at 4:55 PM, Duncan Golicher <dgolicher at gmail.com>wrote:
> Here is a simple speed test using R system.time to record the result after
> reloading the same raster with different tile sizes.
>
> https://dl.dropboxusercontent.com/u/2703650/SpeedTest.html
>
> Not had any time to anotate or explain the code but the test should be
> easily replicable (at least under Ubuntu). I provide a link to the data
> (which was used in the example on the geostat course). The code only runs
> without modification on Linux as I use system to send commands pasted
> toegether in R to the shell. Also needs unix odbc setting up.
>
> Note that the point on raster overlay can be beaten easily for speed by
> the extract function in the R raster package. However the polygon overlays
> are now very fast and compare well with any alternative way of getting the
> result. Using PLR to run R functions within PostGIS is great if you want
> medians, quartiles etc or any other derived property.
>
> Duncan
>
>
>
>
> On Thu, Sep 19, 2013 at 4:32 PM, Duncan Golicher <dgolicher at gmail.com>wrote:
>
>> Out of interest I quickly checked whether the conclusions still hold for
>> PostGIS2.2.
>> The changes made in ST_Clip, and some other functions including st_value
>> seem to have altered not just the absolute timing (much faster) but also
>> the relative timing of operations as a function of tile size.
>> Point on raster overlays are now slower when tile size is small (<50
>> pixels), whereas previously there was an almost linear increase with tile
>> size. Bborie may be able to explain why this change has occurred. I will
>> try to add an update to the weblog at some point in order to clarify the
>> sitation. It is GOOD NEWS as apparently there is now a single optimum tile
>> size for both polygon and point overlays and this does seem to lie at
>> around 200 - 300 pixels using the same example as the weblog, although I
>> have not run enough tests to confirm this. I'll try to find time to confirm
>> this.
>>
>> Duncan
>>
>>
>>
>>
>>
>> On Thu, Sep 19, 2013 at 10:36 AM, Pierre Racine <
>> Pierre.Racine at sbf.ulaval.ca> wrote:
>>
>>> You can also have a look at this article from Duncan Golicher if you are
>>> doing raster/vector analysis:
>>>
>>>
>>> http://duncanjg.wordpress.com/2012/10/30/tile-size-for-raster-vector-overlays-in-postgis/
>>>
>>> > -----Original Message-----
>>> > From: postgis-users-bounces at lists.osgeo.org [mailto:postgis-users-
>>> > bounces at lists.osgeo.org] On Behalf Of Stephen Crawford
>>> > Sent: Friday, September 13, 2013 12:34 PM
>>> > To: postgis-users at lists.osgeo.org
>>> > Subject: Re: [postgis-users] tile size
>>> >
>>> > OK, thanks. I will give that a try.
>>> >
>>> >
>>> >
>>> > On 9/13/2013 12:31 PM, Adam Eskreis wrote:
>>> >
>>> >
>>> > The most common tile size that I've seen in production is 256x256
>>> >
>>> >
>>> > On Fri, Sep 13, 2013 at 10:33 AM, Bborie Park
>>> > <dustymugs at gmail.com> wrote:
>>> >
>>> >
>>> > Steve,
>>> >
>>> > There really isn't. What I do recommend is that if your
>>> raster
>>> > data is not going to change over time (and you don't need to replicate
>>> the
>>> > database), load them as out-db rasters. That way, you can easily
>>> change tile
>>> > size within the database with ST_Tile.
>>> >
>>> >
>>> >
>>> > -bborie
>>> >
>>> >
>>> > On Fri, Sep 13, 2013 at 6:06 AM, Stephen Crawford
>>> > <src176 at psu.edu> wrote:
>>> >
>>> >
>>> > Hi All,
>>> >
>>> > Is there a rule of thumb for determining the
>>> best tile
>>> > size when tiling a raster?
>>> >
>>> > Thanks,
>>> > Steve
>>> >
>>> >
>>> > --
>>> > Stephen Crawford
>>> > Center for Environmental Informatics
>>> > The Pennsylvania State University
>>> > <tel:814.865.9905>
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > postgis-users mailing list
>>> > postgis-users at lists.osgeo.org
>>> > http://lists.osgeo.org/cgi-
>>> > bin/mailman/listinfo/postgis-users
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > postgis-users mailing list
>>> > postgis-users at lists.osgeo.org
>>> > http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-
>>> > users
>>> >
>>> >
>>> >
>>> >
>>> >
>>> >
>>> > _______________________________________________
>>> > postgis-users mailing list
>>> > postgis-users at lists.osgeo.org
>>> > http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>>> >
>>> >
>>> > --
>>> > Stephen Crawford
>>> > Center for Environmental Informatics
>>> > The Pennsylvania State University
>>> > src176 at psu.edu
>>> > 814.865.9905
>>> _______________________________________________
>>> postgis-users mailing list
>>> postgis-users at lists.osgeo.org
>>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>>>
>>
>>
>>
>> --
>> Dr Duncan Golicher
>> Investigador Titular,
>> El Colegio de la Frontera Sur, Chiapas,Mexico
>> Mexico tel +52 1 967 137 94 20
>> Skype name duncangolicher
>>
>> Publications: http://www.mendeley.com/profiles/duncan-golicher
>>
>> Senior lecturer, Bournemouth University, UK
>> Centre for Conservation Ecology & Environmental Change
>> School of Applied Sciences
>> Christchurch House rm C218a
>> Bournemouth University
>> Fern Barrow
>> Poole (Dorset) BH12 5BB UK
>> Tel. +44 (0)1202 961682
>>
>> For list of publications see Researcher ID:
>> http://www.researcherid.com/rid/B-4240-2009
>>
>> dgolicher at bournemouth.ac.uk
>> dgoliche at ecosur.mx
>>
>> Researcher ID:
>> http://www.researcherid.com/rid/B-4240-2009
>>
>
>
>
> --
> Dr Duncan Golicher
> Investigador Titular,
> El Colegio de la Frontera Sur, Chiapas,Mexico
> Mexico tel +52 1 967 137 94 20
> Skype name duncangolicher
>
> Publications: http://www.mendeley.com/profiles/duncan-golicher
>
> Senior lecturer, Bournemouth University, UK
> Centre for Conservation Ecology & Environmental Change
> School of Applied Sciences
> Christchurch House rm C218a
> Bournemouth University
> Fern Barrow
> Poole (Dorset) BH12 5BB UK
> Tel. +44 (0)1202 961682
>
> For list of publications see Researcher ID:
> http://www.researcherid.com/rid/B-4240-2009
>
> dgolicher at bournemouth.ac.uk
> dgoliche at ecosur.mx
>
> Researcher ID:
> http://www.researcherid.com/rid/B-4240-2009
>
> _______________________________________________
> postgis-users mailing list
> postgis-users at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/postgis-users/attachments/20130919/6dea1e28/attachment.html>
More information about the postgis-users
mailing list