[pdal] pgpointcloud selector filter and max_capacity

[Oscar Martinez Rubi]

Thanks for your response!

Well, about the optimal block size. I did a test 1 year ago that 
suggested that using small blocks (fitting to page size) was not as 
beneficial as one would imagine.

I loaded two datasets, one with 20 million points and another with 210 
million points with both compression and without it.

And I tried with different block sizes (from 300 to 5000) to both load 
and query the data.

In the loading (with PDAL) I only saw benefit of using small blocks in 
the loading of the small dataset and if dimensional compression was 
used. In the rest of cases (uncompressed and/or larger dataset) using 
larger blocks was faster in loading. See attached figure.

In the queries, only some complex queries (query 6 in the graph) has 
benefit from smaller block sizes. Other not-so-complex queries (query 2 
and 4 in the graphs) such as rectangles benefit from larger sizes.

Since then, I have been using a block size of around 3000.

O.


On 04-02-15 16:11, Howard Butler wrote:
>> On Feb 4, 2015, at 3:37 AM, Oscar Martinez Rubi <O.MartinezRubi at tudelft.nl> wrote:
>>
>> 2 - There seems to be a hardcoded maximum of 400 points per patch in the pgpointcloud writer. I know that is the recommended patch size but for massive point clouds I use larger number which use to be possible with older PDALs but not anymore.
>> About the capacity I found out that i need to specify the capacity in both writter and chipper.
> Because of the way PDAL used to work, you needed to specify the patch size on both the writer and the filters.chipper. That doesn't have to be the case anymore (but still is). The issue that's left is how to communicate the filter's capacity down to the pgpointcloud writer...
>
> I would note that during my tests, there was a huge performance penalty for patches greater than a database page size due to the way that toast works [1]. With compression turned on, this will allow you to have more points per patch, but I think you still want their total storage to be less than the page size if you can help it.
>
> Howard
>
>
> [1] http://www.postgresql.org/docs/9.4/static/storage-toast.html

-------------- next part --------------
A non-text attachment was scrubbed...
Name: sol1_querying_time.png
Type: image/png
Size: 89430 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20150204/cc0e35f2/attachment-0002.png>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sol1_loading.png
Type: image/png
Size: 58656 bytes
Desc: not available
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20150204/cc0e35f2/attachment-0003.png>