<div dir="ltr"><div><div><div><div><div><div><div><div><div><div><div><div>Hey,<br></div>you can do it with pointcloud (<a href="https://github.com/pramsey/pointcloud" target="_blank">https://github.com/pramsey/pointcloud</a>)<br>
</div>Currently the database I use with 600 millions points is doing fine.<br>
<br></div>In your case the problem is not : i want to render 500
millions points, but more : i want to render the adapted number of
points given a scale (the problem is called generalization).<br><br></div>In
my case using pointcloud you're about 100k pts/second streaming. with
parallel processing you could increase a tad to 500k points/sec.<br>
</div>If your points are truly only xyz+data, I guess you could make it to 1 million pts/sec.<br><br></div>So for your usage I would say your goal would be <br>"which are the 1 million (max!) points in the area that best represent my dataset?"<br>
<br></div>To answer efficiently, you have to use Level Of Detail, which I started to add to pointcloud.<br></div>The idea is the same as for pyramidal raster : each patch (group of points) is ordered with a quad tree order, <br>
</div>so if you read the first 5 (1+4) points you have a representative
subset, if you read 21 (1+4+16) first point you have a more detailed
representative subset, etc.<br><br>The process is illustrated by pictures attached : <br>
_create patches (group) of points regularly spaced (1m3 here).<br><div style="margin-left:40px"><a href="https://github.com/Remi-C/LOD_ordering_for_patches_of_points/blob/master/screen/2014-02-10%2015_18_32-CloudCompare%20-%20%5B3D%20View%201%5D.png">image here</a><br>
</div></div>_then, display points from max details to less details.<br><div style="margin-left:40px"><a href="https://github.com/Remi-C/LOD_ordering_for_patches_of_points/blob/master/screen/2014-02-10%2015_22_24-CloudCompare%20-%20%5B3D%20View%201%5D.png">Level 6 </a><br>
</div><div style="margin-left:40px"><a href="https://github.com/Remi-C/LOD_ordering_for_patches_of_points/blob/master/screen/2014-02-10%2015_24_50-CloudCompare%20-%20%5B3D%20View%201%5D.png">Level 5</a><br></div><div style="margin-left:40px">
<a href="https://github.com/Remi-C/LOD_ordering_for_patches_of_points/blob/master/screen/2014-02-10%2015_25_37-CloudCompare%20-%20%5B3D%20View%201%5D.png">Level 4</a><br></div><div style="margin-left:40px"><a href="https://github.com/Remi-C/LOD_ordering_for_patches_of_points/blob/master/screen/2014-02-10%2015_26_44-CloudCompare%20-%20%5B3D%20View%201%5D.png">Level 3</a><br>
</div><div style="margin-left:40px"><a href="https://github.com/Remi-C/LOD_ordering_for_patches_of_points/blob/master/screen/2014-02-10%2015_27_36-CloudCompare%20-%20%5B3D%20View%201%5D.png">Level 2</a><br></div><div style="margin-left:40px">
<a href="http://2014-02-10 15_28_43-CloudCompare - [3D View 1].png">Level 1</a><br></div><br></div>The
level of detail is just selected by reading more or less points. (so
after computing LOD for whole database once, no computing).<br><br></div>Then about rendering : if you want to render interactively about 1 million points, you should switch to WebGL (<a href="http://potree.org/demo/pompei/pompei.html">http://potree.org/demo/pompei/pompei.html</a>)<br>
<div>
<br>Cheers,<br>Rémi-C</div></div><div class="gmail_extra"><br><br><div class="gmail_quote">2014-03-18 8:40 GMT+01:00 Sandro Santilli <span dir="ltr"><<a href="mailto:strk@keybit.net" target="_blank">strk@keybit.net</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div class="">On Mon, Mar 17, 2014 at 03:41:03PM -0700, Paul Norman wrote:<br>
<br>
> - I have a dataset of 100 million to 500 million points<br>
> across the US<br>
<br>
</div>[...]<br>
<div class=""><br>
> Has anyone got any suggestions for how to efficiently render heatmaps<br>
> from datasets this large? I was thinking of two tables, one with the<br>
> raw data for high zooms and one pre-aggregated into space and time<br>
> bins (e.g. requests in a given box in a given time range) for use<br>
> in lower zooms. The aggregation operation would obviously be slow, but<br>
> I'd only have to do it once.<br>
<br>
</div>Pre-aggregation would surely help, big problem being amount<br>
of records that need to be read from disk. You could also look<br>
at postgis raster as the aggregation type.<br>
<br>
--strk;<br>
<br>
() ASCII ribbon campaign -- Keep it simple !<br>
/\ <a href="http://strk.keybit.net/rants/ascii_mails.txt" target="_blank">http://strk.keybit.net/rants/ascii_mails.txt</a><br>
<div class="HOEnZb"><div class="h5">_______________________________________________<br>
postgis-users mailing list<br>
<a href="mailto:postgis-users@lists.osgeo.org">postgis-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users" target="_blank">http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-users</a><br>
</div></div></blockquote></div><br></div>