<html><head><style>body{font-family:Helvetica,Arial;font-size:13px}</style></head><body style="word-wrap: break-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-space;"><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">Hi Oscar, </div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">This sounds like a slightly more sophisticated version of the work done at Natural Resources Canada for what they call “geohash tree”. They did find that they got pretty good compression (even with the simple ascii-based key!) using the scheme, and it did allow easy random access to subsets of the data.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><a href="http://2013.foss4g.org/conf/programme/presentations/60/">http://2013.foss4g.org/conf/programme/presentations/60/</a></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">The downside was of course the cost of sorting things in the first place, but for a one-time cost on frequently accessed data, it’s not a bad thing. The “libght” soft dependency in pgpointcloud is to a (not so great) implementation of the scheme that I did for them a couple years ago. As a scheme, I think it cuts against the idea of having small patches that is core to the pgpointcloud concept. It makes more and more sense the larger your file is, in that it gets greater and greater leverage for random access.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">ATB,</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;">P.</div><div id="bloop_customfont" style="font-family:Helvetica,Arial;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;"><br></div><div id="bloop_sign_1429267881021870080" class="bloop_sign">
-- <br>
Paul Ramsey<br>
http://cleverelephant.ca<div>http://postgis.net
</div>
<script type="application/ld+json">
{
"@context": "http://schema.org",
"@type": "em",
"name": "John Doe",
"jobTitle": "Graduate research assistant",
"affiliation": "University of Dreams",
"additionalName": "Johnny",
"url": "http://www.example.com",
"address": {
"@type": "PostalAddress",
"streetAddress": "1234 Peach Drive",
"addressLocality": "Wonderland",
"addressRegion": "Georgia"
}
}
</script>
</div> <br><p style="color:#000;">On April 17, 2015 at 11:02:47 AM, Oscar Martinez Rubi (<a href="mailto:o.martinezrubi@tudelft.nl">o.martinezrubi@tudelft.nl</a>) wrote:</p> <blockquote type="cite" class="clean_bq"><span><div bgcolor="#FFFFFF" text="#000000"><div></div><div>
<title></title>
Hi,<br>
<br>
About the XYZ binding for better compression. In our research in
the NL escience center and TU Delft we have been thinking (not
testing yet though) about one possible approach for this.<br>
<br>
It is based on using space filling curves. So, once you have the
points that go in a block you could compute the morton/hilbert code
of the XYZ. Since all the points are close together such codes will
be extremely similar, so one could store only the increments which
could fit in many few bits. We have not tested or compared this
with any of the other compressions but we just wanted to share it
with you just in case you find it useful!<br>
<br>
An additional improvement would be to sort the points within the
blocks according to the morton code. Then, when doing crop/filter
operations in the blocks one can use the morton codes for the
queries similarly to what we presented in our papers with the flat
table (without blocks), I attach one of them (see section 5.2). In
a nutshell: You convert the query region into a set of
quadtree/octree nodes which can be also converted to morton code
ranges (thanks to relation between morton/hilbert curve and a
quadtree/octree). You scale down the ranges to increments (like you
did when storing the point of the block) and then you simply do
range queries in sorted data with a binary algorithm. In this way
you avoid the decompression of the morton code for most of the
block. This filtering is equivalent to a bbox filter so it still
requires a point in polygon check for some of the points.<br>
<br>
Kind Regards,<br>
<br>
Oscar.<br>
<br>
<br>
<div class="moz-cite-prefix">On 16-04-15 18:15, Rémi Cura
wrote:<br></div>
<blockquote cite="mid:CAJvUf_vVt+rSCU2A404dvXjxRP-Fj7+fQaCJoyix=2pxKRyQFA@mail.gmail.com" type="cite">
<div dir="ltr">
<div><span style="font-family:monospace,monospace">epic fail ! I
had avoided html just for you<br></span></div>
<div><span style="font-family:monospace,monospace"><br>
Dataset |subset size |
compressing | decompressing |<br>
|(Million pts)|(Million pts/s)|(Million pts/s)|<br>
Lidar | 473.3
| 4,49 |
4,67 |<br></span></div>
<span style="font-family:monospace,monospace">21-atributes |
105.7 | 1,11 |
2,62 |<br></span>
<div>
<div><span style="font-family:monospace,monospace">Stereo
| 70 |
2,44 |
7,38 |<br>
<br></span></div>
<div><span style="font-family:monospace,monospace">Cheers<br></span></div>
</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2015-04-16 17:42 GMT+02:00 Sandro Santilli
<span dir="ltr"><<a moz-do-not-send="true" href="mailto:strk@keybit.net" target="_blank">strk@keybit.net</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<span class="">On Thu, Apr 16, 2015 at 05:30:12PM +0200, Rémi Cura
wrote:<br>
> OUps<br>
><br>
> Dataset | subset size(Million
pts) | compressing (Million pts/s) |<br>
> decompressing (Million pts/s)<br>
> Lidar |
473.3
|
4,49<br>
> |
__4,67__<br>
> 21 attributes |
105.7
|<br>
> 1,11
|
2,62<br>
> Stereo |
70
|
2,44<br>
> |
7,38<br>
<br></span> These tables aren't really readable here.<br>
Could you make sure to use a fixed-width font to write those
tables<br>
and to keep lines within 70 columns at most ?<br>
<br>
--strk;<br></blockquote>
</div>
<br></div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
pgpointcloud mailing list
<a class="moz-txt-link-abbreviated" href="mailto:pgpointcloud@lists.osgeo.org">pgpointcloud@lists.osgeo.org</a>
<a class="moz-txt-link-freetext" href="http://lists.osgeo.org/cgi-bin/mailman/listinfo/pgpointcloud">http://lists.osgeo.org/cgi-bin/mailman/listinfo/pgpointcloud</a>
</pre></blockquote>
<br>
<hr></div></div></span></blockquote></body></html>