<div dir="ltr"><div dir="ltr"><br clear="all"><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div>On Fri, 29 May 2020 at 10:57, Albion SHABANI <<a href="mailto:albi.dony@hotmail.com">albi.dony@hotmail.com</a>> wrote:<br></div></div></div></div></div></div></div></div></div></div></div><div class="gmail_quote"><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<span style="color:rgb(50,49,48)">Thank you for your advice.</span><br>
</div>
<div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<span style="margin:0px;color:rgb(50,49,48);background-color:rgb(255,255,255);display:inline"><br>
</span></div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<span style="margin:0px;color:rgb(50,49,48);background-color:rgb(255,255,255);display:inline">I have to convert lots of files (like more than 1To and counting) for different projects.</span></div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
I have one with 500g almost (451 las files)</div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<br>
</div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<span style="margin:0px;font-size:14px;color:rgb(36,41,46);background-color:rgb(255,255,255);display:inline">I tried to build all of the input files at once but even If I have an EC2 with 16CPU and 128 Gio RAM it did not work (memory at 100%),
convertion killed by the system. I don't know if the memory problem is the use of docker image.</span></div></div></div></blockquote><div><br></div><div>no... it's not a docker problem, but (AFAIK) allocating meomory for all that points to maintain a scale independent visualization. BTW pdal crew can be more precise on thsi aspect.</div><div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)"><span style="color:rgb(36,41,46);font-size:14px">So, I am trying your solution (</span><span style="color:rgb(50,49,48)">Luigi) </span><span style="color:rgb(36,41,46);font-size:14px">to
avoid memory problems.</span><br></div>
<div style="margin:0px;font-size:12pt;font-family:Calibri,Helvetica,sans-serif;background-color:rgb(255,255,255)">
<br>
</div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">For example, If I have 500go to convert so let's say 700 las files.</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px"><br>
</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">If I want to convert them, I have to specify maximum number of files per sequence
? or just give total_part=700</span></font></div></div></div></blockquote><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div><div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)"><font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px"><br>
</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">In your case u have 256, this number is the number of las files in that folder
? or it is the maximum number of las files in a sequence to be put by entwine.</span></font></div></div></div></blockquote><div><br></div><div>that's a documentation fault and mine toio... I've to contribute back to documentation adding more datails. Parts is the number of suwares in which the covered areas have to be divided => HAVE to be a 2^ number => low number => big areas and points to index => BIIIIIG memory use! There you have to find a compromise with processing time and memory occupation.</div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<br>
</div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">I have the feeling that if I leave the total_parts at 700, entwine is putting
everything in the first sequence and when I will merge there will be only one id (nothing to merge). So it's the same problem as if i launch all of them at once !</span></font></div></div></div></blockquote><div><br></div><div>"I leave the total_parts at 700" could be a bug... should give error because in not a 2^ number. BTW it's amost important for the processing time to spatially order all points in morton ordering, this facilitate a lot the building of the indexes and memory occupation. </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px"><br>
</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">Thank you very much for your help, I'm struggling to understand how does this
work and be able to convert lots of files with entwine.</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px"><br>
</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">and by the way from this docker <a href="https://hub.docker.com/r/connormanning/entwine" id="gmail-m_6617487299205833954LPlnk316475" target="_blank">https://hub.docker.com/r/connormanning/entwine</a> <b>entwine
does not work with e57 files !!</b></span></font></div></div></div></blockquote><div><br></div><div>I see many traffic in dev list/ciommits about this... btw during morton ordering with pdal you can also translate in other formats becasue pdal is able to manage e57 </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex"><div dir="ltr"><div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px"><br>
</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">Best regards,</span></font></div>
<div style="margin:0px;font-size:15px;color:rgb(32,31,30);background-color:rgb(255,255,255)">
<font color="#24292e" face="-apple-system, BlinkMacSystemFont, Segoe UI, Helvetica, Arial, sans-serif, Apple Color Emoji, Segoe UI Emoji"><span style="margin:0px;font-size:14px">Albion</span></font></div>
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
</div>
</div>
</div>
_______________________________________________<br>
pdal mailing list<br>
<a href="mailto:pdal@lists.osgeo.org" target="_blank">pdal@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/pdal" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/pdal</a></blockquote><div><br></div><div>Luigi Pirelli<br><br>**************************************************************************************************<br>* LinkedIn: <a href="https://www.linkedin.com/in/luigipirelli" target="_blank">https://www.linkedin.com/in/luigipirelli</a><br>* Stackexchange: <a href="http://gis.stackexchange.com/users/19667/luigi-pirelli" target="_blank">http://gis.stackexchange.com/users/19667/luigi-pirelli</a><br>* GitHub: <a href="https://github.com/luipir" target="_blank">https://github.com/luipir</a><br>* Book: <a href="https://www.packtpub.com/eu/application-development/mastering-geospatial-development-qgis-3x-third-edition" target="_blank">Mastering QGIS3 - 3rd Edition</a></div><div>* Hire a team: <a href="http://www.qcooperative.net" target="_blank">http://www.qcooperative.net</a><br></div><div>************************************************************************************************** </div></div></div>