<div dir="ltr">Hi<div><br></div><div>I did mine 30GB laz repo in less than 1 day with an old i7 4core and 16GB ram and SSD.</div><div><br></div><div>To avoid any memory problems you should (thanks to the help of Adam steer)</div><div><br></div><div>1) morton ordering all files (this facilitate a lot processing)</div><div>eg with:</div><div>#!/bin/bash<br>files=$( ls ./*.laz | sort )<br>for file in ${files}<br>do<br> filename=$( basename ${file} )<br> echo "Morton ordering: $file"<br> if [[ -f ./morton_ordered/${filename} ]]; then<br> echo "... already processed"<br> continue<br> fi<br> pdal translate -i ${file} -o "./morton_ordered/${filename}" --filter mortonorder<br> if [ $? -ne 0 ]; then<br> echo "failed with: $?"<br> exit 1<br> fi<br>done<br></div><div><br></div><div>2) then build in parts e.g. with </div><div>total_parts=256<br>for (( part=1; part<=$total_parts; part++ ))<br>do<br> echo "************************* ${part} of ${total_parts}"<br> entwine build -i ./morton_ordered/ -o ./entwine/Avila/ --tmp ./temp_process/ --threads 1 -s $part $total_parts<br> if [ $? -ne 0 ]; then<br> echo "$part/$total_parts exit with error: $?"<br> exit 1<br> fi<br>done<br></div><div><div><div dir="ltr" class="gmail_signature" data-smartmail="gmail_signature"><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><div dir="ltr"><div><br></div><div>I leaved thread to 1 to reduce memory occupation during runtime... I do not know what would be the limit. For sure using all 8 thread will create crash for me and my data</div><div><br></div><div>3) do entwine merging with:</div><div>entiwine merge <path for generated in step 2></div><div><br></div><div>regards</div><div><br></div><div>Luigi Pirelli<br><br>**************************************************************************************************<br>* LinkedIn: <a href="https://www.linkedin.com/in/luigipirelli" target="_blank">https://www.linkedin.com/in/luigipirelli</a><br>* Stackexchange: <a href="http://gis.stackexchange.com/users/19667/luigi-pirelli" target="_blank">http://gis.stackexchange.com/users/19667/luigi-pirelli</a><br>* GitHub: <a href="https://github.com/luipir" target="_blank">https://github.com/luipir</a><br>* Book: <a href="https://www.packtpub.com/eu/application-development/mastering-geospatial-development-qgis-3x-third-edition" target="_blank">Mastering QGIS3 - 3rd Edition</a></div><div>* Hire a team: <a href="http://www.qcooperative.net" target="_blank">http://www.qcooperative.net</a><br>**************************************************************************************************</div></div></div></div></div></div></div></div></div></div><br></div></div><br><div class="gmail_quote"><div dir="ltr" class="gmail_attr">On Wed, 27 May 2020 at 17:59, Albion SHABANI <<a href="mailto:albi.dony@hotmail.com">albi.dony@hotmail.com</a>> wrote:<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left:1px solid rgb(204,204,204);padding-left:1ex">
<div dir="ltr">
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Hello ,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
I have very large files like : >500Go and bigger (e57 and las).</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
What is the average time to convert these amount of data with Entwine (using the docker : <a href="https://hub.docker.com/r/connormanning/entwine" target="_blank">https://hub.docker.com/r/connormanning/entwine</a>) or is even possible ?</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
What hardware configuration would you recommend ( ? CPU and ? RAM) ?</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Best Regards,</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
Albion</div>
<div style="font-family:Calibri,Helvetica,sans-serif;font-size:12pt;color:rgb(0,0,0)">
<br>
</div>
</div>
_______________________________________________<br>
pdal mailing list<br>
<a href="mailto:pdal@lists.osgeo.org" target="_blank">pdal@lists.osgeo.org</a><br>
<a href="https://lists.osgeo.org/mailman/listinfo/pdal" rel="noreferrer" target="_blank">https://lists.osgeo.org/mailman/listinfo/pdal</a></blockquote></div>