<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=us-ascii">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Kirk,</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Ran across this. Assuming you are using a linux box to store.</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<a href="https://www.ctrl.blog/entry/how-to-all-out-of-inodes.html#:~:text=That%20default%20depends%20on%20your,16%20KiB%20of%20disk%20space.&text=If%20you've%20a%20small,3%20million%20files%20and%20directories." id="LPlnk392862">https://www.ctrl.blog/entry/how-to-all-out-of-inodes.html#:~:text=That%20default%20depends%20on%20your,16%20KiB%20of%20disk%20space.&text=If%20you've%20a%20small,3%20million%20files%20and%20directories.</a><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
Doug</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> pdal <pdal-bounces@lists.osgeo.org> on behalf of Kirk Waters - NOAA Federal <kirk.waters@noaa.gov><br>
<b>Sent:</b> Wednesday, April 14, 2021 7:48 AM<br>
<b>To:</b> pdal <pdal@lists.osgeo.org><br>
<b>Subject:</b> [EXTERNAL] [pdal] Entwine build failure</font>
<div> </div>
</div>
<div>
<p> </p>
<div>
<table border="2" cellspacing="4" cellpadding="0" style="border-color:black; background-color:yellow; margin-left:auto; margin-right:auto">
<tbody>
<tr>
<td>
<p style="text-align:center"><strong> This email has been received from outside of DOI - Use caution before clicking on links, opening attachments, or responding. </strong></p>
</td>
</tr>
</tbody>
</table>
</div>
<p><br>
<br>
</p>
<div>
<div dir="ltr">While I've been able to build most of my datasets with Entwine, I've run into issues with one in particular. I don't know if it's related to the number of input files, the geographic extent, or just the total volume. After making it through 80%
of the files, it starts getting an exception that it can't open a file for output. This is after running for 128 hours with 10 threads. The logs look like this where things start to go bad:<br>
<br>
Adding 31830 - /san1/reef/lidar4_z/geoid18/data/4800/Block_010/20140122_166500e_3749500n.laz<br>
Done 31828<br>
Adding 31831 - /san1/reef/lidar4_z/geoid18/data/4800/Block_010/20140122_166000e_3749000n.laz<br>
128:20:20 - 80% - 201,421,974,813 - 1,569(1,131)M/h - 216W - 46R - 396A<br>
128:20:30 - 80% - 201,426,046,237 - 1,569(1,465)M/h - 0W - 87R - 605A<br>
128:20:40 - 80% - 201,430,682,909 - 1,569(1,669)M/h - 30W - 43R - 700A<br>
Exception in pool task: writers.las: Couldn't open file '/san1/dem1_z/entwine/geoid18/4800/ept-data/16-24378-21598-65505.laz' for output.<br>
Exception in pool task: writers.las: Couldn't open file '/san1/dem1_z/entwine/geoid18/4800/ept-data/16-24379-21589-65505.laz' for output.<br>
Exception in pool task: writers.las: Couldn't open file '/san1/dem1_z/entwine/geoid18/4800/ept-data/16-24378-21597-65505.laz' for output.<br>
<br>
After that, it's all exceptions and exit with a -9 return code. It might be an issue of the output directory size. The ept-data directory has a size of over 335Mb and contains over 4 million files. It's possible that it's taking so long to work with that directory
that something times out. Another possibility is that I'm running out of inodes on the output drive (that directory is taking over 20% of the total inodes possible on the drive). There's definitely space on the drive. This may not be an issue for the software
to solve. Is there a troubleshooting guide somewhere that I missed?
<div><br clear="all">
<div>
<div dir="ltr" class="x_gmail_signature">
<div dir="ltr">
<div>
<div dir="ltr">
<div>
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div dir="ltr">
<div><font face="arial, helvetica, sans-serif">Kirk Waters, PhD, BJCP | NOAA Office for Coastal Management<br>
</font></div>
<div><font face="arial, helvetica, sans-serif">Applied Sciences Program | 2234 South Hobson Ave<br>
843-740-1227 (empty office) | Charleston, SC 29405</font></div>
<div><font face="arial, helvetica, sans-serif">843-324-2203 (cell during COVID) </font>
<div><font face="arial, helvetica, sans-serif"><a href="https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcoast.noaa.gov%2Fdigitalcoast&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C6ad450199b9c44e2df5908d8ff3d4b89%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637539986664301396%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=4EJy6GLJWxgnVTw4dXqqAOneTShNQaPzwRV5I7VTVFw%3D&reserved=0" originalsrc="http://coast.noaa.gov/digitalcoast" shash="QPPI70FYmuJyZg6G2SaiXaCd8y/+63PkR5tpaSui3/EP3g0rI1Q20sz4uvKX9RMXjchgbX7rsLmALGg5euor8dzNzyNewSPOy9WsucAol93vrkDf0jaTF/ZqSzeQQYbnzgkAhRbKk+6hUvuTUoh36iQx3QVvN1An3vsUGIazRGo=" target="_blank">coast.noaa.gov/digitalcoast</a></font><br>
</div>
</div>
<div><br>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</body>
</html>