[pdal] [EXTERNAL] Entwine build failure

Wed Apr 14 06:04:21 PDT 2021

Doug,
Linux box is doing the writing. Output disk in a storage area network
device, so all I get for disk type is nfs. The inode limit is 21 million
and I suspect a significant portion of that is already taken up with the
more permanent disk contents.

Kirk

On Wed, Apr 14, 2021 at 8:16 AM Newcomb, Doug <doug_newcomb at fws.gov> wrote:

> Kirk,
> Ran across this. Assuming you are using a linux box to store.
>
> https://www.ctrl.blog/entry/how-to-all-out-of-inodes.html#:~:text=That%20default%20depends%20on%20your,16%20KiB%20of%20disk%20space.&text=If%20you've%20a%20small,3%20million%20files%20and%20directories.
>
> Doug
> ------------------------------
> *From:* pdal <pdal-bounces at lists.osgeo.org> on behalf of Kirk Waters -
> NOAA Federal <kirk.waters at noaa.gov>
> *Sent:* Wednesday, April 14, 2021 7:48 AM
> *To:* pdal <pdal at lists.osgeo.org>
> *Subject:* [EXTERNAL] [pdal] Entwine build failure
>
>
>
>
> * This email has been received from outside of DOI - Use caution before
> clicking on links, opening attachments, or responding.  *
>
>
>
> While I've been able to build most of my datasets with Entwine, I've run
> into issues with one in particular. I don't know if it's related to the
> number of input files, the geographic extent, or just the total volume.
> After making it through 80% of the files, it starts getting an exception
> that it can't open a file for output. This is after running for 128 hours
> with 10 threads. The logs look like this where things start to go bad:
>
> Adding 31830 -
> /san1/reef/lidar4_z/geoid18/data/4800/Block_010/20140122_166500e_3749500n.laz
>         Done 31828
> Adding 31831 -
> /san1/reef/lidar4_z/geoid18/data/4800/Block_010/20140122_166000e_3749000n.laz
> 128:20:20 - 80% - 201,421,974,813 - 1,569(1,131)M/h - 216W - 46R - 396A
> 128:20:30 - 80% - 201,426,046,237 - 1,569(1,465)M/h - 0W - 87R - 605A
> 128:20:40 - 80% - 201,430,682,909 - 1,569(1,669)M/h - 30W - 43R - 700A
> Exception in pool task: writers.las: Couldn't open file
> '/san1/dem1_z/entwine/geoid18/4800/ept-data/16-24378-21598-65505.laz' for
> output.
> Exception in pool task: writers.las: Couldn't open file
> '/san1/dem1_z/entwine/geoid18/4800/ept-data/16-24379-21589-65505.laz' for
> output.
> Exception in pool task: writers.las: Couldn't open file
> '/san1/dem1_z/entwine/geoid18/4800/ept-data/16-24378-21597-65505.laz' for
> output.
>
> After that, it's all exceptions and exit with a -9 return code. It might
> be an issue of the output directory size. The ept-data directory has a size
> of over 335Mb and contains over 4 million files. It's possible that it's
> taking so long to work with that directory that something times out.
> Another possibility is that I'm running out of inodes on the output drive
> (that directory is taking over 20% of the total inodes possible on the
> drive). There's definitely space on the drive. This may not be an issue for
> the software to solve. Is there a troubleshooting guide somewhere that I
> missed?
>
> Kirk Waters, PhD, BJCP          | NOAA Office for Coastal Management
> Applied Sciences Program      | 2234 South Hobson Ave
> 843-740-1227 (empty office)   | Charleston, SC 29405
> 843-324-2203 (cell during COVID)
> coast.noaa.gov/digitalcoast
> <https://gcc02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fcoast.noaa.gov%2Fdigitalcoast&data=04%7C01%7Cdoug_newcomb%40fws.gov%7C6ad450199b9c44e2df5908d8ff3d4b89%7C0693b5ba4b184d7b9341f32f400a5494%7C0%7C0%7C637539986664301396%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C2000&sdata=4EJy6GLJWxgnVTw4dXqqAOneTShNQaPzwRV5I7VTVFw%3D&reserved=0>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20210414/c97c6460/attachment-0001.html>