[pdal] Correct destination file guarantee?

[Kirk Waters - NOAA Federal]

This may be too much on the Unix side of thought, but could pdal catch a
signal and clean up the output file. That puts it on the user to interrupt
using the right signal to the process and wouldn't require a copy.

Kirk Waters, PhD, BJCP          | NOAA Office for Coastal Management
Applied Sciences Program      | 2234 South Hobson Ave
843-740-1227 (empty office)   | Charleston, SC 29405
843-324-2203 (cell during COVID)
coast.noaa.gov/digitalcoast



On Tue, Nov 10, 2020 at 7:22 AM Andrew Bell <andrew.bell.ia at gmail.com>
wrote:

>
> My take on this is that it's expensive to write to a temporary file and
> then copy it. Some output files very are large. Since you, the user, are
> the one interrupting the process, it seems that it's up to you to clean up.
> This behavior is also consistent with most other programs.
>
> On Tue, Nov 10, 2020 at 5:50 AM Peder Axensten <Peder.Axensten at slu.se>
> wrote:
>
>> Hi!
>>
>> If pdal is interrupted while saving to the destination file, it might
>> result in a corrupt file. This is not unreasonable, but could be avoided.
>>
>> We use a make script to process large amounts of files and sometimes we
>> have to interrupt processing for different reasons. We then risk to have
>> corrupt files that make will consider final when rerunning the script, so
>> the files are left in a corrupt state.
>>
>> Would it be a good idea to make pdal by default save the contents to a
>> temporary file and then move the temporary file to the destination file?
>> This way either a correct file is produced or nothing. I'm implementing
>> this in the make script – it is somewhat cumbersome but will work ok, I
>> guess.
>>
>> Isn’t it a very attractive and useful guarantee: if the destination file
>> is produced, then it is correct?
>>
>> Best regards,
>>
>> Peder Axensten
>> Research engineer
>>
>> Remote Sensing
>> Department of Forest Resource Management
>> Swedish University of Agricultural Sciences
>> SE-901 83 Umeå
>> Visiting address: Skogsmarksgränd
>> Phone: +46 90 786 85 00
>> peder.axensten at slu.se, www.slu.se/srh
>>
>> The Department of Forest Resource Management is environmentally certified
>> in accordance with ISO 14001.
>>
>> ---
>> När du skickar e-post till SLU så innebär detta att SLU behandlar dina
>> personuppgifter. För att läsa mer om hur detta går till, klicka här <
>> https://www.slu.se/om-slu/kontakta-slu/personuppgifter/>
>> E-mailing SLU will result in SLU processing your personal data. For more
>> information on how this is done, click here <
>> https://www.slu.se/en/about-slu/contact-slu/personal-data/>
>> _______________________________________________
>> pdal mailing list
>> pdal at lists.osgeo.org
>> https://lists.osgeo.org/mailman/listinfo/pdal
>
>
>
> --
> Andrew Bell
> andrew.bell.ia at gmail.com
> _______________________________________________
> pdal mailing list
> pdal at lists.osgeo.org
> https://lists.osgeo.org/mailman/listinfo/pdal
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/pdal/attachments/20201110/f1351a61/attachment.html>