[GRASS-dev] Segment library zero filling

[Markus Metz]

On Mon, May 20, 2019 at 7:18 PM Vaclav Petras <wenzeslaus at gmail.com> wrote:
>
>
>
> On Mon, May 20, 2019 at 11:57 AM Markus Metz <
markus.metz.giswork at gmail.com> wrote:
>>
>>
>>
>> On Mon, May 20, 2019 at 5:39 PM Vaclav Petras <wenzeslaus at gmail.com>
wrote:
>> >
>> > Hi MarkusM and all,
>> >
>> > I'm trying to understand if the Segment_open() [1] function fills with
zeros or not. I don't think it does since it is calling G_malloc (malloc)
or Segment_format_nofill(). However, it is not completely clear to me what
is supposed to be doing because documentation still says it calls
Segment_format() and I don't understand context of the related commit [2,
3] and the usage of lseek and USE_LSEEK are not clear to me from format.c
[4].
>>
>> Segment_open() uses Segment_format_nofill() [1], if it can not use the
all-in-memory cache. The documentation has not been updated accordingly
(yet).
>
>
> Thank you. Just be to be sure: Now, if you want to ensure that it is
zero-filled across platform, you need to do it yourself using
Segment_put(), right?

All modules that use the segment lib use Segment_put() anyway to load data

> (Assuming you want to use Segment_open().)

or Segment_format_nofill() + Segment_init()

> The exact role of lseek() here is still unclear to me (the hole and \0
bytes).
man lseek:
       lseek() allows the file offset to be set beyond the end of the file
(but this does not change the size of the file).  If data is later  written
       at this point, subsequent reads of the data in the gap (a "hole")
return null bytes ('\0') until data is actually written into the gap.

More info: disk space is allocated and file size increases only when actual
data are written into the gap

>
> Would Segment_open_zero_fill() make sense, i.e. do you know if
Segment_fill() faster then Segment_put()?

Segment_put() is used anyway to load the actual data, therefore the
question if a new Segment_open_zero_fill() is an alternative to
Segment_put() is invalid because you need to use Segment_put() in any case
to populate the Segment structure with meaningful data.

Markus M
>
>>
>> The advantage of no fill, only lseek, is that it is faster, the
disadvantage is that any "no space left on device" error will be
encountered only later on, and you always need to check the return code of
Segment_put().
>
>
> Makes sense. I'll document that as well.
>
>>
>>
>> HTH,
>>
>> Markus M
>>
>> [1] https://github.com/OSGeo/grass/blob/master/lib/segment/open.c#L89
>>
>> >
>> > Markus, can you please clarify that for me? I will then update the
documentation with whatever is needed.
>> >
>> > Thanks,
>> > Vaclav
>> >
>> > [1]
https://grass.osgeo.org/programming7/segment_2open_8c.html#ae24d2e794c66c0512b67d7cea8b2ba9a
>> > [2]
https://github.com/OSGeo/grass/commit/7a0d8d749537acd6d5c4baea11dbb6167fdef916
>> > [3] https://trac.osgeo.org/grass/changeset/73268
>> > [4] https://github.com/OSGeo/grass/blob/master/lib/segment/format.c
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/grass-dev/attachments/20190520/ead38974/attachment.html>