[gdal-dev] GDAL performance compared with libtiff

[Immanuel Weber]

Hi Even,

sorry for not getting back earlier, I got distracted by other stuff to fix.
Thanks for your reply. At the moment I will stick with your proposed way of
writing with libtiff and adding geo data with gdal later, as I have to get
that code running quickly. But I will try have a look at it again later.

Now there is another problem slowing the process down. I want to add some
exif and xmp tags to the images. After fiddling arround with libtiff, which
has a rather simple (for my case not really useful) conversion of floating
point values to rational values, I went with exiv2. Tagging with it is very
easy and straightforward. Unfortunately due to some harddrive buffering (i
think) the image is locked for a longer time than the actual writing with
libtiff takes. That blocks exiv2 for quite some time until it gets write
access to that file, on my machine around 50-60 ms, which is 100 times of
the actual writing time. So that's really bad.

Now I'm reviewing my other options. As I'm going to write the geo
information using GDAL i think it should also use if to write the metadata
(I did not test it, but I think it will also have to wait for the drive to
be finished, but then I would stick completely with GDAL, as it's (at the
moment) slower writing compared to libtiff is still a factor of  >10 faster
than the lock takes).

So my question now is, if it is possible to write all kinds of tags using
GDAL. gdalinfo reports some of the tags written with exiv2, but not all,
for examples Exif.Photo.Make is not listed. And if so how do I do that? I
was able to add custom tags using SetMetaData() on the dataset, but these
can (not surprizingly) not be recognized by the applications, which will
use my images.
Maybe you or others can help me with that.

Thanks for your assistance,
Immanuel

2015-08-07 20:25 GMT+02:00 Even Rouault <even.rouault at spatialys.com>:

> Le jeudi 06 août 2015 17:29:06, Immanuel Weber a écrit :
> > Hi all,
> >
> > I'm working on an application where lots of images have to be written.
> > Until now I'm using libtiff for that task. But as I want to add
> > georeferencing and the like I'm having a look at GDAL right now. I did a
> > quick evaluation of its performance related to libtiff and unfortunately
> > it has quite a lot of overhead. I first tested GDAL 1.10 and it was a
> > factor of 6-7 slower than the direct write with libtiff. With GDAL 2.0 it
> > got a little better with a factor of 4. (I took a 1,5 MP image and wrote
> > for a few hundred times to the disk and libtiff takes about 3 ms for
> that,
> > where as GDAL around 12 ms). I read about a performance issue with
> > GDALDataSet::RasterIO vs GDALRasterBand::ReadBlock and tried to implement
> > the storage using GDALRasterBand::WriteBlock* but that almost took more
> > time than the GDALDataSet::RasterIO version.
> >
> > Is there any other way to write tiffs quickly with GDAL, or is that kind
> of
> > overhead expected for the added functionality?
>
> Immanuel,
>
> WriteBlock() with a INTERLEAVE=BAND file should be really close to pure
> libtiff
> performance. Other cases will involve a few extra buffer copies.
>
> Another option is to write the imagery with libtiff as you did, and then
> open
> the file in update mode with GDAL to add the georeferencing. This will just
> rewrite the TIFF header.
>
> >
> > Thanks and best regards,
> > Immanuel
> >
> > *I wasn't really successful in doing that, because I didn't find a way to
> > write pixel interleaved images with that
>
> GDAL blocks are per band. But GDAL will take care of combining several
> blocks
> of same (x,y) for a INTERLEAVE=PIXEL TIFF. To get the best performance, in
> INTERLEAVE=PIXEL, you should do (pseudo-code):
>
> for y in yblocks:
>         for x in xblocs:
>                 for b in bands:
>                         GetRasterBand(b)->WriteBlock(x,y, buffer)
>
> to avoid partial TIFF tiles/strips to be written, read and re-written when
> the
> data for the band comes.
>
> Even
>
> --
> Spatialys - Geospatial professional services
> http://www.spatialys.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20150908/08aae54d/attachment.html>