[gdal-dev] GDAL performance compared with libtiff

Immanuel Weber immanuel.weber at gmail.com
Wed Sep 9 07:59:14 PDT 2015


Hi Even,

thanks for your response. Okay, I see now. 
If I would extend asTIFFTags variable of type GTIFFTags in geotiff.cpp, I should be able to use more of the tifftags, or am I wrong? Maybe I have to have a look at the type handling and some other places where the table is used too.

I had another look at the interaction with exiv2. I now think that has something to do with the fact, that exiv2 memory maps the file, because I can repeatedly open, read and write the file with gdal and libtiff and basic fopen() and the code does not have to wait. I will get in contact with the guys of exiv2 to maybe get some insight.

Thanks for your help so far.
Cheers,
Immanuel

Am 08.09.2015 13:44:54 schrieb Even Rouault <even.rouault at spatialys.com>:
Immanuel,

>
> sorry for not getting back earlier, I got distracted by other stuff to fix.
> Thanks for your reply. At the moment I will stick with your proposed way of
> writing with libtiff and adding geo data with gdal later, as I have to get
> that code running quickly. But I will try have a look at it again later.
>
> Now there is another problem slowing the process down. I want to add some
> exif and xmp tags to the images. After fiddling arround with libtiff, which
> has a rather simple (for my case not really useful) conversion of floating
> point values to rational values, I went with exiv2. Tagging with it is very
> easy and straightforward. Unfortunately due to some harddrive buffering (i
> think) the image is locked for a longer time than the actual writing with
> libtiff takes. That blocks exiv2 for quite some time until it gets write
> access to that file, on my machine around 50-60 ms, which is 100 times of
> the actual writing time. So that's really bad.
>
> Now I'm reviewing my other options. As I'm going to write the geo
> information using GDAL i think it should also use if to write the metadata
> (I did not test it, but I think it will also have to wait for the drive to
> be finished, but then I would stick completely with GDAL, as it's (at the
> moment) slower writing compared to libtiff is still a factor of >10 faster
> than the lock takes).
>
> So my question now is, if it is possible to write all kinds of tags using
> GDAL.

Short answer: no. Only a few TIFF tags can be written currently (the one
listed in "Metadata" paragraph of http://www.gdal.org/frmt_gtiff.html)

Adding support for writing XMP (the user would provide the XML) would be easy.
Writting EXIF data would probably be a bit more involved.

> gdalinfo reports some of the tags written with exiv2, but not all,
> for examples Exif.Photo.Make is not listed. And if so how do I do that?

The list of EXIF tags that are read currently is in gcore/gdalexif.h

> I
> was able to add custom tags using SetMetaData() on the dataset, but these
> can (not surprizingly) not be recognized by the applications, which will
> use my images.
> Maybe you or others can help me with that.

One option if you want to act directly with libtiff, but don't want to close
the GDAL dataset and reopen with libtiff afterwards is to call
GetInternalHandle(NULL) on the GDAL dataset object. It returns a TIFF* handle.
However this is semi-private / implementation detail and what you could do
with it could potentially interact badly with what GDAL does, so to be used
only if really necessary.

Even

>
> Thanks for your assistance,
> Immanuel
>
> 2015-08-07 20:25 GMT+02:00 Even Rouault :
> > Le jeudi 06 août 2015 17:29:06, Immanuel Weber a écrit :
> > > Hi all,
> > >
> > > I'm working on an application where lots of images have to be written.
> > > Until now I'm using libtiff for that task. But as I want to add
> > > georeferencing and the like I'm having a look at GDAL right now. I did
> > > a quick evaluation of its performance related to libtiff and
> > > unfortunately it has quite a lot of overhead. I first tested GDAL 1.10
> > > and it was a factor of 6-7 slower than the direct write with libtiff.
> > > With GDAL 2.0 it got a little better with a factor of 4. (I took a 1,5
> > > MP image and wrote for a few hundred times to the disk and libtiff
> > > takes about 3 ms for
> >
> > that,
> >
> > > where as GDAL around 12 ms). I read about a performance issue with
> > > GDALDataSet::RasterIO vs GDALRasterBand::ReadBlock and tried to
> > > implement the storage using GDALRasterBand::WriteBlock* but that
> > > almost took more time than the GDALDataSet::RasterIO version.
> > >
> > > Is there any other way to write tiffs quickly with GDAL, or is that
> > > kind
> >
> > of
> >
> > > overhead expected for the added functionality?
> >
> > Immanuel,
> >
> > WriteBlock() with a INTERLEAVE=BAND file should be really close to pure
> > libtiff
> > performance. Other cases will involve a few extra buffer copies.
> >
> > Another option is to write the imagery with libtiff as you did, and then
> > open
> > the file in update mode with GDAL to add the georeferencing. This will
> > just rewrite the TIFF header.
> >
> > > Thanks and best regards,
> > > Immanuel
> > >
> > > *I wasn't really successful in doing that, because I didn't find a way
> > > to write pixel interleaved images with that
> >
> > GDAL blocks are per band. But GDAL will take care of combining several
> > blocks
> > of same (x,y) for a INTERLEAVE=PIXEL TIFF. To get the best performance,
> > in INTERLEAVE=PIXEL, you should do (pseudo-code):
> >
> > for y in yblocks:
> > for x in xblocs:
> > for b in bands:
> > GetRasterBand(b)->WriteBlock(x,y, buffer)
> >
> > to avoid partial TIFF tiles/strips to be written, read and re-written
> > when the
> > data for the band comes.
> >
> > Even
> >
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com

--
Spatialys - Geospatial professional services
http://www.spatialys.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20150909/0b6f23db/attachment.html>


More information about the gdal-dev mailing list