[gdal-dev] GDAL performance compared with libtiff

[Even Rouault]

Immanuel,

> 
> sorry for not getting back earlier, I got distracted by other stuff to fix.
> Thanks for your reply. At the moment I will stick with your proposed way of
> writing with libtiff and adding geo data with gdal later, as I have to get
> that code running quickly. But I will try have a look at it again later.
> 
> Now there is another problem slowing the process down. I want to add some
> exif and xmp tags to the images. After fiddling arround with libtiff, which
> has a rather simple (for my case not really useful) conversion of floating
> point values to rational values, I went with exiv2. Tagging with it is very
> easy and straightforward. Unfortunately due to some harddrive buffering (i
> think) the image is locked for a longer time than the actual writing with
> libtiff takes. That blocks exiv2 for quite some time until it gets write
> access to that file, on my machine around 50-60 ms, which is 100 times of
> the actual writing time. So that's really bad.
> 
> Now I'm reviewing my other options. As I'm going to write the geo
> information using GDAL i think it should also use if to write the metadata
> (I did not test it, but I think it will also have to wait for the drive to
> be finished, but then I would stick completely with GDAL, as it's (at the
> moment) slower writing compared to libtiff is still a factor of  >10 faster
> than the lock takes).
> 
> So my question now is, if it is possible to write all kinds of tags using
> GDAL.

Short answer: no. Only a few TIFF tags can be written currently (the one 
listed in "Metadata" paragraph of http://www.gdal.org/frmt_gtiff.html)

Adding support for writing XMP (the user would provide the XML) would be easy. 
Writting EXIF data would probably be a bit more involved.

> gdalinfo reports some of the tags written with exiv2, but not all,
> for examples Exif.Photo.Make is not listed. And if so how do I do that?

The list of EXIF tags that are read currently is in gcore/gdalexif.h

> I
> was able to add custom tags using SetMetaData() on the dataset, but these
> can (not surprizingly) not be recognized by the applications, which will
> use my images.
> Maybe you or others can help me with that.

One option if you want to act directly with libtiff, but don't want to close 
the GDAL dataset and reopen with libtiff afterwards is to call 
GetInternalHandle(NULL) on the GDAL dataset object. It returns a TIFF* handle. 
However this is semi-private / implementation detail and what you could do 
with it could potentially interact badly with what GDAL does, so to be used 
only if really necessary.

Even

> 
> Thanks for your assistance,
> Immanuel
> 
> 2015-08-07 20:25 GMT+02:00 Even Rouault <even.rouault at spatialys.com>:
> > Le jeudi 06 août 2015 17:29:06, Immanuel Weber a écrit :
> > > Hi all,
> > > 
> > > I'm working on an application where lots of images have to be written.
> > > Until now I'm using libtiff for that task. But as I want to add
> > > georeferencing and the like I'm having a look at GDAL right now. I did
> > > a quick evaluation of its performance related to libtiff and
> > > unfortunately it has quite a lot of overhead. I first tested GDAL 1.10
> > > and it was a factor of 6-7 slower than the direct write with libtiff.
> > > With GDAL 2.0 it got a little better with a factor of 4. (I took a 1,5
> > > MP image and wrote for a few hundred times to the disk and libtiff
> > > takes about 3 ms for
> > 
> > that,
> > 
> > > where as GDAL around 12 ms). I read about a performance issue with
> > > GDALDataSet::RasterIO vs GDALRasterBand::ReadBlock and tried to
> > > implement the storage using GDALRasterBand::WriteBlock* but that
> > > almost took more time than the GDALDataSet::RasterIO version.
> > > 
> > > Is there any other way to write tiffs quickly with GDAL, or is that
> > > kind
> > 
> > of
> > 
> > > overhead expected for the added functionality?
> > 
> > Immanuel,
> > 
> > WriteBlock() with a INTERLEAVE=BAND file should be really close to pure
> > libtiff
> > performance. Other cases will involve a few extra buffer copies.
> > 
> > Another option is to write the imagery with libtiff as you did, and then
> > open
> > the file in update mode with GDAL to add the georeferencing. This will
> > just rewrite the TIFF header.
> > 
> > > Thanks and best regards,
> > > Immanuel
> > > 
> > > *I wasn't really successful in doing that, because I didn't find a way
> > > to write pixel interleaved images with that
> > 
> > GDAL blocks are per band. But GDAL will take care of combining several
> > blocks
> > of same (x,y) for a INTERLEAVE=PIXEL TIFF. To get the best performance,
> > in INTERLEAVE=PIXEL, you should do (pseudo-code):
> > 
> > for y in yblocks:
> >         for x in xblocs:
> >                 for b in bands:
> >                         GetRasterBand(b)->WriteBlock(x,y, buffer)
> > 
> > to avoid partial TIFF tiles/strips to be written, read and re-written
> > when the
> > data for the band comes.
> > 
> > Even
> > 
> > --
> > Spatialys - Geospatial professional services
> > http://www.spatialys.com

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com