[Gdal-dev] GDAL data model, caching, GDALProjDef

[Frank Warmerdam]

Hannu Koivisto wrote:
> Greetings,
> 
> I'm having some difficulties in seeing how to best use GDAL in my
> application.  It may be that I'm proceeding exactly as I should,
> but I can't help thinking that my data model requirements overlap
> GDAL's data model / API slightly and I can't see whether I should
> maybe extend GDAL or simply use a smaller subset of its
> functionality.  Also, if you have suggestions on higher level
> libraries layered on top of GDAL that might be suitable for me, I
> am all ears.
> 
> I'm working on a navigation application primarily for touring
> enduro motorcycling uses; basic moving map with GPS, waypoints,
> routes, etc. along the lines of OziExplorer plus some special
> features.  Map display is the hardest part (and also the only part
> relevant to GDAL use) since I need to handle very large maps and
> the software needs to work in handheld computers in addition to
> normal desktop ones.  In my primary use case I have a single map
> splitted to thousands of 800x800 (or so) PNG files.  PNG because it
> seems to compress well in this case and splitted because arbitrary
> regions in PNGs cannot be accessed efficiently.  I have separate
> overviews, some of which are splitted as well due to their large
> sizes.
> 
> My initial idea was to simply use GDAL to load each splitted file
> (that is currently needed for display and that is not already in
> memory) to memory and immediately close the datasets so that GDAL
> does not cache anything that I already do.  And manage overviews
> and cache data on my own.  However, I guess I could alternatively
> create a GDAL driver for such "virtual" maps that are stored to
> multiple files and then rely on GDAL's caching and overview
> capabilities.  One immediate demerit of this approach is that it is
> probably more complicated and the whole map display part, not to
> speak about map loading and caching, is something I'd prefer to
> offload to an existing library since the non-generic stuff for
> which I believe I have something "new" to contribute to is in the
> other features.
> 
> What would you suggest?

Hannu,

I would actually suggest handling the image as one large TIFF file
with tiling, pre-built overviews and deflate/zip compression.  Then you
could use the existing TIFF driver for the whole mosaic, but with similar 
performance characteristics.   This assumes the result is not larger
than 4GB.

A second approach would be to use your PNG files, but to prepare
a ".vrt" file describing how they should all be used as one large
virtual dataset.  However, I think there would be some performance
issues with using a .vrt file with *alot* of files listed.  Also, the
.vrt format is rather insensitive to the ideal block size of the
underlying images, and maintains a secondary level of caching which
does not really add much value.

A third option is to implement a custom multi-file driver that can be
used with any regularly tiled set of GDAL supported files such as your
PNGs.  Doing this well would be a bit of work, perhaps a day or two.

> Another thing, the documentation of GDALDataset::GetProjectionRef
> refers to GDALProjDef ("The returned string defines the projection
> coordinate system of the image in either PROJ.4 format or OpenGIS
> WKT format. It should be suitable for use with the GDALProjDef
> object to reproject positions").  I couldn't find documentation
> about or the implementation of GDALProjDef anywhere.  Am I correct
> that it is obsoleted by the OGR library?

You are correct that it is obsoleted by the OGR library - in particular
use of the OGRSpatialReference and OGRCoordinateTransformation classes.
I have updated the documentation accordingly.

Best regards,

-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam, warmerdam at pobox.com
light and sound - activate the windows | http://pobox.com/~warmerdam
and watch the world go round - Rush    | Geospatial Programmer for Rent