[PROJ] Grid CDN Update and Crowdfunding Request

[Even Rouault]

Chris,

> If PROJ is adding a grid format it makes sense to me to allow it to server
> for deformation grids (just a horizontal/vertical/3d shift with a time
> function) as well the classic case and geoid models.  It seems that we
> could use TIFF tags to handle them with very little extra effort - not that
> I'm familiar with TIFF.  LINZ would be very interested in supporting that
> as an extra feature as we don't have a binary format for publishing our
> deformation grids.  I'm just being a bit opportunistic :-)

TIFF is rather versatile, so it is likely you can make a deformation model fit 
into it.

> On  level of subgrids if the algorithm is to simple pick the first grid that
> includes the point, then that provides all the generality we want (far more
> than simply nested grids), as that doesn't enforce any sort of parent child
> relationship.

Do we need an explicit concept of a parent grid at all ? If we keep the logic 
"first match, first used", then parent grids should be put at the end of the 
chain of the grids
(or at the beginning if we choose for the equivalent "last matched, used", 
where the search would start by the end of the sequence)

> On the other hand it could be less efficient for
> implementation as it would require searching through all the lowest level
> child grids for points only on the parent.

If the number of grids in the file is reasonable, then bbox testing should be 
quite fast. We don't need to load the grid data at all. If it would be larger 
than a few tens of thousands, then we might have to build a in-memory index of 
grid envelopes, but I don't think that would really be needed.
For the network based approach, the idea would be to use the same tricks as 
used for cloud optimized geotiffs for pyramids, that is all TIFF IFD headers 
are put at the beginning of the file, so in a initial GET, we can get all the 
descriptors.

> 
> On using ints rather than float32 the only reason is that there might be
> opportunities for compression.  In many case int2 with scale and offset
> could equally server the purpose, and then you've halved the size of the
> file (and grid memory possibly).  But certainly not essential.

The DEFLATE and LZW algorithm of TIFF can use a mode of the differencial 
predictor designed specifically for IEEE floating point data.
See http://chriscox.org/TIFFTN3d1.pdf , page 3
This is handled by GDAL & libtiff.
That said, the integer-based differencial predictor with scale & offset might 
perhaps perform better. I guess some experimentations should be done to see 
the effects.

> On uncertainties of components that is certainly not a requirement at the
> moment - we don't provide them at the moment.  It is a feature of NTv2, and
> I can certainly imagine it becoming more desirable as accuracy requirements
> of transformations become more demanding and uncertainty of transformations
> becomes more critical.

In TIFF, you can put has many bands as you like, either with pixels of all 
bands packed together (pixel interleaving), or in separate blocks (band 
interleaving). The only constraint is that for a given grid, all bands must 
have the same data type.

> The main thing I would be hoping for is that the grid can support 1/2/3
> dimensional displacements (ideally without requiring all grids to define 3
> dimensional displacements).

Would potentially be possible given my above remark.

> At the risk of pushing the barrow too far, a nice to have would be a grid
> set id in the grid headers so that we could have more than displacement
> field.   Each grid set would include one or more grids.    The search
> algorithm would select and sum the first grid of each set including the
> calculation point.  This could include an optional simple time function for
> each grid to support deformation models.  With that simple addition we
> could completely encode almost any deformation model.

The GDAL_METADATA TIFF tag can hold any arbitrary KEY=VALUE pair:
https://www.awaresystems.be/imaging/tiff/tifftags/gdal_metadata.html 

Even

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com