[postgis-devel] Regular raster blocking (was: Real extent feature lost after metadata as views)

Thu Dec 6 10:32:34 PST 2012

I came up with this (new but not so different) definition or regular_blocking:

----------------------------------------------------------
The regular_blocking constraint for a raster coverage (or raster table) is set to TRUE if and only if the following conditions are fulfilled:

1) All rasters (or tiles) have the same SRID, scale_x, scale_y, skew_x, skew_y, number of band and all the corresponding bands (having the same band number) have the same pixel types and nodata values. 

This is the case when the corresponding columns are set to a non NULL value in the raster_columns view (they are set to a value ONLY when all rasters have the same values). (If same_alignement is set, this condition is fulfilled (but not necessarily the following one about upper left corners).)

2) All rasters (or tiles) have their upper left corners aligned on a grid defined by the upper left corner of the coverage (or table) global extent and the block sizes.

This is the case when each raster upper left corner falls on a grid determined by a rasterization of the coverage extent having scale_x, scale_y, skew_x and skew_y set to match blocksize_x and blocksize_y. (We will need a new function ST_FallsOnGrid(geometry (point), raster) to ensure this condition).

3) All rasters (or tiles) have their width equal to blocksize_x and their height equal to blocksize_y. Only right-most tiles can have their width smaller than max_blocksize_x and only bottom tiles can have their height smaller than max_blocksize_y. Only the right-most, bottom tile of a coverage can have its width smaller than max_blocksize_x AND its height smaller than max_blocksize_y.

This constraint ensures that all tile are of the same size and hence, do not overlap and do not leave gaps in the coverage. The exceptions for right-most and bottom tiles are to ensure that the coverage conserve the same extent as the original loaded raster. Reading applications have to check for the width and height of rasters (or tiles) before blindly copy their data into buffers having their sizes determined by blocksize_x and blocksize_y.

Some rasters (or tiles) might be missing in a raster coverage (or table). Applications should assume such left over spaces are equivalent to nodata values.

The major objective of this regular_blocking definition is to allow some applications to optimize reading of a coverage by predefining a dumping buffer size and blindly copy raster data in it (after a fast check on width and height which can be smaller for right-most and bottom tiles). 

-----------------------------------------------------------
I think there is no need any more to ensure that "All tiles must be non-overlapping" or "There must be no gap" since both are ensured by the conjunction of the three conditions. Any counter example?

This definition remove the constraint over rotated extents and is hence more general. Is there any reason why a regularly blocked coverage should not be rotated? A rotated coverage is just a matter of raster georeference right? A source raster should be loaded tiled as a set of rotated tiles and exported unioned as a big rotated raster no?

Let me know of any inconsistency of syntax error. I wish the definition would end up in this part of the doc:

http://postgis.refractions.net/documentation/manual-svn/using_raster_dataman.html#RT_Loading_Rasters

Pierre

> -----Original Message-----
> From: postgis-devel-bounces at lists.osgeo.org [mailto:postgis-devel-
> bounces at lists.osgeo.org] On Behalf Of Mateusz Loskot
> Sent: Thursday, December 06, 2012 1:20 PM
> To: PostGIS Development Discussion
> Subject: Re: [postgis-devel] Regular raster blocking (was: Real extent feature
> lost after metadata as views)
> 
> On 6 December 2012 16:40, Pierre Racine <Pierre.Racine at sbf.ulaval.ca> wrote:
> >> > That discussion is still open:
> >> >
> >> > Mateusz :
> >> >>Sandro :
> >> >>>Mateusz :
> >> >>>> Sandro :
> >> >>>> > No, I think it should be allowed everywhere
> >> >>>>
> >> >>>> Then, we no longer discuss blocking in a regular grid.
> >> >>>
> >> >>> It'd be a regular-constrained grid
> >> >>
> >> >> I don't understand what you mean.
> >> >> Constraint how?
> >> >
> >> > These are, I belive, the assumptions a reader might want to
> >> > make once block dimensions are defined:
> >> >
> >> >  - Every tile is in exactly _one_ block
> >> >  - No block contains more than _one_ tile
> >> >
> >> > Need anything more ?
> >
> > These two are already taken care by other existing constraints:
> >
> > -"Every tile is in exactly _one_ block" is implied by the fact that each tile has its
> UL corner on the block grid AND no tile can be larger than max_blocksize_x AND
> no tile can be taller than  max_blocksize_y. Actually I propose to make it even
> more restrictive. See below.
> >
> > -"No block contains more than _one_ tile" is implied by the fact that "All tiles
> must be non-overlapping"
> 
> I agree.
> 
> >> Pierre's suggestion, which I described max_blocksize_x, seems to be
> reasonable
> >> and intuitive specification yet doesn't leave too much room for
> interpretation.
> >> Less room for interpretation should also work well for better
> interoperability..
> >
> > I would actually make it more restrictive removing the possibility to accept
> smaller tiles anywhere:
> >
> > " All loaded tiles have the same width and height. Right-most tiles can have
> their width smaller than max_blocksize_x and bottom tiles can have their height
> smaller than max_blocksize_y. Only the right-most, bottom tile of a coverage
> can have its width smaller than max_blocksize_x AND its height smaller than
> max_blocksize_y ".
> 
> I agree on that, sounds perfect.
> 
> > Why not accept smaller tiles anywhere? Because then the concept of regularly
> blocked becomes so loose that it gradually lose most of its meaning.
> 
> That's right.
> 
> > Where is the last effective definition of "regularly tiled" so we build a
> "definitive", unambiguous definition. Is it the one in
> http://trac.osgeo.org/postgis/wiki/WKTRaster/SpecificationFinal01 ?
> 
> I think that is the one we agreed on after first round of discussions.
> Then, post-beta and pre-release changes resulted in this
> http://trac.osgeo.org/postgis/wiki/WKTRaster/SpecificationWorking02
> but I'm not sure how up to date is this one.
> For example, it says "the constraint is purely information", where
> in the current manual for PostGIS 2.0 we have included plans for real
> constraint in future.
> 
> > I must say that I'm more open to the concept of regularly blocking now that it
> is in an automatically generated constraint-based view. Hopefully we are fixing
> the last inconsistency.
> 
> I'm glad about the outcome too.
> 
> > To answer the question: "What about the constraints planned for the regular
> blocking mode? Are we going to keep this plan up?"
> > I would not see the point of removing it if we spend so much effort to define it
> if this was your question.
> 
> Yes, I agree. That's what I aimed to confirm.
> 
> > If your question is "Are we going to make it a dynamic constraint - or properly
> constraint it?"
> > I guess yes as this will be possible as soon as we have the proper functions.
> 
> And that is what I aimed to confirm too, so both answers answer my
> both questions.
> 
> > Are we going to beat the record for the longest thread?
> 
> At least, we have a chance :)
> 
> Best regards,
> --
> Mateusz Loskot, http://mateusz.loskot.net
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel