[postgis-devel] [PostGIS] #2122: [raster] Real extent feature lost after metadata as views
Bborie Park
dustymugs at gmail.com
Fri Nov 30 07:07:13 PST 2012
For strk's questions...
> Does the padding acts as nodata always ?
Yes
> does has_nodata return true in presence of padding ?
Yes
> Is index 0 always the one within the padding or includes it ?
What's index 0?
> how does the loaded implement padding ? Does it only pad the lower-right
> tile or evenly distributes leftover pixels among all tiles ?
It pads the right-most and bottom tiles.
For example, let us say you have a 100 x 100 raster and you want to
tile it by 21 x 21...
raster2pgsql -t 21x21 my100x100raster.tif
The loader will create 25 tiles with 9 tiles having padding. When
tiling, the loader processes from the upper-left corner so the padding
occurs on the right and bottom edge of the raster.
As for adding more to the serialized format, I'm hesitant to do so at
this stage. When are we allowed to make changes to the serialized
format? I was under the impression that it had to be a major version
change.
It is on the todo list to be able to prevent tiles from being padded
(so some tiles will have different width and height) but then this
would violate the regular blocking rule of all tiles having the same
width and height.
http://trac.osgeo.org/postgis/wiki/WKTRaster/Documentation01
***
You can inform PostGIS that the raster layer you are loading meets
certain useful criteria by adding the –k option to the
gdal2wktraster.py Python loader. This will set the "regular_blocking"
attribute of the raster_columns table to true. It implies that:
All loaded tiles have the same width and height,
All tiles do not overlap and their upper left corners follow a
regular block grid,
The global extent of the layer is rectangular and not rotated.
Some blocks (or tiles) can be missing in a regularly blocked table.
Missing tiles are assumed to be filled with the proper nodata value
for each band as determined in the raster_columns table.
***
Part of the issue is that when the python loader was in use,
raster_columns was a table and raster columns only had one constraint
(SRID). With raster_columns being a constraint-based view, the extent
value for each table's raster column is based upon the maximum extent
constraint.
-bborie
On Fri, Nov 30, 2012 at 6:22 AM, Sandro Santilli <strk at keybit.net> wrote:
> My 2 cents, not being such a common raster user.
> Such a change would raise many questions as of what to do with
> the padding on any kind of operation. Does the padding acts
> as nodata always ? does has_nodata return true in presence of
> padding ?
> Is index 0 always the one within the padding or includes it ?
>
> A question:
> how does the loaded implement padding ? Does it only pad the lower-right
> tile or evenly distributes leftover pixels among all tiles ?
>
> All of this seems to me as something that should be dealt with
> in upper levels, not value level. Why not just add support in GDAL
> for "trimming" the raster when it only contains NODATA near the margin ?
>
> --strk;
>
>
>
> On Fri, Nov 30, 2012 at 12:53:10PM +0000, Mateusz Loskot wrote:
>> Folks,
>>
>> I've just reported this ticked and I'd appreciate if we could discuss it
>> and come up with a usable solution.
>>
>> Mat
>>
>> On 30 November 2012 12:51, PostGIS <trac at osgeo.org> wrote:
>> > #2122: [raster] Real extent feature lost after metadata as views
>> > ----------------------------------------------------+-----------------------
>> > Reporter: mloskot | Owner: dustymugs
>> > Type: defect | Status: new
>> > Priority: medium | Milestone:
>> > Component: raster | Version: trunk
>> > Keywords: regular blocking,raster,extent,padding |
>> > ----------------------------------------------------+-----------------------
>> > It seems, we've overlooked one feature while migrating from table to view
>> > for the raster_columns (#1216, #1319). It is quite an important feature.
>> >
>> > With the raster_columns as table, it was possible to set real spatial
>> > extent of raster data, regardless of physical size of raster blob.
>> > It means, it was possible to 'cut off' any possible padding
>> > framing the real raster data.
>> >
>> > With the raster_columns as view, spatial extent is calculated based on
>> > 'worldfile' metadata and actual width and height of raster matrix.
>> >
>> > This is an undesired difference, generally, but seems to mostly affect
>> > uses of regular blocking.
>> >
>> > Previously, it was possible to specify raster2pgsql with arbitrary block
>> > size.
>> > If the block size was not divinding the input raster evenly, the loader
>> > would
>> > generate padding along the right and bottom edges of right-most and
>> > bottom-most tiles, but the spatial extent of the whole tiles coverage
>> > written
>> > into raster_columns did not include that padding.
>> > It was extent of real data, ragardless physical pixel dimensions were
>> > increased
>> > by the padding.
>> >
>> > Currently, with the raster_columns being a view, padding affects spatial
>> > extent.
>> >
>> > For example:
>> >
>> > * Input raster: [http://download.osgeo.org/gdal/data/gtiff/ utm.tif] from
>> > GDAL test data:
>> >
>> > {{{
>> > f:\data\Test\pgraster\blocking>gdalinfo utm.tif
>> > Driver: GTiff/GeoTIFF
>> > Files: utm.tif
>> > Size is 512, 512
>> > Coordinate System is:
>> > PROJCS["NAD27 / UTM zone 11N",
>> > GEOGCS["NAD27",
>> > DATUM["North_American_Datum_1927",
>> > SPHEROID["Clarke 1866",6378206.4,294.9786982139006,
>> > AUTHORITY["EPSG","7008"]],
>> > AUTHORITY["EPSG","6267"]],
>> > PRIMEM["Greenwich",0],
>> > UNIT["degree",0.0174532925199433],
>> > AUTHORITY["EPSG","4267"]],
>> > PROJECTION["Transverse_Mercator"],
>> > PARAMETER["latitude_of_origin",0],
>> > PARAMETER["central_meridian",-117],
>> > PARAMETER["scale_factor",0.9996],
>> > PARAMETER["false_easting",500000],
>> > PARAMETER["false_northing",0],
>> > UNIT["metre",1,
>> > AUTHORITY["EPSG","9001"]],
>> > AUTHORITY["EPSG","26711"]]
>> > Origin = (440720.000000000000000,3751320.000000000000000)
>> > Pixel Size = (60.000000000000000,-60.000000000000000)
>> > Metadata:
>> > AREA_OR_POINT=Area
>> > Image Structure Metadata:
>> > INTERLEAVE=BAND
>> > Corner Coordinates:
>> > Upper Left ( 440720.000, 3751320.000) (117d38'28.21"W, 33d54' 8.47"N)
>> > Lower Left ( 440720.000, 3720600.000) (117d38'20.79"W, 33d37'31.04"N)
>> > Upper Right ( 471440.000, 3751320.000) (117d18'32.07"W, 33d54'13.08"N)
>> > Lower Right ( 471440.000, 3720600.000) (117d18'28.50"W, 33d37'35.61"N)
>> > Center ( 456080.000, 3735960.000) (117d28'27.39"W, 33d45'52.46"N)
>> > Band 1 Block=512x16 Type=Byte, ColorInterp=Gray
>> > }}}
>> >
>> > * Block 256x256
>> >
>> > {{{
>> > raster2pgsql -C -r -t 256x256 public.utm256 utm.tif
>> > }}}
>> >
>> > The loader cut the raster evenly, so no padding added, spatial extent the
>> > same as original:
>> >
>> > {{{
>> > test=# SELECT ST_Extent(rast::geometry) FROM utm256;
>> > st_extent
>> > ------------------------------------
>> > BOX(440720 3720600,471440 3751320)
>> > (1 row)
>> > }}}
>> >
>> > * Block 300x300
>> >
>> > {{{
>> > raster2pgsql -C -r -t 300x300 public.utm300 utm.tif
>> > }}}
>> >
>> > The loader added padding, so spatial extent increased rightwards and
>> > downwards:
>> >
>> > {{{
>> > test=# SELECT ST_Extent(rast::geometry) FROM utm300x300;
>> > st_extent
>> > ------------------------------------
>> > BOX(440720 3715320,476720 3751320)
>> > (1 row)
>> > }}}
>> >
>> >
>> > This is a significant change in behaviour, a regression which
>> > I'm reporting as a bug.
>> >
>> > Apparently, PostGIS Raster driver in GDAL 1.9.2 is affected.
>> > I tried to export the coverages from the two tables loaded above, on
>> > Windows:
>> >
>> > {{{
>> > set CONN=PG:dbname=test user=mloskot mode=2
>> > FOR %%t IN (utm256 utm300) DO (
>> > gdal_translate -of GTIff "%CONN% table=%%t" %%t.tif
>> > gdalinfo %%t.tif
>> > )
>> > }}}
>> >
>> > First, similarly to the SQL results above, gdalinfo reports changed
>> > spatial extent for the utm300.tif file.
>> > Second, the padding strips are exported, whereas in normal situation
>> > they should be discarded.
>> > I'm going to attach both files, utm256.tif and utm300.tif:
>> >
>> >
>> > ----
>> >
>> >
>> > I'd like to bring the previous behaviour back.
>> >
>> > I'd like to propose change in the header of serialised format,
>> > and add 4 slots of 16 bytes:
>> >
>> > {{{
>> > struct rt_raster_serialized_t {
>> > /*---[ 8 byte boundary ]---{ */
>> > uint32_t size; /* required by postgresql: 4 bytes */
>> > uint16_t version; /* format version (this is version 0): 2 bytes */
>> > uint16_t numBands; /* Number of bands: 2 bytes */
>> >
>> > /* }---[ 8 byte boundary ]---{ */
>> > double scaleX; /* pixel width: 8 bytes */
>> >
>> > /* }---[ 8 byte boundary ]---{ */
>> > double scaleY; /* pixel height: 8 bytes */
>> >
>> > /* }---[ 8 byte boundary ]---{ */
>> > double ipX; /* insertion point X: 8 bytes */
>> >
>> > /* }---[ 8 byte boundary ]---{ */
>> > double ipY; /* insertion point Y: 8 bytes */
>> >
>> > /* }---[ 8 byte boundary ]---{ */
>> > double skewX; /* skew about the X axis: 8 bytes */
>> >
>> > /* }---[ 8 byte boundary ]---{ */
>> > double skewY; /* skew about the Y axis: 8 bytes */
>> >
>> > /* }---[ 8 byte boundary ]--- */
>> > int32_t srid; /* Spatial reference id: 4 bytes */
>> > uint16_t width; /* pixel columns: 2 bytes */
>> > uint16_t height; /* pixel rows: 2 bytes */
>> >
>> > /* }---[ 8 byte boundary ]--- */
>> > uint16_t upperLeftX; /* first column of raster data, Zero if no
>> > padding leftwards */
>> > uint16_t upperLeftY; /* first row of raster data, Zero if no padding
>> > upwards */
>> > uint16_t lowerRightX; /* last column of raster data, width if no
>> > padding rightwards */
>> > uint16_t lowerRightY; /* last row of raster data, height if no padding
>> > downwards */
>> > };
>> > }}}
>> >
>> > I'm going to post this issue to postgis-devel too and request for
>> > comments.
>> >
>> > --
>> > Ticket URL: <http://trac.osgeo.org/postgis/ticket/2122>
>> > PostGIS <http://trac.osgeo.org/postgis/>
>> > The PostGIS Trac is used for bug, enhancement & task tracking, a user and developer wiki, and a view into the subversion code repository of PostGIS project.
>>
>>
>>
>> --
>> Mateusz Loskot, http://mateusz.loskot.net
>> _______________________________________________
>> postgis-devel mailing list
>> postgis-devel at lists.osgeo.org
>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel
>
> --
>
> http://www.cartodb.com - Map, analyze and build applications with your data
>
> ~~ http://strk.keybit.net
>
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel
More information about the postgis-devel
mailing list