[postgis-devel] [WKT Raster] Regular blocking in gdal2wktraster.py

[Mateusz Loskot]

Pierre Racine wrote: 
> Mateusz,
> 
> I can see two major user cases that gdal2wktraster.py needs to fulfill:
> 
> 1) Some users want to store big rasters (overlapping or not, e.g.
> landsat images) as many tables. In this case 1 raster = 1 row in the
> raster_column table and hence many tile tables in the database. (100
> landsat = 100 files = 100 tables) Some people might want to store only
> one file, other might want to batch load many.

Pierre,

Are you suggesting that single entry in RASTER_COLUMNS may describe
a raster scattered in N tables?

Can you suggest any analogous case in PostGIS (GEOMETRY_COLUMNS) ?

> 2) Many users will want to upload coverage already tiled (generally not
> overlapping, e.g. SRTM elevation data) as a single table. In this case
> 1 raster = 1 row in the raster_columns table and only one tile table in
> the database.
> 	a) They might want that to store them tiled as they already are.
> This is already possible with wildcards.

Yes.

> 	b) They might want to retile them (in batch) as smaller tiles
> (similar to case 1 but all in the same table).

In this case, users are supposed to do intermediate processing
using already available tools, like gdal_retile or others.
Then, they are able to use gdal2wktraster.py as in case 2.a)

I don't see any point to include retiling functionality in gdal2wktraster.

> Some people might want to store only one raster as a one row tile
> table. This is identical to scenario 2a but without using wilcards (and
> only one -r option).

OK

> We'll have to make sure all these scenarios are possible. Now only 2a
> is possible.

I can confirm that I agree (or not) if I understand what's the case 1)

> For all scenarios, it should be possible to store in-db or out-db. Out-
> db means the importer should copy the tile files in a determined folder
> (case 1 and b2). In case 2a the folder already exist.

ATM, I'm focused on in-db only. IMHO, it's far to early to even plan out-db
as there is completely no support for out-db on the WKT Raster side.

I'm following principles of prototyping, and I'm not trying to draw a big-ass 
plan and then start developing it. I'm making very small steps,
so I can change direction quickly.

> We can also see a big raster as a single coverage and in this case case
> 1 and 2b are very similar except for the fact that there is only one
> file in case 1 and many in case 2. It might be sometime impossible to
> produce a raster big enough to cover a large extent so that scenario 1
> would not be an option but 2b would.

If you mean 1 big raster tiled into a single coverage as single table with
tile-per-row, then I agree.

> In a best practice documentation section (that I plan to write someday)
> I would recommand to users scenario 2a or 2b since this is the storage
> method that will benefit the most from storing raster in the database.

Honestly, I'm starting to worry about too many use cases we are going to offer.
Simple tool is supposed to be simple to use enough, so it doesn't need a
brick-weight handbook, with zillions of recommendations and exceptions.

My plan about gdal2wktraster is as short as this summary:

1) Load 1..N raster files into a single table, one raster file per row.
It's a user's decision, if he wants to load N unrelated/different/heterogeneous
raster files into single table, or if he wants to keep things in some order.
With current gdal2wktraster user has a liberty and tool to achieve both.

2) Load 1 raster file according to principles of regular blocking we have already drawn.
So, 1 raster is tiled according to block size and tiles are loaded into a single table
(one tile per row). So, the whole table makes a regular grid of non-overlapping tiles.

I believe that all other cases will over-complicate things at this stage of
development of WKT raster, and are of a low priority for me.

Best regards,
--
Mateusz Loskot
Senior Programmer, Cadcorp
http://www.cadcorp.com