[postgis-devel] GSoC idea: in-situ access for rasters

[Vladimir Kikhtenko]

Ups, I've completely missed this feature of raster columns. Good
lesson for me to better read manuals first.
>From today's point of view it seems that FDW is useful only to support
satellite swath geolocation, where pixels can overlap each other due
to bow-tie effect. It is not clear for me how this can be supported in
GDAL.

Generally I'm interested in this type of geolocation and overall query
performance enhancements, but I can't propose some definite project
right now.

--
Vladimir


2013/4/17 Pierre Racine <Pierre.Racine at sbf.ulaval.ca>:
> Vladimir,
>
> Could you speak more about the pros and cons of the Foreign Data Wrappers approach VS the out-db storage which is already available for PostGIS rasters?
>
> If there are not so many pros, would you be willing to propose or work on another idea?
>
> Pierre
>
>> -----Original Message-----
>> From: postgis-devel-bounces at lists.osgeo.org [mailto:postgis-devel-
>> bounces at lists.osgeo.org] On Behalf Of Vladimir Kikhtenko
>> Sent: Thursday, April 11, 2013 4:23 AM
>> To: postgis-devel at lists.osgeo.org
>> Subject: [postgis-devel] GSoC idea: in-situ access for rasters
>>
>> Hello, list!
>>
>> I want to present my idea for GSoC'13 project. It is about interacting
>> with raster data without preloading it in the database first.
>> Using Foreign Data Wrappers (quite new Postgres 9 feature) we can
>> create foreign table for some raster file that will read data directly
>> from the file, when the query is issued. I suppose that this foreign
>> table will contain rows for each pixel in the source file and columns
>> for each dataset in it. Also, some columns will represent the
>> geolocation of data. For example table can look like:
>>
>> CREATE FOREIGN TABLE foreign_raster (
>>     xpos int4, -- this two columns describe pixel's position in raster
>>     ypos int4,
>>     footprint geometry OPTIONS (type 'footprint'), -- geolocation of a pixel
>>     layer1 float8 OPTIONS (sds 'Atmospheric Optical Depth'),
>>     layer2 int2 OPTIONS (sds 'Atmospheric Optical Depth Model', type 'byte'),
>> ) SERVER hvault_service
>>   OPTIONS (filename '/some/path/some-file.hdf');
>>
>> Even more, we can create a catalog of files, so foreign table will
>> contain pixels from each of them. This approach can be useful when you
>> have large archive of raster data you want to access like it were in
>> database, but do not want to actually create copy of it due to disk
>> usage constraints. Another option is to store timestamps for files in
>> catalog, so we can request time-series for some point or region of
>> interest.
>>
>> Also FDW API gives possibility to analyze the query tree before
>> execution, which leads to additional optimizations. We can examine
>> query quals and filter catalog, so we will only read files that
>> contribute to the query result. For example, if we store the footprint
>> of whole raster in catalog, and the query contains clause like
>> ST_Contains(footprint, ST_GeometryFromText('POINT( 43.19 64.90)')), we
>> can process only files that contain that point in their footprint.
>>
>> I've already implemented this idea for HDF files as a part of my MSc
>> thesis that will defend in June. See
>> https://github.com/kikht/fdb/tree/master/hvault (sorry, the code is
>> not very well commented, yet). We are using it in our lab to access
>> 100Tb archive of MODIS images. I think it is possible to implement
>> access to files through GDAL, so many file formats would be supported
>> in one move.
>>
>> Do you interested in such project? Any comments or questions are
>> highly appreciated.
>>
>> --
>> Vladimir Kikhtenko
>> Novosibirsk State University, Russia
>> _______________________________________________
>> postgis-devel mailing list
>> postgis-devel at lists.osgeo.org
>> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel
> _______________________________________________
> postgis-devel mailing list
> postgis-devel at lists.osgeo.org
> http://lists.osgeo.org/cgi-bin/mailman/listinfo/postgis-devel