[gdal-dev] FW: Don't we have any ideas for GSoC 2017?

[Even Rouault]

On mercredi 1 mars 2017 12:44:29 CET Casper Børgesen (CABO) wrote:
> Yes this is about spatial indexing :)
> 
> Without looking at the actual VRT code I would guess that parsing the VRT
> file is one thing, but having a VRT file with +100,000 tiles 

Let's say at least 100 bytes for each source declaration, 100 000 tiles would be at least 10 MB 
for the VRT. Big, but perhaps not that much, if it is left open in a quite long section

> would require
> GDAL to look through each tile every time a pixel is requested?

Yes, if you do single pixel queries, that might have bad performance. If you do more 
significant extractions, the cost might decrease.

> Even, if
> your idea below could be implemented into the VRT driver to be able to
> build the index file and gdaladdo to build the overview index files, it
> would be a huge step forward.

Building the index file is the job of gdaltindex. Regarding the overviews, you could have 
several variants: either you have a set of overview tiles, or you can also build the overview 
from the full resolution .vrt

> 
> Is it possible to build the index on the fly into memory when opening the
> VRT file? Is it too memory or time consuming?

If one keeps the current syntax with many XML sources, yes on-the-fly indexing would be the 
only reasonable solution. I don't anticipate it would be particularly memory or time 
consuming for a "reasonable" number of tiles (let's say less than 10 millions). There's already 
infrastructure for that in GDAL, port/cpl_quad_tree.h. Used by the OpenFileGDB driver for 
example since it cannot currently use the native spatia index.

> 
> /Casper
> 
> From: gdal-dev [mailto:gdal-dev-bounces at lists.osgeo.org] On Behalf Of Even
> Rouault Sent: 1. marts 2017 13:25
> To: gdal-dev at lists.osgeo.org
> Cc: Rahkonen Jukka (MML)
> Subject: Re: [gdal-dev] FW: Don't we have any ideas for GSoC 2017?
> 
> On mercredi 1 mars 2017 11:43:12 CET Rahkonen Jukka (MML) wrote:
> > Forwarding to the list because Jordan's trial was not successful.
> 
> Sounds a good idea. I think potential students should be more pro-active in
> proposing and developing their own ideas.
> 
> 
> 
> Regarding "indexing on VRT files", I guess this is about spatial indexing of
> sources in a VRT file. My bet would be that the size of VRT files that
> would need such indexing would be so large than their parsing would be the
> most limiting factor. But I can be wrong.
> 
> Some people who want to make a mosaic from a huge number of tiles do that by
> making VRTs of VRTs of VRTs with a pyramid structure to avoid parsing a lot
> of XML and still having good performance.
> 
> Another option that has been mentionned in the past would be to have a
> variation of the VRT syntax that would reference a tile index (shapefile or
> any OGR datasource) and would thus benefit from its spatial indexing &
> filtering capabilities
> 
> 
> 
> Something like:
> 
> 
> 
> <VRTDataset rasterXSize="20000" rasterYSize="20000">
> 
> <SRS>EPSG:26711</SRS>
> 
> <GeoTransform> 4.4072000000000000e+05, 6.0000000000000000e+01,
> 0.0000000000000000e+00, 3.7513200000000000e+06, 0.0000000000000000e+00,
> -6.0000000000000000e+01</GeoTransform>
> 
> <TileIndex>
> 
> <Filename relativeToVRT="1">index.shp</Filename>
> 
> <!-- <Layername>index</Layername> -->
> 
> </Tileindex>
> 
> <Overview rasterXSize="10000" rasterYSize="10000">
> 
> <TileIndex>
> 
> <Filename relativeToVRT="1">index_ovr_2.shp</Filename>
> 
> </Tileindex>
> 
> </Overview>
> 
> <Overview rasterXSize="5000" rasterYSize="5000">
> 
> <TileIndex>
> 
> <Filename relativeToVRT="1">index_ovr_4.shp</Filename>
> 
> </Tileindex>
> 
> </Overview>
> 
> <VRTRasterBand dataType="Byte" band="1">
> 
> <ColorInterp>Red</ColorInterp>
> 
> </VRTRasterBand>
> 
> <VRTRasterBand dataType="Byte" band="2">
> 
> <ColorInterp>Green</ColorInterp>
> 
> </VRTRasterBand>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/gdal-dev/attachments/20170301/e4e5106b/attachment-0001.html>