<div dir="ltr"><div>Hi all,</div><div><br></div><div>I have a GeoTIFF in S3 and am trying to access it via a VRT. I'm constructing the VRT in Python and then passing the string directly to GDAL like</div><div><br></div><div>    GDALOpenEx("<VRTDataset>...</VRTDataset>", ...)</div><div><br></div><div>I'm getting the results I expect, but very slowly. In my logs, it seems that GDAL is fetching the entire GeoTIFF and copying it to /vsimem/ before reading any data.</div><div><br></div><div><div>DEBUG:rasterio._base:Entering Dataset <open DatasetBase name='<VRTDataset rasterXSize="48949" rasterYSize="15633"><SRS>PROJCS["WGS 84 / UTM zone 46N",GEOGCS["WGS 84",DATUM["WGS_1984",SPHEROID["WGS 84",6378137,298.257223563,AUTHORITY["EPSG","7030"]],AUTHORITY["EPSG","6326"]],PRIMEM["Greenwich",0,AUTHORITY["EPSG","8901"]],UNIT["degree",0.0174532925199433,AUTHORITY["EPSG","9122"]],AUTHORITY["EPSG","4326"]],PROJECTION["Transverse_Mercator"],PARAMETER["latitude_of_origin",0],PARAMETER["central_meridian",93],PARAMETER["scale_factor",0.9996],PARAMETER["false_easting",500000],PARAMETER["false_northing",0],UNIT["metre",1,AUTHORITY["EPSG","9001"]],AXIS["Easting",EAST],AXIS["Northing",NORTH],AUTHORITY["EPSG","32646"]]</SRS><GeoTransform>242785.74302396187,0.018557199999999992,0.0,2634995.766347841,0.0,-0.018557199999991995</GeoTransform><VRTRasterBand band="1" dataType="Byte"><ColorInterp>Red</ColorInterp><ComplexSource><SourceFilename relativeToVRT="0"><a href="http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif">http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif</a></SourceFilename><SourceBand>1</SourceBand><SourceProperties BlockXSize="512" BlockYSize="512" RasterXSize="48949" RasterYSize="15633" dataType="Byte" /><SrcRect xOff="0" xSize="48948" yOff="0" ySize="15632" /><DstRect xOff="0.0" xSize="48948" yOff="-0.0" ySize="15632" /></ComplexSource></VRTRasterBand><VRTRasterBand band="2" dataType="Byte"><ColorInterp>Green</ColorInterp><ComplexSource><SourceFilename relativeToVRT="0"><a href="http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif">http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif</a></SourceFilename><SourceBand>2</SourceBand><SourceProperties BlockXSize="512" BlockYSize="512" RasterXSize="48949" RasterYSize="15633" dataType="Byte" /><SrcRect xOff="0" xSize="48948" yOff="0" ySize="15632" /><DstRect xOff="0.0" xSize="48948" yOff="-0.0" ySize="15632" /></ComplexSource></VRTRasterBand><VRTRasterBand band="3" dataType="Byte"><ColorInterp>Blue</ColorInterp><ComplexSource><SourceFilename relativeToVRT="0"><a href="http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif">http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif</a></SourceFilename><SourceBand>3</SourceBand><SourceProperties BlockXSize="512" BlockYSize="512" RasterXSize="48949" RasterYSize="15633" dataType="Byte" /><SrcRect xOff="0" xSize="48948" yOff="0" ySize="15632" /><DstRect xOff="0.0" xSize="48948" yOff="-0.0" ySize="15632" /></ComplexSource></VRTRasterBand><MaskBand><VRTRasterBand dataType="Byte"><SimpleSource><SourceFilename relativeToVRT="0"><a href="http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif">http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif</a></SourceFilename><OpenOptions><OOI key="OVERVIEW_LEVEL">AUTO</OOI></OpenOptions><SourceBand>mask,1</SourceBand><SourceProperties BlockXSize="512" BlockYSize="512" RasterXSize="48949" RasterYSize="15633" dataType="Byte" /><SrcRect xOff="0" xSize="48948" yOff="0" ySize="15632" /><DstRect xOff="0.0" xSize="48948" yOff="-0.0" ySize="15632" /></SimpleSource></VRTRasterBand></MaskBand></VRTDataset>' mode='r'> context.</div><div>DEBUG:rasterio._io:IO window xoff=0.0 yoff=0.0 width=48948.0 height=15632.0</div><div>DEBUG:rasterio._gdal:CPLE_None in HTTP: Fetch(<a href="http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif">http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif</a>)</div><div>GDAL: GDALOpen(/vsimem/http_1/3fa181a1-425c-4e53-b49f-ea72884da097.tif, this=0x101e9f4c0) succeeds as GTiff.</div><div>DEBUG:rasterio._gdal:CPLE_None in GDAL: GDALOpen(<a href="http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif">http://oin-hotosm.s3.amazonaws.com/5a324a622553e6000ce5aaf5/0/3fa181a1-425c-4e53-b49f-ea72884da097.tif</a>, this=0x101e9f4c0) succeeds as HTTP.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: ScanDirectories()</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 24474x7816 overview.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 12237x3908 overview.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 6119x1954 overview.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 3060x977 overview.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 1530x489 overview.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 765x245 overview.</div><div>DEBUG:rasterio._gdal:CPLE_None in GTiff: Opened 383x123 overview.</div></div><div><br></div><div>Once the entire GeoTIFF has been download, the overviews are accessed as I'd expect, but too late to be useful.</div><div><br></div><div>Is there a way to bypass creation of the /vsimem/ copy of the source dataset when passing a VRT XML doc like this to GDALOpenEx?</div><div><br></div><div>Thanks in advance,</div><div><br></div>-- <br><div class="gmail_signature"><div dir="ltr">Sean Gillies</div></div>
</div>