[gdal-dev] /vsicurl/ and gz files

[Even Rouault]

Le jeudi 10 octobre 2013 20:33:49, Jose Gomez-Dans a écrit :
> Hi,
> 
> I was wondering if it's possible to use /vsitar/ to have a peek in .gz
> files, such as this one:
> http://www.globalbedo.org/GlobAlbedo29/tiles/2004/h17v04/GlobAlbedo.2004001
> .h17v04.nc.gz

This is a gzip file, not a .tar.gz file, so you would have to use /vsigzip , and 
as it is a gzip file over http, you have to combine it with /vsicurl

The syntax would be :

gdalinfo 
/vsigzip//vsicurl/http://www.globalbedo.org/GlobAlbedo29/tiles/2004/h17v04/GlobAlbedo.2004001.h17v04.nc.gz

But... as netCDF library doesn't use GDAL I/O natively, you would also to use 
the vsipreload mechanism, available on Linux for the development version of 
GDAL. See http://lists.osgeo.org/pipermail/gdal-dev/2013-May/036324.html

export LD_PRELOAD=vsipreload.so
gdalinfo 
/vsigzip//vsicurl/http://www.globalbedo.org/GlobAlbedo29/tiles/2004/h17v04/GlobAlbedo.2004001.h17v04.nc.gz

Technically, this works, but as I've just tested, it will be very inefficient 
since the netCDF library asks, just after having detecting the netCDF 
signature from the first bytes, for the size of the netcdf file, and the only 
way to know the size of a gzipped file is to decompress it until its end, which 
means that /vsicurl will have to download it entirely...

All in all, in that case, you have no choice but downloading the whole file 
locally. The experience would perhaps be better (but I've not verified) if the 
file was in a zip archive, that contains the uncompressed size in the metadata.

Even

-- 
Geospatial professional services
http://even.rouault.free.fr/services.html