[gdal-dev] accessing zip files by URL

[Even Rouault]

Le jeudi 24 mars 2011 19:03:57, Joaquim Luis a écrit :
> Even, Chaitanya
> 
> Thanks for hint.
> 
> I did read the http://trac.osgeo.org/gdal/wiki/UserDocs/ReadInZip but it
> hasn't any mention to the need of using "vsicurl"

Yes I am aware that it is lightly documented. I've added recently a few 
pointers though :

- in the GDALOpen() doc : 
http://gdal.org/gdal_8h.html#e97be045eb4701183ad332ffce29745b

which points to :
http://gdal.org/cpl__vsi_8h.html#4f791960f2d86713d16e99e9c0c36258

Which mentions :

"This special file handler can be combined with other virtual filesystems 
handlers, such as /vsizip. For example, 
/vsizip//vsicurl/path/to/remote/file.zip/path/inside/zip"

Perhaps a more didactic page would be needed in the wiki. Any taker :-) ? I 
remind that the wiki is open in writing for everyone with an osgeo id...

> 
> I was about to do the tests you did, so thank you also for that.
> The point here is not to read that file in particular. I just wanted to
> use one as example and one that is not a *.tar.gz that the docs say it
> won't work.

Ah... well actually reading in a .tar or a .tar.gz is now supported ( 
http://gdal.org/cpl__vsi_8h.html#d6dd983338849e7da4eaa88f6458ab64 ). Seems 
that I have changed my mind since. But seeking will still be very slow of 
course (especially if you combine with /vsicurl !). Ok, I'm going to rectify 
the page about that point.

> Ok, I have enough information to make it possible to read some
> compressed files sitting somewhere in the web and have the data land
> directly in the internals of GMT.

> 
> Thanks
> 
> Joaquim
> 
> > Joaquim,
> > 
> > The correct syntax would be :
> > 
> > gdalinfo
> > /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w02
> > 0n90.dem.zip/W020N90.DEM
> > 
> > or just :
> > 
> > gdalinfo
> > /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w02
> > 0n90.dem.zip
> > 
> > because the zip file only contains one single file (W020N90.DEM)
> > 
> > ... But the W020N90.DEM inside the zip file isn't directly recognized by
> > GDAL (even if you download it and unzip the file). It's just a RAW file,
> > that neads an header to tell the dimension, georeferencing, datatype
> > etc, so the above won't directly work.
> > 
> > You can for example create a VRT that refers to the raw file :
> > 
> > <VRTDataset rasterXSize="4800" rasterYSize="6000">
> > 
> >    <GeoTransform>-20, 8.3333333333300008e-03,  0,  90,  0,
> > 
> > -8.3333333333300008e-03</GeoTransform>
> > 
> >    <VRTRasterBand dataType="Int16" band="1" subClass="VRTRawRasterBand">
> >    
> >      <NoDataValue>-9999</NoDataValue>
> >      <SourceFilename
> > 
> > relativetoVRT="0">/vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/
> > SRTM30/w020n90/w020n90.dem.zip</SourceFilename>
> > 
> >      <ByteOrder>MSB</ByteOrder>
> >    
> >    </VRTRasterBand>
> > 
> > </VRTDataset>
> > 
> > I've deduced this VRT from the
> > http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.hdr.zip
> > file that sits next to the .dem.zip file.
> > 
> > It's a shame that they didn't put the .hdr and the .dem file inside the
> > same zip. It would have they been possible to open it directly...
> > 
> > With the SRTM3 in HGT format, you can directly do :
> > 
> > gdalinfo
> > /vsizip/vsicurl/http://dds.cr.usgs.gov/srtm/version2_1/SRTM3/Africa/N00E0
> > 06.hgt.zip
> > 
> > Best regards,
> > 
> > Even
> > 
> >> Hi,
> >> 
> >> How do we access to a compressed files by URL? We can do that, can't we?
> >> 
> >> As an example I try this (on Windows)
> >> 
> >> gdalinfo
> >> /vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90
> >> .de m.zip ERROR 4:
> >> `/vsizip/C:\http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n9
> >> 0.d em.zip' does not exist in the file system,
> >> and is not recognised as a supported dataset name.
> >> 
> >> and this
> >> 
> >> gdalinfo
> >> /vsizip//http://dds.cr.usgs.gov/srtm/version2_1/SRTM30/w020n90/w020n90.d
> >> em. zip
> >> 
> >> but the error message is similar.
> >> 
> >> The idea is to fetch the data directly into GMT, but first I need to
> >> understand well how it works.
> >> 
> >> Thanks
> >> 
> >> Joaquim Luis
> >> _______________________________________________
> >> gdal-dev mailing list
> >> gdal-dev at lists.osgeo.org
> >> http://lists.osgeo.org/mailman/listinfo/gdal-dev