[mapserver-users] Refreshing AWS credentials for the /vsis3/ driver

[Even Rouault]

Le jeudi 27 octobre 2016 18:33:01, Peter Schmitt a écrit :
> On Wed, Oct 26, 2016 at 1:39 PM, Even Rouault <even.rouault at spatialys.com>
> 
> wrote:
> > I've written recently a best practice for generating cloud optimized
> > geotiffs:
> > https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF
> > (it mentions deflate compression, but jpeg-compressed can work of course)
> 
> Thanks again, Even! I did some experimentation with a 19584x19584 file with
> 
> 6 overview levels.  When the image is deflate-compressed:
> > gdal_translate in.tif deflate.tif -co TILED=YES -co COMPRESS=DEFLATE -co
> 
> COPY_SRC_OVERVIEWS=YES
> 
> then this gdalinfo command does two range requests:
> > env AWS_ACCESS_KEY_ID=foo
> 
> AWS_SECRET_ACCESS_KEY=bar CPL_VSIL_CURL_ALLOWED_EXTENSIONS=.tif
> CPL_CURL_VERBOSE=YES
> VSI_CACHE=TRUE gdalinfo /vsis3/pschmitt-test/deflate.tif 2>&1 | grep
> "Content-Range:"
> 
> >  < Content-Range: bytes 0-49151/1038305419
> >  < Content-Range: bytes 49152-147455/1038305419
> 
> When I jpeg-compress
> 
> > gdal_translate deflate.tif jpeg.tif -co TILED=YES -co COMPRESS=JPEG -co
> 
> PHOTOMETRIC=YCBCR -co COPY_SRC_OVERVIEWS=YES
> 
> then gdalinfo on the jpeg-compressed image does 13 range requests:
> > env AWS_ACCESS_KEY_ID=foo
> 
> AWS_SECRET_ACCESS_KEY=bar CPL_VSIL_CURL_ALLOWED_EXTENSIONS=.tif
> CPL_CURL_VERBOSE=YES
> VSI_CACHE=TRUE gdalinfo /vsis3/pschmitt-test/jpeg.tif 2>&1 | grep
> "Content-Range:"
> 
> > < Content-Range: bytes 0-49151/83731710
> > < Content-Range: bytes 83656704-83705855/83731710
> > < Content-Range: bytes 83722240-83731709/83731710
> > < Content-Range: bytes 83705856-83722239/83731710
> > < Content-Range: bytes 25231360-25280511/83731710
> > < Content-Range: bytes 25165824-25214975/83731710
> > < Content-Range: bytes 25214976-25231359/83731710
> > < Content-Range: bytes 7602176-7651327/83731710
> > < Content-Range: bytes 2228224-2310143/83731710
> > < Content-Range: bytes 655360-704511/83731710
> > < Content-Range: bytes 229376-278527/83731710
> > < Content-Range: bytes 131072-180223/83731710
> > < Content-Range: bytes 98304-131071/83731710

Could you post the outpout of "tiffdump jpeg.tif" ? There are some hacks in 
libtiff regarding JPEG compression and in particular JPEG tables that might 
perhaps cause spurious IFD rewriting.

> 
> and 16 range requests if it also has an internal mask band.  It seems JPEG
> compression greatly increases the size of the image metadata.
> 
> Cheers,
> Pete

-- 
Spatialys - Geospatial professional services
http://www.spatialys.com