[mapserver-users] Storing GeoTIFFs in Amazon S3 for use with Mapserver?

[Robert Sanson]

Thanks Frank.

Do you have an example of a mapfile layer section using GDAL with vsicurl to retrieve GeoTIFFs? 

In my case I use a tileindex to access a directory of GeoTIFFs but we want to move them into S3. Here is a layer from my mapfile:

LAYER
  NAME sthl-075m-rural-2005-11
  GROUP 'highres'
  TYPE RASTER
  STATUS ON
  TILEINDEX ../../data/nz/sthl-075m-rural-2005-11_ind
  TILEITEM Location
  MAXSCALE 100000
  METADATA
   'ows_title' 'sthl-075m-rural-2005-11'
   'wms_opaque' '1'
   'wms_group_title' 'highres'
   'wms_resx' '0.75'
   'wms_resy' '0.75'
   'ows_extent' '1088800.0 4743600.0 1314400.0 5089200.0'
  END
 END

Thanks,

Robert

>>> Frank Warmerdam <warmerdam at pobox.com> 5/11/2014 1:45 p.m. >>>
Robert,

I've done some variations on this.

I'd encourage you to watch a video by Mark Korver on this topic from
FOSS4G: http://vimeo.com/106833852 

I used his NAIP bucket in "requestor-pays" mode successfully.  One thing
that I ended up doing when reading the metadata was to use the vsicurl
driver to access the tiffs, so only the required portions were fetched.
His example actually mounts the whole bucket as a file system in which case
things are much easier.

I will note that when using vsicurl to access GeoTIFFs in S3 I found I had
to set the following configuration variables to get it working.

    # Various /vsicurl hackery to get it working with signed aws urls well
    gdal.SetConfigOption('CPL_VSIL_CURL_ALLOWED_EXTENSIONS', 'ter')
    gdal.SetConfigOption('GDAL_DISABLE_READDIR_ON_OPEN', 'YES')
    gdal.SetConfigOption('CPL_VSIL_CURL_USE_HEAD', 'NO')

I do quite a bit of stuff here with MapServer from S3, but I usually fetch
from S3 and store in /tmp for serving so by the time MapServer sees the
file it is completely normal.

Best regards,
Frank


On Tue, Nov 4, 2014 at 2:56 PM, Robert Sanson <
Robert.Sanson at asurequality.com> wrote:

> Does anyone have experience with using Amazon S3 to store GeoTIffs for use
> as a back-end datastore accessible by Mapserver in order to create a WMS
> service?
>
> Thanks,
>
> Robert Sanson
>
>
> Robert Sanson Veterinary Epidemiologist
> Batchelar Centre | Tennent Drive | Palmerston North
> PO Box 585 | Palmerston North 4440 | New Zealand
> p. +64 6 351 7990 | m. +64 21 448 472 | f. +64 6 351 7919
> e. robert.sanson at asurequality.com 
>
> Global experts in food safety and quality | www.asurequality.com 
>
>
>
>
> This email and any attachments are confidential and intended solely for
> the addressee(s). If you are not the intended recipient, please notify us
> immediately and then delete this email from your system.
>
> This message has been scanned for Malware and Viruses by Websense Hosted
> Security.
> www.websense.com 
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org 
> http://lists.osgeo.org/mailman/listinfo/mapserver-users 
>



-- 
---------------------------------------+--------------------------------------
I set the clouds in motion - turn up   | Frank Warmerdam,
warmerdam at pobox.com 
light and sound - activate the windows | http://pobox.com/~warmerdam 
and watch the world go round - Rush    | Geospatial Software Developer