<html><head><meta http-equiv="content-type" content="text/html; charset=utf-8"></head><body dir="auto">You can also use something like goofys which is a fuse provider for s3 which provides s3 access while using normal file path access. It’s useful when you have to deploy to cloud and non cloud. <br><br><div id="AppleMailSignature" dir="ltr">Michael Smith</div><div id="AppleMailSignature" dir="ltr">RSGIS Center </div><div dir="ltr"><br>On Aug 30, 2019, at 7:31 PM, Jan Hartmann <<a href="mailto:j.l.h.hartmann@uva.nl">j.l.h.hartmann@uva.nl</a>> wrote:<br><br></div><blockquote type="cite"><div dir="ltr">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
Impressive, looks like viscurl is a real alternative to local files.
Thanks for the information, I'm certainly going to look further on
this.<br>
<br>
Jan<br>
<br>
<div class="moz-cite-prefix">On 8/30/2019 6:15 PM, Peter Schmitt
wrote:<br>
</div>
<blockquote type="cite" cite="mid:CACkcqcWF-dst_Yd7b3BHN+u=sJBEc=WSoizHKkcTEFyDj5ax3w@mail.gmail.com">
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr" class="gmail_attr">On Fri, Aug 30, 2019 at 9:48
AM Jan Hartmann <<a href="mailto:j.l.h.hartmann@uva.nl" moz-do-not-send="true">j.l.h.hartmann@uva.nl</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px
0.8ex;border-left:1px solid
rgb(204,204,204);padding-left:1ex">
<div bgcolor="#FFFFFF"> Thanks Peter, this is really useful.
Do you have any real-world benchmarks for MapServer that
compare regular file access with vsicurl access, using
optimized Geotifs? I've seen the tests for GDAL at <a href="https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF" target="_blank" moz-do-not-send="true">https://trac.osgeo.org/gdal/wiki/CloudOptimizedGeoTIFF</a>,
but what about servering large map-sets over the web?<br>
<br>
</div>
</blockquote>
<div>I don't have any sort of formal benchmarks... but I can
provide anecdotes from my experiences. Random access to a
256x256 block of extremely large COGs served from fast SSD
would have maybe 100ms response. The same COG accessed with
/vsicurl/: First time the tile is requested about 400ms.
GDAL has a least recently used curl cache. When a tile is
cached, subsequent requests can be served maybe about 100ms.
My cloud provider is AWS... so access from s3 using
MapServer running in a Docker container on an AWS EC2
instance in the us-east-1 region... and the tile is
requested from my local wireless network in Colorado.</div>
<div><br>
</div>
<div>When the data is a COG in s3 and MapServer runs near the
cloud storage, performance is quite good. I've scaled only
to dozens of users. Allegedly s3 can scale to 5,500
requests per second per prefix in s3. <a href="https://docs.aws.amazon.com/AmazonS3/latest/dev/optimizing-performance.html" moz-do-not-send="true">https://docs.aws.amazon.com/AmazonS3/latest/dev/optimizing-performance.html</a> Presumably
it would scale fairly well by adding more MapServer
processes.</div>
<div> </div>
</div>
</div>
</blockquote>
<br>
</div></blockquote><blockquote type="cite"><div dir="ltr"><span>_______________________________________________</span><br><span>mapserver-users mailing list</span><br><span><a href="mailto:mapserver-users@lists.osgeo.org">mapserver-users@lists.osgeo.org</a></span><br><span><a href="https://lists.osgeo.org/mailman/listinfo/mapserver-users">https://lists.osgeo.org/mailman/listinfo/mapserver-users</a></span></div></blockquote></body></html>