<div dir="ltr"><div><div>Thank you so much, Mark! That's a very convenient/effiecient architecture for us since our engine is already in AWS. Thanks again for your help!<br><br></div>Best regards,<br></div>Ahmed<br></div><div class="gmail_extra"><br><div class="gmail_quote">On Tue, Mar 17, 2015 at 11:07 PM, Mark Korver <span dir="ltr"><<a href="mailto:mwkorver@gmail.com" target="_blank">mwkorver@gmail.com</a>></span> wrote:<br><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div dir="ltr"><div><div><div><div><div><div><div>In order to serve a lot of requests you generally need to cache. That would be true of any query against a db, or against a shapefile. The point is, you don't want to go to db or to disk.<br></div>In order make this happen for maps the usual pattern is to serve tiles. If your data gets updated every 5 minutes, and in your case it is the whole thing, not just some part. Then you need to make sure no map tile lives longer than 5 minutes, actually less than 5 minutes.<br></div>One simple way to solve for this is to use a Content Delivery Network. <br></div>something like this:<br><a href="http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html" target="_blank">http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html</a><br></div>You would point the CDN at your TMS or WMTS server. which would never have to make the same tile more than once every few minutes.<br></div>Another related point would be to put the tile server behind a load balancer, then point the CDN at the load balancer.<br></div>on Amazon that would be an ELB.<br></div>-Mark<br></div><div class="gmail_extra"><br><div class="gmail_quote"><div><div class="h5">On Tue, Mar 17, 2015 at 1:41 PM, Lime, Steve D (MNIT) <span dir="ltr"><<a href="mailto:Steve.Lime@state.mn.us" target="_blank">Steve.Lime@state.mn.us</a>></span> wrote:<br></div></div><blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><div><div class="h5">
<div link="blue" vlink="purple" lang="EN-US">
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">MapCache would be a *<b>much</b>* better option than using MapServer in tile mode. You still need something to render tiles (e.g. MapServer) but MapCache manages
the tiles and is the interface between your client application and the tiled data. The brute force method would be replace the data and then blow away the cache and let MapCache (and MapServer) rebuild as users request tiles. The first request will result
in MapCache making a call to MapServer to generate the tile/metatile, but subsequent requests for tiles will use the cached tiles.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Others on the list may well have more experience and better ideas.<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Steve<u></u><u></u></span></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"><u></u> <u></u></span></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif""> Ahmed Aboulenein [mailto:<a href="mailto:aboulenein.ahmed@gmail.com" target="_blank">aboulenein.ahmed@gmail.com</a>]
<br>
<b>Sent:</b> Tuesday, March 17, 2015 2:54 PM<br>
<b>To:</b> Lime, Steve D (MNIT)<br>
<b>Cc:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<b>Subject:</b> Re: [mapserver-users] Best integration with dynamic map information<u></u><u></u></span></p><div><div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi Steve,<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Thank you so much for the informative reply, much appreciated!<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">It's actually a new dataset copy every 5 minutes, it seems like replacing a shape file would be most efficient.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">We are not using MapCache, do you think it would be a good option for our use case with this frequent updates?<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Thanks again, Steve!<u></u><u></u></p>
</div>
<p class="MsoNormal">Best regards,<u></u><u></u></p>
</div>
<p class="MsoNormal">Ahmed<u></u><u></u></p>
</div>
<div>
<p class="MsoNormal"><u></u> <u></u></p>
<div>
<p class="MsoNormal">On Tue, Mar 17, 2015 at 8:03 PM, Lime, Steve D (MNIT) <<a href="mailto:Steve.Lime@state.mn.us" target="_blank">Steve.Lime@state.mn.us</a>> wrote:<u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">I’ll let others weigh in on throughput. Seems like that should be a matter of resources and how much
hardware you can throw at the problem. Are you using MapCache to serve tiles? MapServer itself is good for rendering tiles or more likely, metatiles.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">On thread safety MapServer itself isn’t doing any file locking. Presumably the OS or database is
handling contention issues. I’d think the right way to go for data storage depends on your update process itself. If you want to incrementally tweak lines as new information becomes available then a database backend seems more reasonable – I’d think you could
detect change easier and perhaps only retile very specific areas as a result. If instead you’re getting a new copy of your entire dataset every 5 minutes then perhaps a shapefile would work best – just replace the file and trigger retiling.</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d">Steve</span><u></u><u></u></p>
<p class="MsoNormal"><span style="font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1f497d"> </span><u></u><u></u></p>
<p class="MsoNormal"><b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">From:</span></b><span style="font-size:10.0pt;font-family:"Tahoma","sans-serif"">
<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a> [mailto:<a href="mailto:mapserver-users-bounces@lists.osgeo.org" target="_blank">mapserver-users-bounces@lists.osgeo.org</a>]
<b>On Behalf Of </b>Ahmed Aboulenein<br>
<b>Sent:</b> Tuesday, March 17, 2015 9:19 AM<br>
<b>To:</b> <a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<b>Subject:</b> [mapserver-users] Best integration with dynamic map information</span><u></u><u></u></p>
<div>
<div>
<p class="MsoNormal"> <u></u><u></u></p>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Hi ALL,<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">We serve traffic lines that gets updated every 5 minutes from our feeds. We would like to use Map Server to generate transparent tiles showing colored polylines of the traffic.<u></u><u></u></p>
</div>
<p class="MsoNormal">I am newbie to Map Server, I got the server and workshop running and also I think I have some good understanding of the map file and layers concept. I have a question about integrating
our traffic information as a data source to the MapServer.<u></u><u></u></p>
</div>
<p class="MsoNormal">Important aspects for us:<u></u><u></u></p>
</div>
<p class="MsoNormal">1. Performance: since we are updating large areas every 5 minutes. We will need throughput like 10K images per second. Does this sound okay?<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">2. Threads safety: my concern what happens if while updating the source, the MapServer is actually rendering an image from old data. Any locks?<u></u><u></u></p>
</div>
<p class="MsoNormal">I understand our options are:<u></u><u></u></p>
</div>
<p class="MsoNormal">1. Data files: such as shape files or KML files.<u></u><u></u></p>
</div>
<p class="MsoNormal">2. Database: PostGIS for example.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">3. Service that pulls data: WMS server - I didn't dig much here.<u></u><u></u></p>
</div>
<p class="MsoNormal" style="margin-bottom:12.0pt">Thanks in advance for your help, some pointers or information to go further.<u></u><u></u></p>
</div>
<p class="MsoNormal">Best regards,<u></u><u></u></p>
</div>
<p class="MsoNormal">Ahmed<u></u><u></u></p>
</div>
</div>
</div>
</div>
</div>
</div>
<p class="MsoNormal"><u></u> <u></u></p>
</div>
</div></div></div>
</div>
<br></div></div>_______________________________________________<br>
mapserver-users mailing list<br>
<a href="mailto:mapserver-users@lists.osgeo.org" target="_blank">mapserver-users@lists.osgeo.org</a><br>
<a href="http://lists.osgeo.org/mailman/listinfo/mapserver-users" target="_blank">http://lists.osgeo.org/mailman/listinfo/mapserver-users</a><br></blockquote></div><br></div>
</blockquote></div><br></div>