[mapserver-users] Best integration with dynamic map information

[Mark Korver]

In order to serve a lot of requests you generally need to cache. That would
be true of any query against a db, or against a shapefile. The point is,
you don't want to go to db or to disk.
In order make this happen for maps the usual pattern is to serve tiles. If
your data gets updated every 5 minutes, and in your case it is the whole
thing, not just some part. Then you need to make sure no map tile lives
longer than 5 minutes, actually less than 5 minutes.
One simple way to solve for this is to use a Content Delivery Network.
something like this:
http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html
You would point the CDN at your TMS or WMTS server. which would never have
to make the same tile more than once every few minutes.
Another related point would be to put the tile server behind a load
balancer, then point the CDN at the load balancer.
on Amazon that would be an ELB.
-Mark

On Tue, Mar 17, 2015 at 1:41 PM, Lime, Steve D (MNIT) <
Steve.Lime at state.mn.us> wrote:

>  MapCache would be a **much** better option than using MapServer in tile
> mode. You still need something to render tiles (e.g. MapServer) but
> MapCache manages the tiles and is the interface between your client
> application and the tiled data. The brute force method would be replace the
> data and then blow away the cache and let MapCache (and MapServer) rebuild
> as users request tiles. The first request will result in MapCache making a
> call to MapServer to generate the tile/metatile, but subsequent requests
> for tiles will use the cached tiles.
>
>
>
> Others on the list may well have more experience and better ideas.
>
>
>
> Steve
>
>
>
> *From:* Ahmed Aboulenein [mailto:aboulenein.ahmed at gmail.com]
> *Sent:* Tuesday, March 17, 2015 2:54 PM
> *To:* Lime, Steve D (MNIT)
> *Cc:* mapserver-users at lists.osgeo.org
> *Subject:* Re: [mapserver-users] Best integration with dynamic map
> information
>
>
>
> Hi Steve,
>
> Thank you so much for the informative reply, much appreciated!
>
> It's actually a new dataset copy every 5 minutes, it seems like replacing
> a shape file would be most efficient.
>
> We are not using MapCache, do you think it would be a good option for our
> use case with this frequent updates?
>
> Thanks again, Steve!
>
> Best regards,
>
> Ahmed
>
>
>
> On Tue, Mar 17, 2015 at 8:03 PM, Lime, Steve D (MNIT) <
> Steve.Lime at state.mn.us> wrote:
>
> I’ll let others weigh in on throughput. Seems like that should be a matter
> of resources and how much hardware you can throw at the problem. Are you
> using MapCache to serve tiles? MapServer itself is good for rendering tiles
> or more likely, metatiles.
>
>
>
> On thread safety MapServer itself isn’t doing any file locking. Presumably
> the OS or database is handling contention issues. I’d think the right way
> to go for data storage depends on your update process itself. If you want
> to incrementally tweak lines as new information becomes available then a
> database backend seems more reasonable – I’d think you could detect change
> easier and perhaps only retile very specific areas as a result. If instead
> you’re getting a new copy of your entire dataset every 5 minutes then
> perhaps a shapefile would work best – just replace the file and trigger
> retiling.
>
>
>
> Steve
>
>
>
> *From:* mapserver-users-bounces at lists.osgeo.org [mailto:
> mapserver-users-bounces at lists.osgeo.org] *On Behalf Of *Ahmed Aboulenein
> *Sent:* Tuesday, March 17, 2015 9:19 AM
> *To:* mapserver-users at lists.osgeo.org
> *Subject:* [mapserver-users] Best integration with dynamic map information
>
>
>
> Hi ALL,
>
> We serve traffic lines that gets updated every 5 minutes from our feeds.
> We would like to use Map Server to generate transparent tiles showing
> colored polylines of the traffic.
>
> I am newbie to Map Server, I got the server and workshop running and also
> I think I have some good understanding of the map file and layers concept.
> I have a question about integrating our traffic information as a data
> source to the MapServer.
>
> Important aspects for us:
>
> 1. Performance: since we are updating large areas every 5 minutes. We will
> need throughput like 10K images per second. Does this sound okay?
>
> 2. Threads safety: my concern what happens if while updating the source,
> the MapServer is actually rendering an image from old data. Any locks?
>
> I understand our options are:
>
> 1. Data files: such as shape files or KML files.
>
> 2. Database: PostGIS for example.
>
> 3. Service that pulls data: WMS server - I didn't dig much here.
>
> Thanks in advance for your help, some pointers or information to go
> further.
>
> Best regards,
>
> Ahmed
>
>
>
> _______________________________________________
> mapserver-users mailing list
> mapserver-users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20150317/f985a680/attachment-0001.html>