[mapserver-users] Best integration with dynamic map information

[Ahmed Aboulenein]

Thank you so much, Mark! That's a very convenient/effiecient architecture
for us since our engine is already in AWS. Thanks again for your help!

Best regards,
Ahmed

On Tue, Mar 17, 2015 at 11:07 PM, Mark Korver <mwkorver at gmail.com> wrote:

> In order to serve a lot of requests you generally need to cache. That
> would be true of any query against a db, or against a shapefile. The point
> is, you don't want to go to db or to disk.
> In order make this happen for maps the usual pattern is to serve tiles. If
> your data gets updated every 5 minutes, and in your case it is the whole
> thing, not just some part. Then you need to make sure no map tile lives
> longer than 5 minutes, actually less than 5 minutes.
> One simple way to solve for this is to use a Content Delivery Network.
> something like this:
>
> http://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html
> You would point the CDN at your TMS or WMTS server. which would never have
> to make the same tile more than once every few minutes.
> Another related point would be to put the tile server behind a load
> balancer, then point the CDN at the load balancer.
> on Amazon that would be an ELB.
> -Mark
>
> On Tue, Mar 17, 2015 at 1:41 PM, Lime, Steve D (MNIT) <
> Steve.Lime at state.mn.us> wrote:
>
>>  MapCache would be a **much** better option than using MapServer in tile
>> mode. You still need something to render tiles (e.g. MapServer) but
>> MapCache manages the tiles and is the interface between your client
>> application and the tiled data. The brute force method would be replace the
>> data and then blow away the cache and let MapCache (and MapServer) rebuild
>> as users request tiles. The first request will result in MapCache making a
>> call to MapServer to generate the tile/metatile, but subsequent requests
>> for tiles will use the cached tiles.
>>
>>
>>
>> Others on the list may well have more experience and better ideas.
>>
>>
>>
>> Steve
>>
>>
>>
>> *From:* Ahmed Aboulenein [mailto:aboulenein.ahmed at gmail.com]
>> *Sent:* Tuesday, March 17, 2015 2:54 PM
>> *To:* Lime, Steve D (MNIT)
>> *Cc:* mapserver-users at lists.osgeo.org
>> *Subject:* Re: [mapserver-users] Best integration with dynamic map
>> information
>>
>>
>>
>> Hi Steve,
>>
>> Thank you so much for the informative reply, much appreciated!
>>
>> It's actually a new dataset copy every 5 minutes, it seems like replacing
>> a shape file would be most efficient.
>>
>> We are not using MapCache, do you think it would be a good option for our
>> use case with this frequent updates?
>>
>> Thanks again, Steve!
>>
>> Best regards,
>>
>> Ahmed
>>
>>
>>
>> On Tue, Mar 17, 2015 at 8:03 PM, Lime, Steve D (MNIT) <
>> Steve.Lime at state.mn.us> wrote:
>>
>> I’ll let others weigh in on throughput. Seems like that should be a
>> matter of resources and how much hardware you can throw at the problem. Are
>> you using MapCache to serve tiles? MapServer itself is good for rendering
>> tiles or more likely, metatiles.
>>
>>
>>
>> On thread safety MapServer itself isn’t doing any file locking.
>> Presumably the OS or database is handling contention issues. I’d think the
>> right way to go for data storage depends on your update process itself. If
>> you want to incrementally tweak lines as new information becomes available
>> then a database backend seems more reasonable – I’d think you could detect
>> change easier and perhaps only retile very specific areas as a result. If
>> instead you’re getting a new copy of your entire dataset every 5 minutes
>> then perhaps a shapefile would work best – just replace the file and
>> trigger retiling.
>>
>>
>>
>> Steve
>>
>>
>>
>> *From:* mapserver-users-bounces at lists.osgeo.org [mailto:
>> mapserver-users-bounces at lists.osgeo.org] *On Behalf Of *Ahmed Aboulenein
>> *Sent:* Tuesday, March 17, 2015 9:19 AM
>> *To:* mapserver-users at lists.osgeo.org
>> *Subject:* [mapserver-users] Best integration with dynamic map
>> information
>>
>>
>>
>> Hi ALL,
>>
>> We serve traffic lines that gets updated every 5 minutes from our feeds.
>> We would like to use Map Server to generate transparent tiles showing
>> colored polylines of the traffic.
>>
>> I am newbie to Map Server, I got the server and workshop running and also
>> I think I have some good understanding of the map file and layers concept.
>> I have a question about integrating our traffic information as a data
>> source to the MapServer.
>>
>> Important aspects for us:
>>
>> 1. Performance: since we are updating large areas every 5 minutes. We
>> will need throughput like 10K images per second. Does this sound okay?
>>
>> 2. Threads safety: my concern what happens if while updating the source,
>> the MapServer is actually rendering an image from old data. Any locks?
>>
>> I understand our options are:
>>
>> 1. Data files: such as shape files or KML files.
>>
>> 2. Database: PostGIS for example.
>>
>> 3. Service that pulls data: WMS server - I didn't dig much here.
>>
>> Thanks in advance for your help, some pointers or information to go
>> further.
>>
>> Best regards,
>>
>> Ahmed
>>
>>
>>
>> _______________________________________________
>> mapserver-users mailing list
>> mapserver-users at lists.osgeo.org
>> http://lists.osgeo.org/mailman/listinfo/mapserver-users
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.osgeo.org/pipermail/mapserver-users/attachments/20150318/39c5bb11/attachment-0001.html>