[OpenLayers-Users] How do you handle large amounts of data?

[David Fawcett]

If you are interested in using MapServer, it now supports clustering
of points.

http://mapserver.org/mapfile/cluster.html

David.

On Tue, Sep 6, 2011 at 8:26 AM, Michael <mdm at yachtpc.com> wrote:
> The thing is that 20,000 points is more points than anyone can reasonably
> perceive on the screen to get any useful information.  You are on the right
> track to consider clustering.
>
> My application (tracking 70 boats across the Pacific) gives me a raw data
> set of about 15,000 points each with lat, lon, time, and boat id.  That's a
> lot of data to process and to upload.
>
> What I am doing instead is pre-thinning the data using the Douglas-Peuker
> algorithm on the server side, which nicely reduces the data to any level of
> detail you care to specify.  I got about a 90% reduction in point count with
> no real loss of information on tracks.
>
> If your data is not lines, but discrete points, then I am less versed.
> Yeah, clustering sounds good.  You may also want to use the BBOX strategy to
> only fetch the points that are relevant to your map.
>
> I can post the douglas-peuker code somewhere if that would be of value.
>
> Michael
>
> On 9/6/2011 6:15 AM, Tito, Joseph wrote:
>
> I’m  trying to expand one of my  current application to handle larger
> amounts of data. Right now the application comfortably handles 2000-3000
> points at a time, but I'd like to expand its capabilities to handle at LEAST
> 20,000 points. More would be nice, but the minimum requirement is 20,000.
> Displaying these 20,000 points as X number of ‘clusters’ is totally
> acceptable, as long as it gives a good visual representation of the full
> data set.
>
>
>
> One thing I need to take into consideration is that I have multiple data
> types represented as their own layers, and each user has the ability to turn
> on/off layers based on their personal preferences.
>
> My question to everyone is, how have others solved this problem? I have a
> couple server side ideas to "pre-cluster" data in mind, but I want to see
> how others have tackled this problem before I attack it. If anyone has links
> to other relevant posts or websites, please share!
>
>
>
> Thank you
>
>
>
> Joe Tito
>
> _______________________________________________
> Users mailing list
> Users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/openlayers-users
>
> _______________________________________________
> Users mailing list
> Users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/openlayers-users
>
>