[OpenLayers-Users] How do you handle large amounts of data?

[Michael]

The thing is that 20,000 points is more points than anyone can 
reasonably perceive on the screen to get any useful information.  You 
are on the right track to consider clustering.

My application (tracking 70 boats across the Pacific) gives me a raw 
data set of about 15,000 points each with lat, lon, time, and boat id.  
That's a lot of data to process and to upload.

What I am doing instead is pre-thinning the data using the 
Douglas-Peuker algorithm on the server side, which nicely reduces the 
data to any level of detail you care to specify.  I got about a 90% 
reduction in point count with no real loss of information on tracks.

If your data is not lines, but discrete points, then I am less versed.  
Yeah, clustering sounds good.  You may also want to use the BBOX 
strategy to only fetch the points that are relevant to your map.

I can post the douglas-peuker code somewhere if that would be of value.

Michael

On 9/6/2011 6:15 AM, Tito, Joseph wrote:
>
> I'm  trying to expand one of my  current application to handle larger 
> amounts of data. Right now the application comfortably handles 
> 2000-3000 points at a time, but I'd like to expand its capabilities to 
> handle at LEAST 20,000 points. More would be nice, but the minimum 
> requirement is 20,000. Displaying these 20,000 points as X number of 
> 'clusters' is totally acceptable, as long as it gives a good visual 
> representation of the full data set.
>
> One thing I need to take into consideration is that I have multiple 
> data types represented as their own layers, and each user has the 
> ability to turn on/off layers based on their personal preferences.
>
> My question to everyone is, how have others solved this problem? I 
> have a couple server side ideas to "pre-cluster" data in mind, but I 
> want to see how others have tackled this problem before I attack it. 
> If anyone has links to other relevant posts or websites, please share!
>
> Thank you
>
> Joe Tito
>
>
>
> _______________________________________________
> Users mailing list
> Users at lists.osgeo.org
> http://lists.osgeo.org/mailman/listinfo/openlayers-users
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.osgeo.org/pipermail/openlayers-users/attachments/20110906/cce7061d/attachment.html