I'm trying to tackle a web GIS problem and need to figure out the best strategy. I have a map with 30,000 cells on it forming a grid for the province of Alberta. I have a list of data points for each cell – about 50 indicator fields and values for every cell. For example, one value might be human population, so each cell would have a value for human population for that cell. I want to make an interactive map that can show those values visually either with squares for the cells, or a blended heat-map type of look. Beyond that, I'd like to be able to make compound calculations on all of the data points and use those calculations to make additional maps. Taking into consideration the calculations, there are roughly 300,000 permutations to the map. To top it off, it is also temporal; there are 16 different time slices. That means 16 time slices x 300,000 permutations.
My strategy so far:
1) On the server side, calculate the cell values dynamically at run time and send the values to the client (all 30,000 of them). This translates to roughly a value for every 3×3 pixel square in a 525px x 525px image on the screen.
2) On the client side with open layers, use a 1px png that can be resized and have the alpha value adjusted as needed to show the different color gradient corresponding to the data value for each cell. The png would be resized as the map zooms in or out. Theoretically they would be positioned precisely to form a blanket coverage of the map as squares or else using the heat map approach they would overlap each other enough to form a blanket coverage.
The Question:
Is it reasonable to calculate those cell values dynamically on the server and then send to an OpenLayers interface in real time? I'm confident the server can perform the calculations, but can OpenLayers reasonably display 30,000 data points at a time on a map? Is it reasonable to expect to be able to create a blanket coverage using point data? I guess I'm trying to create the equivalent of a vector map that has 30,000 shapes giving 100% blanket coverage of the map, but using point data to simplify it.
The alternative strategies in my mind are:
a) to generate the maps in real time on the server and send as rasterized layers to the client
b) to generate all of the permutations ahead of time and store them on the server as rasterized maps
Any thoughts? Is my approach way off and barking up the wrong tree? Any advice on a better method?
I'd sure appreciate any thoughts on the issue! If you're interested, I might even be looking to contract a developer to help with it.
Thanks so much!
Noah
Best Answer
This questions has been asked a number of times. 30k points, straight up, will not work on an OL map. Or even on a Flash/Silverlight map.
Rough (rough!) order of magnitude numbers to remember - 100 points on a JS map (openlayers), 1,000 points in a Flash Map (e.g ArcGIS Flash or Silverlight), 10,000 points on a desktop app (ArcGIS Desktop) are your fine performance levels. These are "actual" points, not hidden or faked as outlined below.
Not only that, it's not very nice to interact with 30k points. How do I access "that one" marker behind another marker?
You have 2 options:
1) Create raster, like you mentioned, on server. Serve up a WMS service and when your user clicks/hovers over a point, go back to server for the data
2) Reduce # of points you display. Primarily via SERVER-SIDE clustering and also via bounding box trimming. Client-side clustering is "ok" and "cute" but won't help you at 30k points. It'll be slow. So you setup your layer with the bbox strategy and OL will make calls to server on pan/zoom for you, and you return back a new set of data that's trimmed to the point where you return less than 100-300 points.