I'm researching the best way to display a very large weather dataset (850mb +/-) for folks to use on their mobile and desktop devices. This huge dataset (1 point every 12km across the USA) that I'd like to show on a leftlet map will be using OpenStreet map layers. This database will be updated daily if not hourly with fresh data.
I'm looking for thoughts on what type of database to use, how to best display (speed) this data and any other advice you professionals have to share. There will be around 12 layers users could switch on an off.
I have been looking into GeoServer for example, and think that might be a good route. I am not interested in using Google fusion tables and would like to run this on my own server as the data is already produced locally.
Also, the data is currently available is in the NetCDF format. Looks like GeoServer can use this.
Best Answer
Geoserver is a good way to go.
You could use the NETCDF Datastore to create the layer and style the Raster using SLD Raster Symbolizer it so that it overlays your OSM base layer transparantly.
To handle the updating of the data without bringing the Layers down for long, you could use the Geoserver REST API via Curl. To make this work, you would have two DataStores configured, one active, one being updated. Both assigned to the same workspace. The Layer for the Weather Data would be created and styled for each datastore, but named slightly differently.
A Layer Group would have your OSM Base Layer and the current Weather data layer configured
To switch out to the new data, simply use the GeoServer REST API (Layer Tutorial Here!) to delete the Active Layer Group and recreate the Layer Group Again with the layer from the datastore with the updated information.
This whole process could be handled in you Script, using Curl to get the data, drop into the appropriate data directory for the datastore and then sending the API commands.
You may need to issue a Reload to Geoserver via the REST API as well, although Geoserver is supposed to recognize the change automatically.