I am using the latest GeoServer (2.15.1) and GeoWebCache (1.15.1) in integrated mode on a Windows 2012 server. My raster data consists of GeoTIFFs (each TIFF is 47MB in size and covers 200m x 200m floor area) that have been uploaded to GeoServer and grouped together into layer groups (all done via the REST API). Each layer group represents an area in England, UK (therefore I use projection EPSG:27700). A layer group may contain 100 or so GeoTIFFs or potentially thousands. I am caching at scales 1:5000, 1:2500, 1:1000 and 1:500.
GeoWebCache works fine when the layer group contains 100 or o GeoTIFFs. I am able to produce 35,000 or so cached PNG files (total for all 4 scales) in around 20 minutes using all 16 tasks. I load these in m OpenLayers map via WMTS calls and all works fine.
However, when I attempt to cache a layer group that consists of 2226 GeoTIFFs, GeoWebCache was timimg out (a total of 9.4 million PNG tiles). Therefore I have tweaked the settings for both GeoServer and GeoWebCache as follows (so it no longer times out). After 3 hours of caching using 16 tasks, each task has only completed 1,248 tiles and report a further 55 days remaining (though the time remaining does decrease every half hour or so, i.e. after 3.5 hours it states 48 days remaining after 1,632 tiles completed on each thread)!
The time it is taking is no good, as I have a further 25 layer groups to cache! I have kept a close eye on the java process in Task Manager and it never spiked beyond using 5.5 GB of memory. Currently been fluctuating between 3.5 to 5 GB for past half hour.
GeoServer settings updated (not sure if I am correct to do so):
-
Increased the Java Heap Space by updating startup.bat (-Xms2048m -Xmx8192m – 2GB and 8GB max). There is 16GB on the server (the server is dedicated just for Geoserver/GeoWebCache purposes)
-
Updated the WMS settings vis GeoServer admin GUI for "Resource consumption limits" by setting all 4 settings to 0: Max rendering memory (KB), Max rendering time (s), Max rendering errors (count), Max number of dimension values. If I am correct then these settings prevent the timeout now.
-
I have also switched to Production logging and turned off "Log to Std Out"
-
NOTE: I have not updated any values in gwc-gs.xml such as hardMemoryLimit, concurrenyLevel, etc. Nor have I changed the metatiling size (using default 4 x 4).
-
I have however enabled CORS and changed the GeoServer serviceStrategy to SPEED (updated web.xml for both these settings)
GeoWebCache settings updated (via GeoServer admin –> Tile Caching –> Caching Defaults):
- Tile Locking Mechanism = NIO based lock (interprocessing locking)
Is there any way to optimize GeoServer/GeoWebCache to speed up the caching? Is it even sensible to cache such huge amounts of GeoTIFFs grouped together into layer groups? I know I can cache each layer/GeoTIFF individually but that then mean I need to add thousands of layers to my OpenLayers map in order to display/load each layer from the cache which means thousands of http requests per layer.
UPDATE: I have since found this link posted in 2010 by a member of the GeoSolutions team in Italy. It lists the estimated time for GeoWebCache to seed/generate different numbers of tiles. (Not sure how accurate/relevant it is to the current version of GeoWebCache):
https://docs.google.com/spreadsheets/d/1t9OJb-Z9vnHYKw6fWR82Wml4Dj9ePnNoOzqDw3v_PwU/edit#gid=0
Best Answer
You should create an image mosaic layer rather than a layer group (which assumes the layers are roughly on top of each other and doesn't have any indexing). Checkout the tutorial for more details.
If you intend to display the images when the user is zoomed out a long way then you will also benefit from using an image pyramid which will prevent GeoServer from having to open too many files at once.
Finally, unless top speed is absolutely essential I wouldn't bother pre-seeding the tiles in advance, and just let the cache fill up as your users visit areas of interest. I do recommend checking your boss's house out as that is the first place he will check, so it looks good if you have that in the cache before the demo.