[GIS] Database for offline slippy map tiles

leafletslippy

I currently have an offline HTML5 map application (built on Leaflet & KendoUI with custom additions) that has an app manifest and works fine on multiple platforms. However, I am hesitant to use the manifest to store the actual map tiles this way(PNG files stored as a TMS style Tile Cache).

Issues:

  • There could be a lot of tiles (10MB – 50MB) in about 1,000 PNG files
  • The initial download could be really slow (and hard to show progress to the user)
  • App Manifests work or they don't if they don't the entire offline caching will fail (accord to [whatwg.org][1])
  • The offline user will occasionally reconnect and need to get refreshes of the Tiles, These are small deltas but the app manifest mechanism would reload all the js, css, and PNG files as soon at the manifest updates

Alternative idea: keep the web application separate from the storage of the slippy map tiles. Store the tiles in a web application friendly database

Update:

[PouchDB recently added support for binary blobs. I am getting good initial results. See:
https://stackoverflow.com/questions/16721312/using-pouchdb-as-an-offline-raster-map-cache ]

Question:
What does the collective wisdom (and experience) say about the following choices for a JavaScript friendly DB:

  1. SqlLite
  • Seems like you need to create a native app wrapper for this to allow it to talk to JavaScript
  • E.g. Add you DLL to a native program for windows, and PhoneGap for android/IOS
  1. WebSQL
  • depricated
  • but it was a SQL Lite that I could easily generate and distribute from the host web server
  1. IndexDB

Requirements:

  • Fast initial population (via download) to the client Web DB
  • Compatible with current Leaflet TileLayer API (i.e. I would rather not write a custom layer, but if needed …) (e.g. MbTiles)
  • Platform: Windows Laptops, but Android and IOS tablets desired (I can wait till IndexDB is released, don't need immediate support)
  • I would rather not write a native app (EXE, IOS, Android) but it if that is the best way…
  • Server Side generation of web maps (this will be an automated process). The user picks a location, chooses maps, and they get dynamically transformed and turned into a slippy tile cache (this work is already largely done).
  • Fast bulk initial download
  • Map change delta refreshes (I will write this logic, based on constant stock numbers, and update date logic)
  • Minimal impact on current Leaflet & KendoUI web application

Key background idea: while the web app is fairly stable, the slippy map tiles are generated on the fly for your location and what type of problem you are doing (at the back-end). So I thought of two other ways for transferring the initial 'big bang' and then updates:

Zip File (probably not a good idea – as it adds server load) also expansion on client machine will require user interaction, but it does allow slippy tiles to use local url's

HTML5 File API: I have not looked at this in great detail. But it appears has most operations to create a local file tree in TMS format: http://www.html5rocks.com/en/tutorials/file/filesystem/ what will be interesting to test is the performance (e.g. can I use web works to maximize the bandwidth to disk and across the net). IndexDB is not widely implemented to be web worker friendly (sync interface: https://stackoverflow.com/questions/10698728/indexeddb-in-web-worker-on-firefox

I have found some additional info about using IndexDB with Leaflet:

https://github.com/calvinmetcalf/leaflet.pouch (syncs couchdb with indexdb for offline)
Also here are some tests for the read/write speeds for indexdb, websql, & local store:
http://jsperf.com/indexeddb-vs-localstorage/15

And here is how to use the read/write file API from javascript: (and also to ask to increase storage limits) http://www.html5rocks.com/en/tutorials/file/filesystem/

Tom MacWright (aka tmcw) gave some good feedback. Your example is really going to help when I get to creating custom layers to ingest the binary blobs.

I did some testing yesterday with IndexedDB and by using some polyfills and libraries I think it will solve my issues. Now it is time to put some sweat equity into this, and I will report back.

If you want to see my results of my study on client side databases see:

https://stackoverflow.com/questions/14113278/storing-image-data-for-offline-web-application-client-side-storage-database

Best Answer

PhoneGap and MBTiles.

WebSQL & IndexDB will not be enough. 'Windows Laptops' will not be the same code as mobile devices.

Related Question