In QGIS, if I open some GeoPackage file like mylayers.gpkg, it will create two files named mylayers.gpkg-shm and mylayers.gpkg-wal in the same folder. After I close QGIS, these files disappear. I understand that they are temporary files, but I wonder what their purpose is.
QGIS – Understanding .gpkg-shm and .gpkg-wal Files in GeoPackage
geopackageqgis
Related Solutions
QGIS is made to use Write Ahead Logging (WAL) by commit https://github.com/qgis/QGIS/commit/f939e9cff598b95e95b0de099d0c9a92eed0ea9c
The new behavior should be to open gpkg database file in WAL journal mode if file is on local disk but turn it back to the default journal mode when the connection is closed.
I made a test with QGIS 2.18.2 and that seems to work. I opened a gpkg database with "Add Vector Layer" and immediately -wal and -shm files appear in the same directory. When I removed the layer from QGIS both files disappear. I verified with spatialite-gui that the journal mode was really turned back by making a pragma request
PRAGMA journal_mode;
You must not ever delete manually -wal and -shm files. If they still are there in the directory I would check if there are any other clients than QGIS having connections. If not, opening and closing db with QGIS may clear the temporary files and turn logging to default mode. If that does not help, try with the pragma command
PRAGMA journal_mode=DELETE;
It is not possible to turn the journal mode from WAL to anything else if there are any other open connections. Only the last connection can do the change. Also, journal mode is persistent and gpkg database that is set to WAL mode remains in WAL mode until it is explicitly set to something else. I guess that this may cause troubles sometimes. I made this simple test:
- Open a gpkg that is originally in journal_mode=delete with QGIS -> journal mode turns into WAL
- Open the same gpkg with spatialite-gui -> journal mode is still WAL
- Close the layer from QGIS -> journal mode is WAL in spatialite-gui
- Close gpkg from spatialite-gui -> the -wal and -shm files disappear but journal mode remains WAL
In this state the gpkg database will work from local disk but not from network drives if copies of it are given to end users!
Edit:
These are the two ways reported by Even Roualt for turning WAL mode off from QGIS:
People can either define the OGR_SQLITE_JOURNAL environment variable to DELETE or set the QGIS setting "/qgis/walForSqlite3" (in advanced mode) to false, and this will prevent QGIS from enabling WAL on opening. The drawback is potential deadlocks in some situations where a reader and writer would run concurrently.
Geopackage is in fact a container format (like zip files). It as based on SQLite, thus it has database functionality. See more about Geopackage here: https://www.geopackage.org/
You can save several layers (vector and raster) as well as styles, QGIS projects etc. to the same Geopackage file.
To make an example (download the file):
I loaded the pre-installed vector base-map (type
worldin the coordinates field at the very bottom of your QGIS window), than applied a style to it. With geomtry by expression, I created the centroid of the countries (point layer) and the boundary (line layer), applying different styles for each.Than I saved the first layer:
right click layer / Export / Save Features As...and created a new Geopackage.Now, I saved the two other layers to the same Geopackage:
right click layer / Export / Save Features As...an select the same Geopackage, but be sure to define another Layername in the dialog window so that the already existing layer will not be overwritten.Now, save the project to the same Geopackage:
Menu Project / Save as [second entry] / Geopackage(see screenshot). In the dialog that opens, forconnectionselect the same Geopackage file, give a name to the project and click OK.
Click the following link to download the Geopackage created this way. To open it, in your QGIS main window go to the Browser panel and navigate to the folder where you stored the Geopackage. You can expand the Geopackage entry to see what it contains - compare the screenshot: you see the different layers as well as the QGIS project (red arrow). Double click the project to open the whole project or one of the layers to open just this one. When opened, it should look like this (if you just open the Geopackage, e.g. by drag and drop, only the layers contained will be loaded, but not the project - thus settings in the project like styles will not be loaded):
In this way, you can add additional layers etc. to the Geopackage. Click the curved double arrow icon at the top of the Browser panel to refresh the display of the Geopackage's content.
In this way, you can share whole projects, including the data by sending just a single file. However, be aware that many settings (project!) are QGIS-specific and may also vary from version to version of QGIS. So for long-term archiving, I would not recommend to save everything in one large Geopackage - it's also difficult to keep an overview of what you have and as a complex binary format, you need special software (like QGIS) to access Geopackages.
Best Answer
As the SQLite core of a GeoPackage is a single-file based RDB, it needs to utilize temporary support structures to guarantee atomic transaction management.
SQLite supports different mechanics to achieve this, and the QGIS specific GPKG implementation defaults to Write Ahead Logging (WAL), available in SQLite > 3.7, where
*.gpkg-walfile holds the WAL for the current connection; this file keeps a progressive data log of the transactional state of the DB betweenCOMMITs (orROLLBACKs)*.gpkg-shmSHhared Memory file manages concurrent access to the DB via an index to the WAL