Deleting folder and processed files in QGIS using PyQGIS

deletedirectorypermissionerrorpyqgis

My script creates a folder named 'output' and saves processed files there. I am trying to delete this folder and all files within it, if I need to re-run the script (see reproducible example at the bottom).

However, in my first attempt re-running the script I get the following error:

Traceback (most recent call last):
File "C:\OSGeo4W\apps\Python39\lib\code.py", line 90, in runcode
exec(code, self.locals)
File "", line 1, in
File "", line 17, in
File "C:\OSGeo4W\apps\Python39\lib\shutil.py", line 740, in rmtree
return _rmtree_unsafe(path, onerror)
File "C:\OSGeo4W\apps\Python39\lib\shutil.py", line 618, in _rmtree_unsafe
onerror(os.unlink, fullname, sys.exc_info())
File "C:\OSGeo4W\apps\Python39\lib\shutil.py", line 616, in _rmtree_unsafe
os.unlink(fullname)
PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:/…/output\point_layer.gpkg'

If I try to run it again, then it works. If I try one more time, it does not run (same error); and if I try again, then it works, etc..

When I manually remove the layers, then the script works in the next run. If I manually type QgsProject.instance().removeAllMapLayers() in the Python console after successfully running the script, it will also work in the next run.

Based on what I have researched so far, I guess this happens because the layers in the 'output' folder are not entirely closed/removed from QGIS. How can I make it work?

Reproducible code:

from qgis.core import *
from qgis import processing
import os
import shutil

#Current .qgz folder
dir = QgsProject.instance().readPath("./") + '/'

#Path to output folder
path_output = os.path.join(dir,'output')

#Remove 'output' folder and files within it, if exists
if os.path.exists(path_output):
    #Remove all layers from project
    QgsProject.instance().removeAllMapLayers()
    #Remove output folder and files within it
    shutil.rmtree(f'{path_output}')

#Create 'output' folder, if it does not exist
if not os.path.exists(path_output):
    os.makedirs(path_output)

#Create sample layer
vl = QgsVectorLayer("Point", "temporary_points", "memory")
pr = vl.dataProvider()
vl.startEditing()
pr.addAttributes([QgsField("name", QVariant.String)])
fet = QgsFeature()
fet.setGeometry( QgsGeometry.fromPointXY(QgsPointXY(15,60)) )
fet.setAttributes(["Johny"])
pr.addFeatures([fet])
vl.commitChanges()
QgsProject.instance().addMapLayer(vl)

#Reproject and save new layer in 'output' folder
params = {'INPUT':vl,\
          'TARGET_CRS':QgsCoordinateReferenceSystem('EPSG:4326'),\
          'OUTPUT': f'{path_output}/point_layer.gpkg'}
result = processing.run('qgis:reprojectlayer', params)

#Remove sample layer
QgsProject.instance().removeMapLayer(vl)
#Add reprojected layer
reprojected_layer = QgsVectorLayer(result["OUTPUT"], "new_layer", "ogr")
QgsProject.instance().addMapLayer(reprojected_layer) #adiciona layer pts reprojetado.

Best Answer

It seems to me a problem of timing, whereby the line: shutil.rmtree(f'{path_output}') is being called before the layer has been completely released. I tried a few approaches to solve the issue including connecting a function to remove the directory to the QgsProject.mapLayersRemoved signal, which I found was unsuccessful.

The only way I was able to reliably make this work was to wrap the entire logic in a function, remove all map layers from the project, then call the function with a small delay using QTimer.singleShot().

from qgis.core import *
from qgis import processing
from PyQt5.QtCore import QTimer
import os
import shutil


def add_layer():
    #Current .qgz folder
    dir = QgsProject.instance().readPath("./") + '/'

    #Path to output folder
    path_output = os.path.join(dir,'output')

    #Remove 'output' folder and files within it, if exists
    if os.path.exists(path_output):
        #Remove output folder and files within it
        shutil.rmtree(path_output)

    #Create 'output' folder, if it does not exist
    if not os.path.exists(path_output):
        os.makedirs(path_output)

    #Create sample layer
    vl = QgsVectorLayer("Point", "temporary_points", "memory")
    pr = vl.dataProvider()
    pr.addAttributes([QgsField("name", QVariant.String)])
    fet = QgsFeature()
    fet.setGeometry( QgsGeometry.fromPointXY(QgsPointXY(15,60)) )
    fet.setAttributes(["Johny"])
    pr.addFeatures([fet])
    QgsProject.instance().addMapLayer(vl)
    
    #Reproject and save new layer in 'output' folder
    params = {'INPUT':vl,\
              'TARGET_CRS':QgsCoordinateReferenceSystem('EPSG:4326'),\
              'OUTPUT': f'{path_output}/point_layer.gpkg'}
    result = processing.run('qgis:reprojectlayer', params)

    #Remove sample layer
    QgsProject.instance().removeMapLayer(vl)
    # Refresh map canvas
    iface.mapCanvas().refresh()
    #Add reprojected layer
    reprojected_layer = QgsVectorLayer(result["OUTPUT"], "new_layer", "ogr")
    QgsProject.instance().addMapLayer(reprojected_layer) #adiciona layer pts reprojetado.
        

# Remove all layers from project
QgsProject.instance().removeAllMapLayers()
# Refresh map canvas
iface.mapCanvas().refresh()
# Call add_layer function with 200ms delay
QTimer.singleShot(200, add_layer)

Interestingly, for me the above code also works with those lines:

if os.path.exists(path_output):
    shutil.rmtree(path_output)

commented out entirely.

Related Question