In this script I use 2 nested 'for' loops to extract a series of variables by month (month = 0 – 11) from 5 netcdf files and then mosaic together each extracted variable. The script runs but after processing about 3 variables, I get the following error statement:
ExecuteError: ERROR 999999: Error executing function. Failed to copy
raster dataset Failed to execute (MosaicToNewRaster).
My suspicion is that the cache memory has been exceeded. I tried using arcpy.Delete_management(SCRATCH) but then received the error that the scratch space no longer existed after only one variable was processed. Maybe I am putting that statement in the wrong place? Any other suggestions?
# Import arcpy module
import arcpy
import os
# Set Geoprocessing environments
arcpy.env.scratchWorkspace = "C:\\WORKSPACE\\Process_4"
SCRATCH = arcpy.env.scratchWorkspace
# Script arguments
ENA = arcpy.GetParameterAsText(0)
if ENA == '#' or not ENA:
ENA = "F:\\GISdata\\Projects\\USGSClimateChange\\climateproj\\srf_ena_eh5_2030-2034_avg_trim_v4.nc"
NRM = arcpy.GetParameterAsText(1)
if NRM == '#' or not NRM:
NRM = "F:\\GISdata\\Projects\\USGSClimateChange\\climateproj\\srf_nrm_eh5_2030-2034_avg_trim_v4.nc"
PNW = arcpy.GetParameterAsText(2)
if PNW == '#' or not PNW:
PNW = "F:\\GISdata\\Projects\\USGSClimateChange\\climateproj\\srf_pnw_eh5_2030-2034_avg_trim_v4.nc"
PSW = arcpy.GetParameterAsText(3)
if PSW == '#' or not PSW:
PSW = "F:\\GISdata\\Projects\\USGSClimateChange\\climateproj\\srf_psw_eh5_2030-2034_avg_trim_v4.nc"
SRM = arcpy.GetParameterAsText(4)
if SRM == '#' or not SRM:
SRM = "F:\\GISdata\\Projects\\USGSClimateChange\\climateproj\\srf_srm_eh5_2030-2034_avg_trim_v4.nc"
Output_Location = arcpy.GetParameterAsText(5)
if Output_Location == '#' or not Output_Location:
Output_Location = "F:\\GISdata\\Projects\\USGSClimateChange\\climateproj\\summary\\"
#Variable list
for variable in ['SNOW', 'SMR', 'SWI', 'T0', 'T33', 'TA']:
# SCRATCH = arcpy.env.scratchWorkspace I attempted to place SCRATCH within the loop thinking that it would be recreated. Nope.
#Month Iteration
for month in [0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11]:
ENA_Layer = SCRATCH +"\\ENA"+variable+str(month)
NRM_Layer = SCRATCH +"\\NRM"+variable+str(month)
PNW_Layer = SCRATCH +"\\PNW"+variable+str(month)
PSW_Layer = SCRATCH +"\\PSW"+variable+str(month)
SRM_Layer = SCRATCH +"\\SRM"+variable+str(month)
# Process: Make NetCDF Raster Layer (169)
arcpy.MakeNetCDFRasterLayer_md(ENA, variable, "x", "y", ENA_Layer, "", "time "+ str(month), "BY_INDEX")
# Process: Make NetCDF Raster Layer (157)
arcpy.MakeNetCDFRasterLayer_md(NRM, variable, "x", "y", NRM_Layer, "", "time "+ str(month), "BY_INDEX")
# Process: Make NetCDF Raster Layer (145)
arcpy.MakeNetCDFRasterLayer_md(PNW, variable, "x", "y", PNW_Layer, "", "time "+ str(month), "BY_INDEX")
# Process: Make NetCDF Raster Layer (133)
arcpy.MakeNetCDFRasterLayer_md(PSW, variable, "x", "y", PSW_Layer, "", "time "+ str(month), "BY_INDEX")
# Process: Make NetCDF Raster Layer (121)
arcpy.MakeNetCDFRasterLayer_md(SRM, variable, "x", "y", SRM_Layer, "", "time "+ str(month), "BY_INDEX")
# Process: Mosaic To New Raster (25)
arcpy.MosaicToNewRaster_management("\"" + ENA_Layer +";"+ NRM_Layer +";"+ PNW_Layer +";"+ PSW_Layer +";"+ SRM_Layer +"\"", Output_Location, variable+"_"+str(month)+"2032", "PROJCS['WGS_1984_Lambert_Conformal_Conic',GEOGCS['GCS_WGS_1984',DATUM['D_WGS_1984',SPHEROID['WGS_1984',6378137.0,298.257223563]],PRIMEM['Greenwich',0.0],UNIT['Degree',0.0174532925199433]],PROJECTION['Lambert_Conformal_Conic'],PARAMETER['false_easting',0.0],PARAMETER['false_northing',0.0],PARAMETER['central_meridian',-102.3000030517578],PARAMETER['standard_parallel_1',30.0],PARAMETER['standard_parallel_2',60.0],PARAMETER['latitude_of_origin',52.0],UNIT['Kilometer',1000.0]]", "", "1", "1", "MEAN", "FIRST")
arcpy.Delete_management(SCRATCH)
Best Answer
FeatureLayers and RasterLayers are layers created in memory, not in the scratch workspace (in background processing they create a file referencing the memory position but that is all).
In order to remove those layers residing in memory, you need to explicitly delete them one by one using arcpy (they do not fall out of scope until Python exits). Note that deleting the variable holding them in a Pythonic way (del varname) will NOT work.
Your code needs to be revised as follows (changes in the end of the code block):
This is not explicitly mentioned in the help files from ESRI, but it is necessary if you want to deal with data in memory. If you monitor the memory use from the task manager you will notice the drop as soon as the Delete statements are hit (in my case, from 1.6GB use it drops to 100MB in every loop).