I have a script written in Python that uses ArcPy to create maps of the kernel densities of thousands of datasets. Each dataset is relatively small, but there about 50,000 sets in total.
When I run my script, it stops after about 10-15 minutes and gives the "python.exe has stopped working" popup. The Windows 7 event viewer tells me it is an Exception Code 0x0000005, which appears to be an access violation. The datasets the code works on are not the same each time, so it is not a specific set that is causing the problem.
Watching the task manager when the script is running, I noticed that python.exe is increasing its memory usage every few seconds, which makes me believe there is a memory leak somewhere. I thought Python had garbage collection? Is there something that arcPy is doing?
EDIT:
So I tracked down the part of my code causing the problem, but I don't know HOW it is leaking. I think I am cleaning everything up. Does anyone spot something I missed?
def makeMap(asno, isKdens):
try:
mxd = arcpy.mapping.MapDocument("F:\\testFiles\\testMap.mxd") ###Reference to basemap document
df = arcpy.mapping.ListDataFrames(mxd, "Layers")[0] ###Get data frame reference in var
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
if isKdens:
###Set up layer file
lyrFile = arcpy.mapping.Layer(r"F:\\testFiles\\output\\AS_"+asno+"_KDens.lyr")
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
else:
###Set up layer file
lyrFile = arcpy.mapping.Layer(r"F:\\testFiles\\output\\AS_"+asno+".lyr")
lyrFile.visible = True
###Add layer file
arcpy.mapping.AddLayer(df, lyrFile)
###ref added layer
lyr = arcpy.mapping.ListLayers(mxd)[0]
lyr.visible = True
#If this is a raw map, just select points.
if isKdens==0:
arcpy.SelectLayerByAttribute_management ("pointslyr", "NEW_SELECTION", "\"asno\" = "+asno+"")
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
#If KDens map, apply symbology
else:
arcpy.ApplySymbologyFromLayer_management(lyr, templateLayer)
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
###Zoom to selection
df.zoomToSelectedFeatures()
lyrExtent = lyr.getSelectedExtent()
df.extent = lyrExtent
###Export to PDF!!!
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
if isKdens:
arcpy.mapping.ExportToPDF(mxd, r"F:\\testFiles\\output\\AS_"+asno+"_KDens_Map.pdf")
##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
else:
arcpy.mapping.ExportToPDF(mxd, r"F:\\testFiles\\output\\AS_"+asno+"_Map.pdf")
except:
print arcpy.GetMessages()
#####DELETION
del mxd, df, lyrFile, lyrExtent, lyr
print "DELETED MXD AND DFS"
EDIT 2
So, I also tried turning off logging, since I have heard that can cause memory leaks, too. There is still something leaking. I also tried using Delete_management
instead of del
, but that just messes things up. It almost seems like my code is not running sequentially. A delete after a reference still gives a "references before defined" error…
Anyway, now I am trying to locate the memory leak using memory analyzer packages. Out of all of them that work with Python, I only got objgraph
to work. I placed a objgraph.show_growth
call before and after the function whose code is above. Here are the results:
Before
tuple 11959 +11959
function 3643 +3643
dict 1632 +1632
wrapper_descriptor 1086 +1086
cell 927 +927
list 635 +635
builtin_function_or_method 590 +590
property 551 +551
method_descriptor 390 +390
weakref 373 +373
After
weakref 398 +25
wrapper_descriptor 1105 +19
tuple 11977 +18
dict 1644 +12
member_descriptor 172 +4
getset_descriptor 213 +3
method_descriptor 392 +2
instance 15 +2
list 636 +1
`_RLock 2 +1
It doesn't look like any of my objects are leaking, though I am not quite sure what instance
means in the second growth dump. Anyone have any insights?
Best Answer
Just use the
Delete
tool. Callarcpy.management.Delete("rdlayer")
after saving the layer file.