[GIS] ArcPy script causing python.exe to stop working (memory issue?)

arcpymemorypython.exe-stopped-working

I have a script written in Python that uses ArcPy to create maps of the kernel densities of thousands of datasets. Each dataset is relatively small, but there about 50,000 sets in total.

When I run my script, it stops after about 10-15 minutes and gives the "python.exe has stopped working" popup. The Windows 7 event viewer tells me it is an Exception Code 0x0000005, which appears to be an access violation. The datasets the code works on are not the same each time, so it is not a specific set that is causing the problem.

Watching the task manager when the script is running, I noticed that python.exe is increasing its memory usage every few seconds, which makes me believe there is a memory leak somewhere. I thought Python had garbage collection? Is there something that arcPy is doing?

EDIT:
So I tracked down the part of my code causing the problem, but I don't know HOW it is leaking. I think I am cleaning everything up. Does anyone spot something I missed?

def makeMap(asno, isKdens):
try:
    mxd = arcpy.mapping.MapDocument("F:\\testFiles\\testMap.mxd")   ###Reference to basemap document
    df = arcpy.mapping.ListDataFrames(mxd, "Layers")[0]     ###Get data frame reference in var
    ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    if isKdens:
        ###Set up layer file
        lyrFile = arcpy.mapping.Layer(r"F:\\testFiles\\output\\AS_"+asno+"_KDens.lyr")
    ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    else:
        ###Set up layer file
        lyrFile = arcpy.mapping.Layer(r"F:\\testFiles\\output\\AS_"+asno+".lyr")
        lyrFile.visible = True
    ###Add layer file
    arcpy.mapping.AddLayer(df, lyrFile)

    ###ref added layer
    lyr = arcpy.mapping.ListLayers(mxd)[0]
    lyr.visible = True

    #If this is a raw map, just select points.
    if isKdens==0:
        arcpy.SelectLayerByAttribute_management ("pointslyr", "NEW_SELECTION", "\"asno\" = "+asno+"")
    ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    #If KDens map, apply symbology
    else:
        arcpy.ApplySymbologyFromLayer_management(lyr, templateLayer)
    ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++

    ###Zoom to selection
    df.zoomToSelectedFeatures()
    lyrExtent = lyr.getSelectedExtent()
    df.extent = lyrExtent

    ###Export to PDF!!!
    ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    if isKdens:
        arcpy.mapping.ExportToPDF(mxd, r"F:\\testFiles\\output\\AS_"+asno+"_KDens_Map.pdf")
    ##++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
    else:
        arcpy.mapping.ExportToPDF(mxd, r"F:\\testFiles\\output\\AS_"+asno+"_Map.pdf")
except:
    print arcpy.GetMessages()
#####DELETION
del mxd, df, lyrFile, lyrExtent, lyr
print "DELETED MXD AND DFS"

EDIT 2
So, I also tried turning off logging, since I have heard that can cause memory leaks, too. There is still something leaking. I also tried using Delete_management instead of del, but that just messes things up. It almost seems like my code is not running sequentially. A delete after a reference still gives a "references before defined" error…

Anyway, now I am trying to locate the memory leak using memory analyzer packages. Out of all of them that work with Python, I only got objgraph to work. I placed a objgraph.show_growth call before and after the function whose code is above. Here are the results:

Before

tuple                         11959    +11959
function                       3643     +3643
dict                           1632     +1632
wrapper_descriptor             1086     +1086
cell                            927      +927
list                            635      +635
builtin_function_or_method      590      +590
property                        551      +551
method_descriptor               390      +390
weakref                         373      +373

After

weakref                 398       +25
wrapper_descriptor     1105       +19
tuple                 11977       +18
dict                   1644       +12
member_descriptor       172        +4
getset_descriptor       213        +3
method_descriptor       392        +2
instance                 15        +2
list                    636        +1
`_RLock                    2        +1

It doesn't look like any of my objects are leaking, though I am not quite sure what instance means in the second growth dump. Anyone have any insights?

Best Answer

Just use the Delete tool. Call arcpy.management.Delete("rdlayer") after saving the layer file.

Related Question