[GIS] arcpy JoinField_managment RuntimeError

arcpyattribute-joinsenterprise-geodatabasepython

My script is supposed to pull weekly flu data from a department of health website, parse it for that week, and then push the data up to our ArcSDE server so that it can be displayed on our Web Mapping application. The data is in a .csv format and the script parses the data just fine, but when it comes to doing a join with a feature class with county outlines I get a

Traceback (most recent call last):
File "C:\Users\username\Desktop\Projects\flu_mapping\script\flu_data0.6.0.py", >line 55, in
arcpy.JoinField_management ("memorycounty", "ID", outputcsv, "Statecode",)
File "C:\Program Files (x86)\ArcGIS\Desktop10.1\arcpy\arcpy\management.py", line 5383, >in JoinField
raise e
RuntimeError: Object: Error in executing tool

This wasn't an issues until I started using the open() command to create a new .csv file to parse the data to, instead of manually making one.

Here is my code

import os
import arcpy
from arcpy import env
import math
import numpy
import csv
import urllib2
import tempfile

#working directory

fludirectory = 'C:\\Users\\username\\Desktop\\Projects\\flu_mapping\\GIS_data\\'

#pull flu data from ND Department of health
fludata = urllib2.urlopen ('http://www.ndflu.com/data/weekenddata.txt')



#arcpy workspace
arcpy.env.workspace = "in_memory"

#load data so that numpy can use it
rows = numpy.loadtxt(fludata, int, delimiter=",",skiprows=4, usecols =(2,3,4,5,6,7,8,9))

#parse the data and write it to the CSV file
createcsv = fludirectory + "\\" + 'output_data.csv'
outputcsv = open (createcsv, 'wb')

week = max(rows [:,0])

with outputcsv as csvfile:
    flucsv = csv.writer(csvfile, delimiter=",",
                            quotechar='|', quoting=csv.QUOTE_MINIMAL)
    flucsv.writerow (["Statecode","flu_num"])
    for row in rows:
        if row[0] == week:

            countid = row[1]
            totalflu = sum(row[2:])
            csvoutput = '%s, %s' %(countid, totalflu)
            print csvoutput
            flucsv.writerow ([countid, totalflu]) 


outputcsv.close

#make a copy of the county dataset

#county dataset
countydata = fludirectory + "\\" + 'county_ID_code.shp'
#make a temporary feature layer
arcpy.MakeFeatureLayer_management (countydata, "memorycounty")

#join output csvoutput to the in_memory shapefile
arcpy.JoinField_management ("memorycounty", "ID", outputcsv, "Statecode",)
#test the script
arcpy.FeatureClassToFeatureClass_conversion ("memorycounty", fludirectory, 'testoutput.shp')

Also at the end I am using Feature Class to Feature Class to see if everything worked (obviously it's not) and that will be replaced by code that pushes the feature class to the ArcSDE server.

Best Answer

There are a few issues with the code. First some useful snippets that I encourage using:

import os

# Use raw strings, since they are easier; never end paths with a slash
fludirectory = r'C:\Users\username\Desktop\Projects\flu_mapping\GIS_data'

# Join paths this way
createcsv = os.path.join(fludirectory, 'output_data.csv')

# NumPy uses max like this
week = rows[:, 0].max()

The bulk of the issues are really with the I/O processing:

  • outputcsv.close does nothing without ()
  • close() is not needed if you are using the with x as : block
  • the with block should open the file, but that is not done
  • open() returns a file pointer. These should not be passed to functions like arcpy.JoinField_management

Try structuring the CSV processing like this

with open(createcsv, 'wb') as fp:
    flucsv = csv.writer(fp, delimiter=",", quotechar='|', quoting=csv.QUOTE_MINIMAL)
    flucsv.writerow(["Statecode", "flu_num"])
    ...
# don't call fp.close()!

Lastly, try using the file path (rather than the file pointer) to avoid the RuntimeError:

arcpy.JoinField_management("memorycounty", "ID", createcsv, "Statecode")
Related Question