[GIS] Writing to a text file from arcpy.ListFields()

arcgis-10.2arcpyfields-attributes

I'm trying to replicate output to a text file from an attribute table using the write() method to transfer the values to the .txt file. The program writes the values but groups each row of values into braces [] and returns a u"value" format for each value. Additionally blanks are represented by the "None". Finally, the order of the field values are wrong even though I tried to build an order using a criteria list to refine the arcpy.ListFields()

This is the output I'm now getting:

[100.0, u'', 304604, 1298977.349872, 295805.476068, u'113984001', u'113984001', None, None, 4, u'D472B444-7F3C-40E9-85B1-A6A700EBFBEE', u'', 2030, u'BT-501'][100.0, u'', 32204, 1343091.764101, 194014.427182, u'134531001', u'134531001', None, None, 4, u'D18279D5-34D7-46A7-9342-A6AB00A55F48', u'', 3660, u'IS-501']

I would like to get this:

D472B444-7F3C-40E9-85B1-A6A700EBFBEE,113984001,113984001,1298977.349872,295805.476068,304604,,100,BT-501,2030,4,,,
D18279D5-34D7-46A7-9342-A6AB00A55F48,134531001,134531001,1343091.764101,194014.427182,32204,,100,IS-501,3660,4,,,
1ACC967C-241F-4C07-BB7A-A6AC00B632E3,9666001,9666001,1275486.924555,280097.24814,259088,,100,SH-501B,,2,,,A

Here's my code:

inFields = ["UniqueID","Address_ID","Account_ID","X","Y","Ref_ID","Side","Score","Route","Sequence","Service_Day","Demand_2","Demand_3","Week"]

    outputWS = r"C:\RS_Data\Workspace\BATCH_CO_161_imports\gisdb\layers.gdb"

        finalFC = os.path.basename(layer)
        print finalFC
        day = time.strftime("%m%d%Y")
        arcpy.FeatureClassToFeatureClass_conversion(layer, outputWS, finalFC)
        outPath = 'S:\\Shared\\RouteSmart\\CO_161_Cleanscapes\\To_Tower'
        name = os.path.basename(finalFC)
        dataFields = arcpy.ListFields(finalFC)
        file = os.path.join(outPath, ("{0}{1}{2}".format(day,name,".txt")))
        print file
        f = open(file, "w")
        writeFields = []
        for field in dataFields:
            if field.name in inFields:
                writeFields.append(field)
        for row in arcpy.SearchCursor(finalFC):
            fieldVals = [row.getValue(field.name) for field in writeFields]
            f.writelines(str(fieldVals))
        del row
        f.close()

Is there a way I can specify the order in which the values are written, remove the u"value" formatting as well as the []'s around each row, replace "None" with "" and have it write each row as a new line in the text file?

After taking a mix tips from the answers I've gotten it close. The None's are still there but that's the only thing left to resolve. Here's the modified code:

        arcpy.FeatureClassToFeatureClass_conversion(layer, outputWS, finalFC)[0]
        outPath = 'S:\\Shared\\RouteSmart\\CO_161_Cleanscapes\\To_Tower'
        name = os.path.basename(finalFC)

        dataFields = [i.name for i in arcpy.ListFields(finalFC)]

        file = os.path.join(outPath, ("{0}{1}{2}".format(day,name,".txt")))
        print file
        f = open(file, "w")

        writeFields = []

        for field in inFields:
            if field in dataFields:
                writeFields.append(field)

        for row in arcpy.SearchCursor(finalFC):
            # No longer need to reference name property (field.name)
            fieldVals = [row.getValue(field) for field in writeFields]
            # Replace nulls with empty strings
            fieldsVals = ['' if i is None else i for i in fieldVals]
            # Create a comma-separated string of values
            fieldVals = [str(field) for field in fieldVals]
            out_string = ','.join(fieldVals)
            # Write the string--not the list--to the table
            f.writelines(out_string)
        del row
        f.close()

    arcpy.Delete_management(layer)
    arcpy.Delete_management("OldSeq")

Best Answer

Though there are easier ways to achieve your goal, here's a minimal-modification fix for your script. See the included comments:

inFields = ["UniqueID","Address_ID","Account_ID","X","Y","Ref_ID","Side","Score","Route","Sequence","Service_Day","Demand_2","Demand_3","Week"]

outputWS = r"C:\RS_Data\Workspace\BATCH_CO_161_imports\gisdb\layers.gdb"

finalFC = os.path.basename(layer)
print finalFC
day = time.strftime("%m%d%Y")

arcpy.FeatureClassToFeatureClass_conversion(layer, outputWS, finalFC)
outPath = 'S:\\Shared\\RouteSmart\\CO_161_Cleanscapes\\To_Tower'
name = os.path.basename(finalFC)

# Changed to a list comprehension to grab the field names directly
dataFields = [i.name for i in arcpy.ListFields(finalFC)]

file = os.path.join(outPath, ("{0}{1}{2}".format(day,name,".txt")))
print file
f = open(file, "w")

writeFields = []

for field in inFields:
    if field in dataFields:
        writeFields.append(field)

for row in arcpy.SearchCursor(finalFC):
    # No longer need to reference name property (field.name)
    fieldVals = [row.getValue(field) for field in writeFields]
    # Replace nulls with empty strings
    fieldVals = ['' if i is None else i for i in fieldVals]
    # Create a comma-separated string of values
    out_string = ','.join(fieldVals)
    # Write the string--not the list--to the table
    f.writelines(out_string)
del row
f.close()

More info on:

The join method on strings

List comprehensions

Related Solutions

[GIS] How to check for empty values in fields of a featureclass

I just wanted to point out that just using if not row.Stand may yield undesired results (remember, explicit is better than implicit from the Zen of Python). Take the following example:

>>> for i, sample in enumerate(['a',1,None, '', ' ', 0]):
    if not sample:
        print 'value "{}" at index {} is empty'.format(sample, i)


value "None" at index 2 is empty
value "" at index 3 is empty
value "0" at index 5 is empty

We can see here that a single space returns true, because it is a character. We are also removing zeros (which may or may not be a "NULL" value, depends on the situation).

Instead, it may be better to test if it is None or if the string contains any non-whitespace characters:

>>> for i, sample in enumerate(['a',1,None, '', ' ', 0]):
    if sample is None or not str(sample).strip():
        print 'value "{}" at index {} is empty'.format(sample, i)


value "None" at index 2 is empty
value "" at index 3 is empty
value " " at index 4 is empty

Now we have removed any values that are just white space. This may not be a big issue, but me having prior work experience in an assessors office, there were plenty of parcel fields that were supposed to have values and only contained white space would not be flagged in the simple if not row.Field example. Here is what you can use to test for a single field:

rows = arcpy.SearchCursor(fc,"","","","")
for row in rows:
   if row.Stand is None or not str(row.Stand).strip():
        print 'empty row!'
        break

Or, if you want to test all fields:

import arcpy
fc = r'C:\TEMP\Huxley.gdb\Watermain\WtrValve'
fields = dict((f.name, []) for f in arcpy.ListFields(fc) if not f.required)

rows = arcpy.SearchCursor(fc,"","","","")
for row in rows:
    for f in fields.keys():
        fields[f].append(row.getValue(f))

for field, values in fields.iteritems():
    if any(map(lambda s: s is None or not str(s).strip(), values)):
        print 'Field: "{}" has empty values'.format(field)

This yielded:

>>> 
Field: "VAL_TYPE" has empty values
Field: "GNSS_HEIGHT" has empty values
Field: "HL_ACCU" has empty values
Field: "PHOTO_1" has empty values
Field: "COMMENTS" has empty values
Field: "PROJ_NUMB" has empty values
Field: "PDF_LOC" has empty values
Field: "YR_INST" has empty values
Field: "TOP_COND" has empty values
Field: "VAL_USE" has empty values
Field: "ASBUILT" has empty values
Field: "PROJ_NAME" has empty values

[GIS] Very Simple ArcPy script using arcpy.SearchCursor and row.getValue

Welcome to the GIS SE. I have edited your script. Please note:

I have updated the search cursor to include the new, faster "da" approach. Please read up on using cursors here
The script below will print out the value of the fld_name for each row in the table.

Please use the raw input format when trying to identify file locations.

import arcpy
in_table = r"c:/ZZtest/test.mdb/table"
with arcpy.da.SearchCursor(in_table, ["fld_name"]) as scursor:
    for srow in scursor:
        print srow[0]
del srow, scursor

Best Answer

Related Solutions

[GIS] How to check for empty values in fields of a featureclass

[GIS] Very Simple ArcPy script using arcpy.SearchCursor and row.getValue

Related Question