[GIS] From text data create a FeatureClass with table with python

arcgis-9.3csvfeature-datasetpythonshapefile

I'm searching how the output of one program (text file with some writings, but also with data as numbers, dates, times, coordinates, all separated with commas) can be transformed to a Feature Dataset.

1) .txt -> .csv -> gp.TabletodBase and then it's nearly done..
But get problems writing csv with Excel dialect – all row is writen in one cell. So, I create the delimiter ';' to get the data into cell by cell.

f= open(rawtable, 'r')
f2 = open(outtable, 'wb')
csv.register_dialect('dotcoma', delimiter=';')
writer=csv.writer(f2, dialect='dotcoma')

for line in itertools.islice(f, 6, 7): #reading labels that are in 6'th row
    labels= line.strip('\n').split(",")
    writer.writerow(labels)

But well, then Arcgis does not recognize the table..
I have been trying to cast the data, like integers, floats, strings.. but that did not help..

2) .txt -> .shp… —-> THAT WORKS!!!

The shapefile does not have a type of time.. So was skiping it.

EDITED
However tried the Pyshp library. But after creating fields blindly I record the tuple of all data in a row and I'm getting an error: IndexError: tuple index out of range..

I simply write fields from the list of data, depending to which group they go:

for fieldname in labelkeys:
    if fieldname in t_int:
        w.field(fieldname, 'N', 8, 0) #1probl was here as I made '8' vs. 8 ! stpd
    ...
    else:
        w.field (fieldname,'C', 25)

Another problem with coordinates. I was writing floats… and it did not like it.. So I had to write integers.
Plus one cannot pass a tuple to w.record().
Thanks to StackExchange I've learned the benefits of asterix 🙂 and one simply has to write w.record(*mylist).

3) .txt -> feature class…
Here I have problems writing the entire row at once..(like the list of values) into a NewRow. Since field names are made automatically I can not for each make like row.fieldname=somevariable. I have tried while looping to SetValues row.setValue(labelkeys[unit], data[unit]) and then at the end of a list rows.InsertRow(row) , but it seemed that it sets just one value. Though I tried by hand and got an error in data casting..

The error comes that I read data like string '34', cast it to int(34) and try to write into field with type 'DOUBLE' (since I grouped some data and wanted to make more generic types).

if labelkeys[unit] in t_int:
    if data[unit]:
        row.setValue(labelkeys[unit], int(data[unit]))

So for now.. nothing is really working..

Maybe there is some other way I have not though about?

If not, could anyone help to get the things work?…

Best Answer

In version 9.3 there is a toolbox called Create Features from Text File, which can do what you require. The help files are quite detailed and are found here.

The following code sample should be simple enough to adapt to reflect your dataset:

# Create geoprocessing dispatch object
import arcgisscripting
gp = arcgisscripting.create()

# Set up inputs to tool
inTxt = r"C:\temp\StreamPoints.txt"
inSep = "." #the symbol used to indicate a decimal point
output = r"C:\temp\Streams.shp"

# Run tool
gp.CreateFeaturesFromTextFile(inTxt, inSep, output, "#")

UPDATE:

If you require numerous attributes of a range of types to be written to a feature dataset from a text file or csv file, it my be easiest to use pyshp. I wrote up an answer to a similar question about pyshp a few days ago and will reproduce the relevant parts of this answer below.

The pyshp/xbase datatyping has always been tricky for me until I found this user guide for the xbase format and as a result of this question I have written a small note on my blog regarding the relevant pyshp datatypes, part of which I have pasted below:

  • C is ASCII characters
  • N is a double precision integer limited to around 18 characters in length
  • D is for dates in the YYYYMMDD format, with no spaces or hyphens between the sections.
  • F is for floating point numbers with the same length limits as N
  • L is for logical data which is stored in the shapefile's attribute table as a short integer as a 1 (true) or a 0 (false). The values it can receive are 1, 0, y, n, Y, N, T, F or the python builtins True and False

There is a DateTime type as well which should allow you to store correctly formatted time data as well using the same syntax as the above datatypes.

Here is an example python script showing the use of the various pyshp datatypes being read in from lists, which could be populated with your data using the csv module:

import shapefile as shp

#Input parameters
out_file = 'GPS_Pts.shp'

#Set up blank lists for data
x,y,id_no=[1.0,2.4,3.9],[4.3,5.5,6.1],[55,16,10]
date,notes = ['20121012','20121012','20121011'],['This','is a','string']
logic = [True,False,True]

#Set up shapefile writer and create empty fields
w = shp.Writer(shp.POINT)
w.autoBalance = 1 #ensures gemoetry and attributes match
w.field('X','F',10,5)
w.field('Y','F',10,5) #float - needed for coordinates
w.field('Date','D') #date
w.field('Notes','C',50) #string
w.field('ID_N','N',12) #double
w.field('Logic','L',1)#boolean

#loop through the data and write the shapefile
for j,k in enumerate(x):
    w.point(k,y[j]) #write the geometry
    w.record(k,y[j],date[j], notes[j], id_no[j], logic[j]) #write the attributes

#Save shapefile
w.save(out_file)
Related Question