pyqgis – Fixing QVariant Error When Converting DataFrame into Attribute Table Using PyQGIS

Question

I have a df which looks like that:

ID	Happy
0	Very
1	Little

I'm trying to convert it into an attribute table:

headers = [col for col in df.columns]
fieldlist = QgsFields()
fieldlist.append(QgsField(headers[0],QVariant.Int))

for name in headers[1:]:
   fieldlist.append(QgsField(name, QVariant.String))

for i in df.index.to_list():
   featur = QgsFeature()
   newrow = df[headers].iloc[i].tolist()
   featur.setAttributes(newrow)
   sink.addFeature(featur,QgsFeatureSink.FastInsert)

Although I get this error:

Feature could not be written to
Output_layer_9a58cfa4_ee1e_4b97_a6e8_dfc0410a280a: Could not store
attribute "ID": Could not convert value "" to target type

Why do I get this even though I state that the first column (QgsField) has numbers and the others have Strings?

Answer 1

I believe it is because you are passing Pandas datatypes to the attributes of the QgsFeatures.

import pandas as pd

## make a toy dataframe
data = [[0, 'Very'], [1, 'Little']]
df = pd.DataFrame(data, columns = ['ID', 'Happy'])

## check dtypes
df.dtypes

## returns
df.dtypes
ID        int64
Happy    object
dtype: object

Here is a function that will convert int and float Pandas dtypes to native Python types. It may have to be be extended if you have more fields with different types.

def convert_dtype(data):
    """ function to convert pandas data types to native python type """
       
    def conversion(element):
        ## try and except are used because strings (in this context) behave differently  
        ## to ints/floats and do not have a dtype attribute
        try:
            if element.dtype.name == 'int64':
                return int(element)
            elif element.dtype.name == 'float64':
                return float(element)
            else:
                return element
        except:
            return element
       
    return [conversion(x) for x in data]

    
headers=[col for col in df.columns]
fieldlist=QgsFields()
fieldlist.append(QgsField(headers[0],QVariant.Int))
for name in headers[1:]:
   fieldlist.append(QgsField(name,QVariant.String))
for i in df.index.to_list():
   featur=QgsFeature()
   newrow=df[headers].iloc[i].tolist()

   ######
   ## convert the pandas dtypes to python types
   converted = convert_dtype(newrow)
   ######

   ## use the converted types to pass to the feature
   featur.setAttributes(converted)
   sink.addFeature(featur)