attribute-joinscsvqgis
Using QGIS I've joined tabular text data (all in fields of less than 255 characters) to a shapefile via a CSV file. All looks good in the joined table but when I save to a new shapefile to make the join permanent the long text fields get truncated to between 50-100 characters. How can I avoid this? It seems the problem is in the saving/exporting process since the initial join is fine.
I'm running QGIS 1.7.0 on Mac 10.6.8
Thanks,
Matt
You do not write which QGIS version you are running. QGIS Lisboa has a bug in encoding utf-8 data, while earlier and newer version should handle it right.
You may find some workarounds here:
How to read Greek fonts (ISO-8859-7) in shapefile attributes within QGIS 1.8.0?
It's best if you make sure the two fields have the same data type.
The field type can't be changed for shapefiles. But it's easy to create a new empty text field and make that equal to the integer field.
You can use a field calculator expression that looks something like:
tostring( "INT_FIELD" ) and simply replace INT_FIELD with the name of the actual integer field.
You can easily change a CSV layer attribute value from text <-> number too.
If you are getting null values, you could try keeping the join, but just close and re-open that attribute table to see if the joined values are there (or even close/re-open QGIS).
I've been able to successfully join a csv to a shapefile with a mixture of field types so string vs integer shouldn't be the issue.
Looks like in order to get rid of the null values, they have to actually have a matching record in the corresponding table that joins are being made from. But in the image below, you can see that some do have a matching record:
Best Answer
The problem lies with the OGR library used by QGIS to read the data from the CSV file. It does not attempt to automatically determine the datatype and length of each field in the file like some software, instead relying on an accompanying .csvt file described in the OGR CSV page.
In the absence of a .csvt file, my guess is that OGR reports string field sizes as a default of 80 characters meaning that when a shapefile is saved after joining to a CSV, the shapefile's fields corresponding to those derived from the CSV will be set to this length, and text values longer than this will be truncated.
If a .csvt file is created for the CSV file to be joined then the field sizes are correctly read and passed through to the shapefile following joining, and text truncation is avoided.
(Tested on QGIS 1.8.0)