QGIS Tools Changing Special Characters – How to Fix Unreadable Outputs

encodingpyqgisqgis

QGIS tools change special characters (õ, à) of attributes into unreadable characters.

Example:

# load 1 wfs feature
uri = "http://geo.vliz.be/geoserver/MarineRegions/ows?service=WFS&version=1.0.0&request=GetFeature&typeName=MarineRegions:eez&Filter=%3Cogc:Filter%20xmlns:ogc=%22http://www.opengis.net/ogc%22%3E%3Cogc:PropertyIsLike%20matchCase=%22false%22%20wildCard=%22*%22%20singleChar=%22.%22%20escapeChar=%22!%22%3E%3Cogc:PropertyName%3Egeoname%3C/ogc:PropertyName%3E%3Cogc:Literal%3E*Gomera*%3C/ogc:Literal%3E%3C/ogc:PropertyIsLike%3E%3C/ogc:Filter%3E"
vlayer = QgsVectorLayer(uri, "test", "WFS")

# save as shapefile with UTF-8 encoding
QgsVectorFileWriter.writeAsVectorFormat(layer = vlayer, fileName = 'H:/Gomera.shp', fileEncoding = 'utf-8', driverName = 'ESRI Shapefile')

# run qgis tool    
import processing
processing.run("native:fixgeometries", {'INPUT':'H:/Gomera.shp','OUTPUT':'H:/Gomera_fix.shp'})

In the output shapefile, the special characters have been changed into unreadables (compare left to right attribute tables):

Qgis changes special characters

The input shapefile and output shapefiles are both UTF-8 encoded.
This happens with several tools, for example with fixgeometries, buffer or grass7:v.buffer, both in the python console as well as in the GUI.

How can I solve this problem?
This seems something related to internal QGIS tools rather than problems with reading special characters as input as discussed here:

I'm running QGIS 3.0 on Windows.

Best Answer

Use:

orgEncoding=QgsSettings().value('/Processing/encoding') # save setting
QgsSettings().setValue('/Processing/encoding', 'utf-8') # set uft8

# run qgis tool    
import processing
processing.run("native:fixgeometries", {'INPUT':'H:/Gomera.shp','OUTPUT':'H:/Gomera_fix.shp
QgsSettings().setValue('/Processing/encoding', orgEncoding) # set saved value

For set the value manuellenter image description here

Related Question