[GIS] Is it possible to use UTF-8 characters for shapefile attributes

encodingfields-attributesshapefile

I heard that according to a specification shapefile attributes stored in dbf-files cannot have UTF-8 as encoding.
And then I found out that an additional file with the ending CPG can be used to define the code page to identify the character set used in the dbf file.

For me this sounds contradictory.

So my question is now – is it possible to use UTF-8 characters for shapefile attributes?

and – additional question, since encoding and character sets are always an unpopular topic, do You know a page where the topic is described in detail?

Best Answer

Yes, you can use UTF-8 characters in shapefile attributes. The document (http://support.esri.com/en/knowledgebase/techarticles/detail/21106) explains it in detail.

Related Solutions

QGIS – How to Fix Shapefile’s Character Encoding Using ogr2ogr on the Command Line

Quoting from the GDAL documentation for the Esri Shapefile driver:

An attempt is made to read the LDID/codepage setting from the .dbf file and use it to translate string fields to UTF-8 on read, and back when writing. LDID "87 / 0x57" is treated as ISO8859_1 which may not be appropriate. The SHAPE_ENCODING configuration option may be used to override the encoding interpretation of the shapefile with any encoding supported by CPLRecode or to "" to avoid any recoding. (Recoding support is new for GDAL/OGR 1.9.0)

Depending on which GDAL/OGR version you are using, ogr2ogr may be trying to translate your data to UTF-8 or not doing nothing at all.

So you would do either:

ogr2ogr --config SHAPE_ENCODING "UTF-8" -f "MySql" MySql:"basemap,host=127.0.0.1,user=myUser,password=myPass,port=3306" -lco engine=MYISAM "C:/path/to/data/tl_2012_us_county.shp" -nln county -nlt "geometry" -s_srs EPSG:4269 -t_srs EPSG:3857

ogr2ogr --config SHAPE_ENCODING "" -f "MySql" MySql:"basemap,host=127.0.0.1,user=myUser,password=myPass,port=3306" -lco engine=MYISAM "C:/path/to/data/tl_2012_us_county.shp" -nln county -nlt "geometry" -s_srs EPSG:4269 -t_srs EPSG:3857

Finally, check that your MySQL database is using UTF-8 and not Latin-1.

[GIS] Stop automatic ANSI_1252 to UTF-8 encoding when adding a DXF layer

One possible solution to the problem: Open the DXF file in a texteditor and save the file with the encoding that is used by QGIS. In my case I had a DXF file exported from AutoCAD with encoding UTF-8, with Swedish text. I found out that by trial and error that saving the DXF-file with ANSI encoding the text was displayed correctly in QGIS.

I used notepad++ to open the DXF-file and converted the encoding to ANSI and saved the file and then opened the file again in QGIS.

Best Answer

Related Solutions

QGIS – How to Fix Shapefile’s Character Encoding Using ogr2ogr on the Command Line

[GIS] Stop automatic ANSI_1252 to UTF-8 encoding when adding a DXF layer

Related Question