MATLAB: What encoding is used by the file I/O functions such as ‘fileread’

filefilereadfopeni/oMATLAB

I am using the "fileread" function to read a text file. What encoding is used when reading this file?

Best Answer

Starting in MATLAB R2020a, the "fileread" function along with many other high-level and low-level IO functions use automatic character set detection to determine the encoding of the file. The encoding is determined automatically. Note that this encoding detection adds some overhead to the "fileread" operation.

In cases where additional overhead is undesirable, or where the encoding is already known, an alternative workflow is to use "fopen" to specify the encoding of the file and use other low-level IO functions to read the data with this encoding. For example, the following will open a file for reading with UTF-8 encoding:

>> fid = fopen('test.txt', 'r', 'n', 'UTF-8'); % Set encoding to be UTF-8

More information on automatic file encoding can be found in the File I/O documentation:

https://www.mathworks.com/help/releases/R2020a/matlab/import_export/import-text-data-files-with-low-level-io.html#br4sz4c

Related Solutions

MATLAB: What are the limitations on encoding schemes in low-level file I/O functions in MATLAB 7.2 (R2006a)

Support for character encoding schemes has these limitations:

*Surrogate pairs are not supported. Each surrogate pair is read as a replacement character, the equivalent of char(26).

*Stateful character encoding schemes are not supported.

*Byte order marks are not interpreted in any special way. Your code must skip them if necessary.

*Scanning of numbers, using fscanf, is supported only for character encoding schemes that are supersets of ASCII. (Most popular character encoding schemes, with the exception of UTF-16, are such supersets.)

MATLAB: What did they change to ‘fread’ on 2020a

https://www.mathworks.com/help/matlab/release-notes.html?category=data-import-export&rntext=&startrelease=R2020a&endrelease=R2020a&groupby=release&sortby=descending&searchHighlight=

File I/O functions, such as fscanf and fprintf, use automatic character set detection and UTF-8 encoding by default
Behavior change
As of R2020a, character-oriented file I/O functions such as fscanf, fgets, and fgetl trigger automatic character set detection when reading a file that was opened using fopen without a specified encoding.
Similarly, fprintf defaults to using UTF-8 encoding when writing a file that was opened using fopen without a specified encoding

You probably have a large file, and fread() is having to scan through a lot of the file to try to figure out what the character encoding is.

I would suggest that better code for you would be

beg4c = fread(fid, [1 4], 'uint8=>char');

Best Answer

Related Solutions

MATLAB: What are the limitations on encoding schemes in low-level file I/O functions in MATLAB 7.2 (R2006a)

MATLAB: What did they change to ‘fread’ on 2020a

Related Question