MATLAB: Do I get an OutOfMemory error when using FREAD and FPRINTF with large amounts of text (ASCII) data

MATLAB

When I try to read a 300 MB text file using FOPEN and FREAD, I get an OutOfMemory error, even though the largest contiguous free block on my machine is shown to be 500 MB.
fId = fopen('largefile.txt', 'rt');
y = fread(fId, 'uint8=>char')';
fclose(fId);
Also, if I try to write 300 MB of data to a text file using FPRINTF, I also get an OutOfMemory error.

Best Answer

The data type "char" is represented internally in MATLAB as Unicode, which means that 2 Byte of memory are needed per character in order to represent the data. To work around this issue, you will either need to have a largest contiguous free block that is at least twice the size of the data that you are loading, or you could read the data as an unsigned char, as follows:
fId = fopen('largefile.txt', 'rt');
y = fread(fId, 'uchar=>uchar')';
fclose(fId);
The function FPRINTF in MATLAB is vectorized, unlike the FPRINTF function in the C programming language. Because of this, there is some extra overhead in parsing the MATLAB data and preparing it before it can be passed to the low-level C-FPRINTF function. This overhead may or may not be reflected in the output of FEATURE MEMSTATS. So in the end, more memory is needed to write a file using FPRINTF than the amount of memory that is used by the data passed to it.
MATLAB is a higher programming language, because of this; it needs to convert the data to a low-level format first, before passing it to a low-level function.
Related Question