MATLAB: Does accessing a memory-mapped file in MATLAB 7.1 (R14SP3) consume more memory than the size of the file itself

MATLAB

I am using MEMMAPFILE to map a binary file containing structs into MATLAB. The file is 60MB. However, the first time I try to access data from the memory map object, I see in the Windows Task Manager that memory usage increases by 900MB. I would like to know why MATLAB is using so much more memory to access my data file than the actual size of the file.

Best Answer

Let's assume your data file consists of an array of 10,000 structs, and each of those structs has 1,000 fields. Each field of a struct requires header information in order to be accessed by MATLAB which contributes to memory overhead. So, even though the file is 60MB, once it is accessed in MATLAB, all of the struct fields contributes to the memory overhead you observe in Windows Task Manager. There are two possible approaches to deal with this issue:
(1) If the data you extract from the data file are all numbers, this suggests you can create a more memory-efficient binary file for MATLAB that consists of an array of 13229 structs. Each struct has one field, which is a 1x1115 vector of singles.
(2) Another approach is to use a matrix to store the singles. The row index would be equivalent to the struct number, and each row of data would be analogous to the 1x1115 vectors in the previous approach.