MATLAB: Append bug maybe.

appendsave function

In the .mat file, append the variable with the append command, which has the same name as the original variable in the file. Although it is overwritten, the file size is the sum of the two variables, which results in the file getting larger and larger. In fact, multiple variables in the file After saving from the workspace is very small. We think this is a bug and there is no real coverage. wonder if it is or not

Best Answer

This is by design. When you -append, then the .mat file as a whole is not rewritten: instead the place in the .mat file that stored the variable is marked as unused and the new information is put at the end of the file.

If you were to later save a smaller version of the same variable, then the now-empty block in the file might be re-used, but this is not promised.

-append makes no attempt to "compact" the file to remove the hole. It was designed for the situation where small update time was more important than smallest file.

Related Solutions

MATLAB: How to append variables in a specific .mat file in a specific path

matfile = fullfile(PathName, FileName);
save(matfile, 'NameOfVariableToAppend', '-append');

This will typically leave the other variables stored in the file in the same positions they already are in. If the variable to append already exists in the file then it would typically overwrite the existing block for that variable if the storage space required is no larger than it was before, but if the variable to append already exists in the file and the new version is larger then the existing storage block in the file would typically be marked as unused and the new version would be appended to the file. It is not documented as to whether the entire file might ever be rewritten to fill in "holes" if the file gets fragmented from these kinds of updates.

Note that if the variable already exists in the file, then the new data will replace the old data, rather than being added on to the end of it.

MATLAB: MATLAB function “save” and “-V7.3”

Your observation is correct:

With the -v7.3 flag you can store variables > 2GB with compression
Without the -v7.3 flag (e.g. if the default version is set to -v7 or lower) you have no (or less) compression, but cannot store large arrays.

It is surprising, that the compression cannot be disabled manually, because it can be very time-consuming. But this was not the case when the v7.3 format has been developped, because the hard disks have been slower and more expensive. Therefore a compression has been a good idea.

Can you use HDF5 files?

Best Answer

Related Solutions

MATLAB: How to append variables in a specific .mat file in a specific path

MATLAB: MATLAB function “save” and “-V7.3”

Related Question