MATLAB: How to read and work with a large Excel file in MATLAB

csvdatadatastoreexcellargeMATLABtall

I have huge data saved on a single Excel csv file (approximatly 90 GB). I would like to know how can I read the data from this file. And, what is the maximum file size (csv file) that I can work with?

Best Answer

In general, the amount of data that can be loaded into memory depends on your machine's RAM.
To workaround the memory limitation, MATLAB has a feature called 'datastore'. This feature allows you to read and process your data without loading everything into memory at once.
You can refer to the "tabularTextDatastore" to learn more about 'datastore' for CSV file:
https://www.mathworks.com/help/matlab/ref/matlab.io.datastore.tabulartextdatastore.html
You can also refer to the following documentation for a more general overview about our 'datastore' feature:
https://www.mathworks.com/help/matlab/datastore.html
Additionally, if your workflow also demands manipulating the data as array after you import them as 'datastore', you can leverage another feature called Tall array:
https://www.mathworks.com/help/matlab/tall-arrays.html
Related Question