MATLAB: How to work with big data

big data

Hi everyone!

Currently I am working using two big data sets. The first one is 6927 kb (already in .mat format) and the second is of size 897 621 KB (in format .csv). The problem is that when I tried to work with them the computer stops working. Does anybody know how to make them less heavy? otherwise (less preferable option) how to slipt it to be able to work with them?

I am working with Matlab R2012 b 64-bit

Thanks a lot for your help!

🙂

Best Answer

Use:

 M = dlmread(filename,delimiter,[R1 C1 R2 C2])

To access the file in chunks.

Related Solutions

MATLAB: How to separate month, day of the week, and hour of the day for each row of a csv time series file so that I can do a factor analysis for month, day, hour

Use a timetable!

https://www.mathworks.com/help/matlab/timetables.html

https://www.mathworks.com/help/matlab/matlab_prog/create-timetables.html

Import your csv file as table and then create a timetable from it .Then you can do whatever you want to do with it, like monthly mean, yearly sum, etc. For example,

data_table = readtable('yourfile.csv');
data_timetable = table2timetable(data_table);

and then you could use retime

https://www.mathworks.com/help/matlab/ref/retime.html

 hourly_mean_table = retime(data_timetable,'hourly','mean')

MATLAB: How to Call a function onto a separate script

Try something like this

TotalMinutes = 123;  % Whatever
[Hours, Minutes, Message] = MyTimeConversion(TotalMinutes)
uiwait(helpdlg(Message));

Best Answer

Related Solutions

MATLAB: How to separate month, day of the week, and hour of the day for each row of a csv time series file so that I can do a factor analysis for month, day, hour

MATLAB: How to Call a function onto a separate script

Related Question