MATLAB: Working With Very Large Array

Question

For my work I need to create and work with very large arrays of data (on the order of 1,000*400,000) to simulate many simultaneous random walks. I was wondering what would be the best way to go about this, as I run out of memory before I even get close to creating an array of that size.

Obviously I'll need to write it to the disk in some way. I essentially need to create the array one row at a time, creating the next row from the previous one.

Answer 1

Try looking into using a matfile

doc matfile

It allows you to work with a large data set from file rather than from memory. Access is slower, of course, but not massively so, especially for the first index of the array. It gets slower for accessing subsequent dimensions (or at least it did last time I tested it which was a little while ago).