MATLAB: Using tall arrays without parallel computing toolbox

big datalicense errorMATLABparallel computingParallel Computing Toolboxtall arrays

I am using tall arrays to read in a large dataset in MATLAB. When I call the tall() function, it automatically tries to start a parallel pool, for example:

A = [2,1;1,3];
B = tall(A);
Starting parallel pool (parpool) using the 'local' profile ...

But there is some issue with my parallel computing toolbox license, so this causes MATLAB to crash. Is there some way I can use tall arrays without starting a parallel pool? Thanks in advance!

Best Answer

https://www.mathworks.com/help/parallel-computing/run-tall-arrays-on-a-parallel-pool.html

If you want to turn off automatically opening a parallel pool, change your parallel preferences. If you turn off the Automatically create a parallel pool option, then you must explicitly start a pool if you want the tall function to use it for parallel processing. See Specify Your Parallel Preferences.

Related Solutions

MATLAB: Tall array assignment with tall logical vector

The documentation does not match the actual implementation by MathWorks. The error is generated at line 32 of fullfile(matlabroot, '\toolbox\matlab\bigdata\+matlab\+bigdata\+internal\+adaptors\GeneralArrayParenIndexingMixin.m') after the simple test if istall(b) (b being the assignor). So it is generated as soon as you try to assign a tall array regardless of whether or not its size matches that of the destination.

This is worthy of a bug report to Mathworks.

With regards to datetime, the input format can include a timezone, but you also have to specify which timezone you want the datetime in:

datetime('2016-12-31 00:01:00 UTC', 'InputFormat', 'yyyy-MM-dd HH:MM:ss z', 'TimeZone', 'UTC')

MATLAB: Accumarray with tall arrays

You can use findgroups and splitapply to do this, like so:

% Make some example data
tt = tall(table(datetime(2017, 03, randi([1 31], 100, 1)), ...
    rand(100, 1), 'VariableNames', {'Date', 'Value'}));
% group by date
g = findgroups(tt.Date);
% Call splitapply to find the sum of values on each unique date
sumAndDate = splitapply(@(v, d) {sum(v), d(1)}, tt.Value, tt.Date, g)

Note that because findgroups for tall arrays doesn't support the second output argument, I've concocted a slightly unusual function for the splitapply stage which returns both the sum and the datetime corresponding to that group.

Best Answer

Related Solutions

MATLAB: Tall array assignment with tall logical vector

MATLAB: Accumarray with tall arrays

Related Question