MATLAB: Do I receive an empty data set when I try to transfer huge data ( in order of 1.5 GB) between clients and workers while using Distributed Computing Toolbox 3.2 (R2007b)

Question

I have a huge data set (in the order of 1.5 GB) and I want to do a data transfer between my client machine( WIN XP-64 bit) and workers. When I use DFEVALASYNC to evaluate the datasets and GETALLOUTPUTARGUMENTS to transfer the dataset, it returns an empty structure on the worker's machine. However, I am able to save the data set as a MAT file.

Answer 1

This is an expected behavior when using the Distributed Computing Toolbox 3.2 (R2007b).

The reason for this behavior is that the size limit of data transfers among the distributed computing objects is limited by the Java Virtual Machine (JVM) memory allocation which is 50 MB for R2007b.

As a workaround, upgrade to Parallel Computing Toolbox 3.3 (R2008a) or higher becasue of the increased data transfer limit to 300 MB (on a 32-bit machine) and 2 GB (on a 64-bit machine).