MATLAB: I am unable to perform database operations such as FETCH or EXEC inside the PARFOR LOOP in MATLAB 7.14 R2012a

Parallel Computing Toolbox

I have created a database connection using JDBC driver in my client MATLAB process. I open 8 workers using MATLABPOOL OPEN command and then within the PARFOR loop body, I execute the following code
>>dbConn = database(char(dataSource), char(userID), char(passwd), char(jdbcDriver), char(URL));
>> matlabpool open 8
>>parfor j = 1:numBlocks
>> sql = [ ' and SEQ_NO between ', num2str((j - 1)* maxBlockSize + 1),  ' and ', num2str( (j - 1)* maxBlockSize + re)];
>>curs_a = exec(dbConn, sql);
>>curs = fetch(curs_a);
>>matlabpool close
I get the following error message:
 Starting matlabpool using the 'local' configuration ... connected to 8 labs.
 Warning: com.mathworks.toolbox.database.databaseConnect@24ef2645 is not serializable 
 K>> err.message
 ans = Undefined function 'fetch' for input arguments of type 'struct'.
 Error stack:
 RunDHSimulation>(parfor body) at 119

Best Answer

Within PARFOR LOOP, database operations like exec and fetch are not possible even if database connection object is supplied to these workers.
The main reason is because, MATLABPOOL OPEN , opens new processes (not new functions). The client process (PCT) which holds the database connection object cannot share or send the object to other processes. This is analogous to file handles. In regular programming language like C++, two different C++ applications or processes (not functions) cannot share file handles as they are separate processes or applications in memory. Similarly database connection object which is created in one process (PCT Client process) cannot be shared with worker processes. Workers in PCT are not functions but are separate processes in the operating system.
The ultimate main reason for this is that database connection objects are TCP/IP ports and multiple processes cannot access the same port. That is why database connection object are not serialized. Ports cannot be serialized.
The workaround for this is to establish the database connection within the PARFOR LOOP. This means that each worker will open a new port to the database and communicate to the database through its individual port. But this also means the database will have to bear the overhead of maintaining multiple connections with different processes. This might be a performance issue based on the application.
Related Question