MATLAB: Dynamic allocation of mxArray structure

MATLABmexmxreallocmxsetmmxsetn

Hi everyone,
I have a mex function in C that internally generates an unpredictably large volume of data to be output back to Matlab. I would like the return to be a Struct Matrix of the 1 by N, where N is the unknown large number. The memory should be allocated by chunks, so that in case of insufficient memory the calculation can be stopped and the return that does fit into memory handled safely back to Matlab.
It seems that the combination of mxSetN() and mxRealloc() is a proper combination, but I can't find sufficient info on how to use mxRealloc to allocate space for additional elements of structural matrix.
Here is a minimal code example of what I'm trying to do
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[]) 
{
  mwSize i;
  mxArray *mxp;
    nlhs[0]=mxCreateStructMatrix(1,1,0,NULL); //structure with one element and no fields for now
    while (continue calculation)
    { 
      //one more round of calculations that creates a lot of output
      //stored in structure g with double* field1,..,K
      mxAddField(nlhs[0],"field1");
      mxp=mxCreateDoubleMatrix(appropriate,size,mxREAL);
      if (mxp==NULL) {mexErrMsgTxt("Failed to allocate memory!")}
      mxSetField(nlhs[0],i,"field1",mxp);
      ...
      mxAddField(nlhs[0],"fieldK");
      mxp=mxCreateDoubleMatrix(appropriate,size,mxREAL);
      if (mxp==NULL) {mexErrMsgTxt("Failed to allocate memory!")}
      mxSetField(nlhs[0],i,"fieldK",mxp);
      i++;
      mxSetM(nlhs[0],i)
      //What then?
    }
  }

Best Answer

Letting an array grow iteratively consumes a lot of resources. Therefore it is a good idea to create the new data in chunks and reply them separately.
As James said already, mxCreateDoubleMatrix stops the Mex function immediately if it is not successful. Then all formerly creatzed data are lost. A workaround would be to call the Mex function for each junk in a loop and collect the data in Matlab:
allChunk = cell(1, 1000);
iChunk   = 0;
while ~ready
  try
    newChunk = myMexfunction();
    iChunk   = iChunk + 1;
    allChunk{iChunk} = newChunk;
    if iChunk == numel(allChunk)      % Container exhausted
       allChunk{iChunk + 1000} = [];  % Expand
    end
  catch
    disp('Memory exhausted');
    ready = true;
  end
end
But if te MEX function could not allocate any memory anymore, Matlab is in danger to crash also. You cannot do a lot of computations anymore, most of all you cannot combine the chunks to a large array.
Perhaps it is better to write the junks to the disk (SSD preferrably), such that an exhaustion is less likely and the memory is not filled at all. This would be much better than a strategy which is expected to cause an out-of-memory crash.
Related Question