MATLAB: Passing object array from C# to MATLAB

MATLAB Compilermcr

I have a program where I have to basically send a database to from C# code to MATLAB. If I do this using the COM interface, then I just send a giant object[,] array and it works fine. It sends it pretty much instantly. If I am using the MCR with a compiled .dll, then I have to convert this data to a giant MWCellArray. This table is pretty big (65,000 x 365). Converting this to the MWCellArray takes too long. I am using this because the type of data in each cell is not consistent.

Is there an alternative to using a MWCellArray that would allow me to send a large object array with arbitrary data from C# to the MATLAB Runtime?

I am using the compiled runtime for R2017a.

Best Answer

So I think I found a solution but I don't understand why it works. If anyone knows, please let me know.

My old code was:

MWCellArray output = new MWCellArray(rows, columns);
for (int i =0; i < rows; i++) {
    for (int j = 0; j < columns; j++) {
        output[i+1, j+1] = WrapObject<T>(input[i, j]);        
    }
}
return output;

In this code, T might be MWCellArray and an empty MWCellArray would be returned if the input value is null. I think this was a big part of the problem.

Changing WrapObject to return null and using this to this code speeds it up a little:

MWCellArray output = new MWCellArray(rows, columns);
for (int i =0; i < rows; i++) {
    for (int j = 0; j < columns; j++) {
        MWArray value = WrapObject<T>(input[i, j]);
        if (value != null)
            output[i+1, j+1] = value;
    }
}
return output;

But this is what actually works and takes probably 5% as long as the original code:

MWArray[,] cache = new MWArray[rows, columns];
for (int i = 0; i < rows; i++) {
    for (int j = 0; j < columns; j++) {
        MWArray value = WrapObject<T>(input[i, j]);
        if (value != null)
            cache[i, j] = value;
    }
}
MWCellArray output = new MWCellArray(rows, columns);
for (int i =0; i < rows; i++) {
    for (int j = 0; j < columns; j++) {
        MWArray value = cache[i, j];
        if (value != null)
            output[i+1, j+1] = value;
    }
}

Why does building directly on the MWCellArray slow everything down?

Related Solutions

MATLAB: Writing mat file from C++ using API, empty matrix

Some problems:

1) data2 doesn't point to any of your double data ... it points to pointer_to_double values. So using memcpy with this pointer isn't going to copy any of your double data.

2) Even if data2 did point to your double data (it doesn't), the method you are using to allocate your rows of double data does not guarantee that the memory for the rows will be contiguous in memory ... in fact they could be scattered all over in memory. So attempting to copy all of the rows as one contiguous block of data using memcpy could easily seg fault.

I have never been a fan of allocating array data like you are doing. It prevents you from doing anything reliably downstream in your code that expects the data to be contiguous. It is better IMO to just allocate the entire block of memory up front to make sure it is contiguous and then to set up your row pointers accordingly if that is the syntax you really want to use in your code (which btw I am also not a fan of).

3) You didn't delete any of your new allocations (the row pointers and the row data).

4) You don't copy the correct number of bytes in this line:

        memcpy((void*)mxGetPr(pArr), (void*)data2, sizeof(data2));

Remember, data2 is a pointer, not an array. So sizeof(data2) is just the size of a pointer ... it is not the size of the data that it points to. You will need to supply a manual calculation for that argument. (Side point: the conversion to pointer_to_void will happen automatically ... you don't have to explicitly cast that)

-------

E.g., I prefer something like this if you insist on the [ ][ ] syntax in your code:

        double *data1 = new double[rows*columns];  // Ensures data is contiguous
        double **data2 = new double*[rows];
        data2[0] = data1;  // Point to first row
        for (int i = 1; i < rows; i++) {
            data2[i] = data2[i-1] + columns;  // Point to subsequent rows
        }
  
        for (int i = 0; i < rows; i++) {
            for (int j = 0; j < columns; j++) {
                    data2[i][j] = vTest[j];
            }
        }

Then for the memcpy, use data1:

        memcpy( mxGetPr(pArr), data1, sizeof(*data1)*rows*columns );  // Manually calculate #bytes

Then at the end you only have data2 and data1 to delete.

MATLAB: How To change this C++ code to matlab code

function y=AttackerSuccessProbability(q,z)
p=1-q;
lambda=z*q/p;
sum=1;
for k=0:z
    poisson=exp(-lambda);
    for i=1:k
        poisson=poisson*lambda/i;
    end
    sum=sum-poisson*(1-((q/p).^(z-k)));
end
y=sum;
end

This one works.

Best Answer

Related Solutions

MATLAB: Writing mat file from C++ using API, empty matrix

MATLAB: How To change this C++ code to matlab code

Related Question