MATLAB: Mex usage efficiency question

mexpcl

Hi, I am new to mex. im using mex with PCL (Point Cloud Library). what i want to do is calculate different features of the point cloud. for each feature calculation i have a different mex file.

my question is which of the following two options is more efficient:

Give the mex as input a file name and have it open that file and create the PCL cloud class (the basic structure to do anything) in each of the mexs.
Open the file in MATLAB (or a different mex), store the point data in a MATLAB matrix and pass this matrix to each of the other mex files (and create the cloud class in each mex using this data).

so i guess my question is what would probably be more efficient – reading a file to create a class or manually creating a class every time ?

Best Answer

You don't say definitely whether or not you will need the PCL cloud class data in a MATLAB variable for some reason (maybe some calculations based on m-code). My comments below assume you don't really need the PCL cloud class data on the MATLAB side.

(1) Give the mex as input a file name and have it open that file and create the PCL cloud class (the basic structure to do anything) in each of the mexs.

This approach allows you to build the PCL cloud class in native C++ using presumably already available code. The downside of doing it exactly as you propose, using multiple mex files, is that you need to duplicate the reading and creating of the PCL cloud class for each separate mex routine. I prefer this approach over (2) below, but would change the approach to only one mex routine (see last comment).

(2) Open the file in MATLAB (or a different mex), store the point data in a MATLAB matrix and pass this matrix to each of the other mex files (and create the cloud class in each mex using this data).

The approach requires you to write new code to create a semblance of the PCL cloud class in a MATLAB variable. And each time you want to calculate something with this variable in a mex routine, you will need to copy all of it into its native C++ format first. This is not going to be efficient, and requires new code.

"... for each feature calculation i have a different mex file ..."

If you are looking for efficiency, then I would not do this. Rather, I would have one mex routine with all of the code in it for all of the calculations I wanted to do. When calling the mex routine, pass in an argument that specifies which function you want to do. E.g., this argument could be a string. If you passed in 'create' then it would read a file to create the PCL class variable and store it in a persistent memory block. Other strings would cause calculations to be performed on that PCL class variable. Passing in 'delete' would delete the PCL class variable. Etc. The advantage of this approach is that you only need to read in and create the PCL class variable once, only one copy of it exists in memory, and you can calculate different functions on it without re-reading it into memory. You would need to register a mexAtExit function to automatically do the 'delete' function if the mex file itself was cleared from memory.

Related Solutions

MATLAB: How to store a complex number on a binary file

OK, here are the mex files. There are two separate files, one for writing a single variable (numeric or char or logical) to disk, and a separate file for reading the variable from the disk. The real and imag parts are written to two separate files to keep the file sizes down. There is a fair amount of error checking in them, but not a lot of testing so I won't claim they are foolproof yet. The advantage of these mex files is that they don't make copies of your data to do the file writing and they put the real and imag parts in separate files. So they shouldn't blow your memory and hopefully the file sizes will be small enough for your purposes. These are intended to be used on the same machine and same MATLAB version for writing and reading (otherwise there may be variable size or endian issues to deal with). To use them:

- Save writing file as writevar.c somewhere on the MATLAB path

- Save reading file as readvar.c in the same directory as writevar.c

- Change your default directory to this directory

- Issue the following commands at the MATLAB prompt:

mex writevar.c
mex readvar.c

If you get an interactive question popping up asking you about compilers, just enter Y (let MATLAB search for them) and then enter the number of a C/C++ compiler that shows up in the list (e.g., lcc).

32-bit MATLAB ships with the lcc compiler, but if you are running a 64-bit version of MATLAB then you would have to get a 64-bit C compiler installed before you can compile the mex files.

writevar.c
----------
// writevar.c writes a variable to file(s), one each for real and imag
// Companion file to readvar.c
// Syntax: writevar(filename,var)
//         filename = char string containing base filename
//         var = full numeric or logical or char variable
// Note: writes real part to filename_r.bin, and imag part to file filename_i.bin
// Programmer: James Tursa
// Date: Feb 7, 2013
// Building:  mex writevar.c
#include "mex.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef MWSIZE_MAX
#define  mwSize    int
#endif
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  FILE *fp;
  char *filename;
  short *sp;
  mwSize i, nf, nelements, ndim;
  mwSize *dims;
  size_t elements_written, element_size;
  void *data_ptr;
  mxClassID classid;
  if( nrhs != 2 ) {
    mexErrMsgTxt("Need exactly two inputs ... (filename,variable)");
  }
  if( nlhs ) {
    mexErrMsgTxt("Too many outputs.");
  }
  if( !mxIsChar(prhs[0]) || mxIsEmpty(prhs[0]) ) {
    mexErrMsgTxt("1st argument must be base filename string.");
  }
  if( !(mxIsNumeric(prhs[1]) || mxIsLogical(prhs[1]) || mxIsChar(prhs[1])) || mxIsSparse(prhs[1]) ) {
    mexErrMsgTxt("2nd argument must be full numeric or logical or char variable.");
  }
  nf = mxGetNumberOfElements(prhs[0]);
  filename = mxMalloc(nf+7);
  sp = mxGetData(prhs[0]);
  for( i=0; i<nf; i++ ) {
    filename[i] = *sp++;
    if( filename[i] == 0 ) {
      mexErrMsgTxt("Null (0) character detected in filename");
    }
  }
  strcpy(filename+nf,"_r.bin");
  mexPrintf("Opening Real File: %s\n",filename);
  fp = fopen(filename, "wb");
  if( !fp ) {
    mexErrMsgTxt("File for real part will not open.");
  }
  classid = mxGetClassID(prhs[1]);
  element_size = mxGetElementSize(prhs[1]);
  nelements = mxGetNumberOfElements(prhs[1]);
  data_ptr = mxGetData(prhs[1]);
  ndim = mxGetNumberOfDimensions(prhs[1]);
  dims = mxGetDimensions(prhs[1]);
  mexPrintf("Writing real part to file ...\n");
  fwrite( &classid, sizeof(classid),  1, fp );
  fwrite( &element_size, sizeof(element_size),  1, fp );
  fwrite( &nelements, sizeof(nelements),  1, fp );
  fwrite( &ndim, sizeof(ndim),  1, fp );
  fwrite( dims, sizeof(*dims), ndim, fp );
    elements_written = fwrite( data_ptr, element_size, nelements, fp );
    fclose(fp);
  if( elements_written != nelements ) {
    mxFree(filename);
    mexErrMsgTxt("ERROR: Not all elements were written to file.");
  }
  if( mxIsComplex(prhs[1]) ) {
    filename[nf+1] = 'i';
      mexPrintf("Opening Imag File: %s\n",filename);
      fp = fopen(filename, "wb");
      if( !fp ) {
      mxFree(filename);
        mexErrMsgTxt("File for imag part will not open.");
      }
      data_ptr = mxGetImagData(prhs[1]);
      mexPrintf("Writing imag part to file ...\n");
      fwrite( &classid, sizeof(classid),  1, fp );
      fwrite( &element_size, sizeof(element_size),  1, fp );
      fwrite( &nelements, sizeof(nelements),  1, fp );
      fwrite( &ndim, sizeof(ndim),  1, fp );
      fwrite( dims, sizeof(*dims), ndim, fp );
        elements_written = fwrite( data_ptr, element_size, nelements, fp );
        fclose(fp);
      if( elements_written != nelements ) {
        mexErrMsgTxt("ERROR: Not all elements were written to file.");
      }
  }
  mxFree(filename);
}
readvar.c
---------
// readvar.c reads a variable from file(s), one each for real and imag
// Companion file to writevar.c
// Syntax: var = readvar(filename)
//         filename = char string with base filename
// Note: reads real part from filename_r.bin, and imag part from file filename_i.bin
//       gives an error if the files do not contain exactly same number and type of data
// Programmer: James Tursa
// Date: Feb 7, 2013
// Building: mex readvar.c
#include "mex.h"
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef MWSIZE_MAX
#define  mwSize    int
#endif
void mexFunction(int nlhs, mxArray *plhs[], int nrhs, const mxArray *prhs[])
{
  FILE *fp;
  char *filename;
  short *sp;
  mwSize i, nf, nelements_r, nelements_i, ndim_r, ndim_i;
  mwSize element_size_r, element_size_i;
  mwSize *dims_r, *dims_i;
  size_t elements_read;
  void *data_ptr;
  mxClassID classid_r, classid_i;
  if( nrhs != 1 ) {
    mexErrMsgTxt("Need exactly one input ... (filename)");
  }
  if( nlhs > 1 ) {
    mexErrMsgTxt("Too many outputs.");
  }
  if( !mxIsChar(prhs[0]) ) {
    mexErrMsgTxt("Argument must be base filename string.");
  }
  nf = mxGetNumberOfElements(prhs[0]);
  filename = mxMalloc(nf+7);
  sp = mxGetData(prhs[0]);
  for( i=0; i<nf; i++ ) {
    filename[i] = *sp++;
    if( filename[i] == 0 ) {
      mexErrMsgTxt("Null (0) character detected in filename");
    }
  }
  strcpy(filename+nf,"_r.bin");
  mexPrintf("Opening Real File: %s\n",filename);
  fp = fopen(filename, "rb");
  if( !fp ) {
    mexErrMsgTxt("File for real part will not open.");
  }
  mexPrintf("Reading real part from file ...\n");
  fread( &classid_r, sizeof(classid_r),  1, fp );
  fread( &element_size_r, sizeof(element_size_r),  1, fp );
  fread( &nelements_r, sizeof(nelements_r),  1, fp );
  fread( &ndim_r, sizeof(ndim_r),  1, fp );
  dims_r = mxMalloc(ndim_r*sizeof(*dims_r));
  fread( dims_r, sizeof(*dims_r), ndim_r, fp );
  plhs[0] = mxCreateNumericArray( ndim_r, dims_r, classid_r, mxREAL );
  data_ptr = mxGetData(plhs[0]);
    elements_read = fread( data_ptr, element_size_r, nelements_r, fp );
    fclose(fp);
  if( elements_read != nelements_r ) {
    mxFree(filename);
    mexErrMsgTxt("Not enough values were read from the real file");
  }
  filename[nf+1] = 'i';
  mexPrintf("Opening Imag File: %s\n",filename);
  fp = fopen(filename, "rb");
  mxFree(filename);
  if( !fp ) {
    mexPrintf("No Imag File ... returning real variable.\n");
    mxFree(dims_r);
    return;
  }
  mexPrintf("Reading imag part from file ...\n");
  fread( &classid_i, sizeof(classid_i),  1, fp );
  fread( &element_size_i, sizeof(element_size_i),  1, fp );
  fread( &nelements_i, sizeof(nelements_i),  1, fp );
  fread( &ndim_i, sizeof(ndim_i),  1, fp );
  dims_i = mxMalloc(ndim_i*sizeof(*dims_i));
  fread( dims_i, sizeof(*dims_i), ndim_i, fp );
  if( classid_i != classid_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag classid");
  }
  if( element_size_i != element_size_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag element size");
  }
  if( nelements_i != nelements_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag number of elements");
  }
  if( ndim_i != ndim_r ) {
    mexErrMsgTxt("Mismatch Real vs Imag number of dimensions");
  }
  for( i=0; i<ndim_i; i++ ) {
      if( dims_i[i] != dims_r[i] ) {
      mxFree(dims_i);
      mxFree(dims_r);
        mexErrMsgTxt("Mismatch Real vs Imag dimensions");
      }
  }
  mxFree(dims_i);
  mxFree(dims_r);
  data_ptr = mxMalloc(nelements_i*element_size_i);
    elements_read = fread( data_ptr, element_size_i, nelements_i, fp );
    fclose(fp);
  if( elements_read != nelements_i ) {
    mxFree(filename);
    mexErrMsgTxt("Not enough values were read from the imag file");
  }
  mxSetImagData(plhs[0],data_ptr);
}

MATLAB: Issue: MATLAB crashes running mex function at inconsistent points in function.

I haven't looked at your code in detail, but I do have a couple of comments based on a cursory look.

1) It appears to me that your architecture is set up so that the only place the new object handle is kept is in a returned MATLAB variable (uint64). This puts a heavy reliance on the user to NEVER lose that variable. If they did then your routine could not recover. It is locked and there is no way to unlock it because there is no way to pass in a valid handle (via the uint64) to get it unlocked. I would advise keeping track of that handle in your mex routine (e.g., a top level variable) so that if you need to clear the mex function for some reason and you don't have the proper uint64 to pass in, you can still do so (e.g., add another input command so that the mex function can delete the object on its own and unlock itself).

2) It looks like this loop is intended to copy data from an mxArray variable into a C++ vector:

  for (long int nn=0; nn<NRow; nn++){
    for (long int mm=0; mm<NCol; mm++){
      Vec.push_back((double)mxGetPr(Cons_prhs)[0]);
                      mxGetPr(Cons_prhs)[0]+=1;
    }
  }

But what looks to me like your "advance to next value" logic doesn't make sense. mxGetPr(Cons_prhs)[0] is the double value itself, not some pointer. So you end up incrementing the first element of the original mxArray by 1 each iteration, rather than stepping through the double elements. I think you meant something more like this instead:

        double *pr = mxGetPr(Cons_prhs);
    for (long int nn=0; nn<NRow; nn++){
      for (long int mm=0; mm<NCol; mm++){
        Vec.push_back(*pr++);
      }
    }

3) This memory copy doesn't look right either:

mxArray * IntVector2MX(const std::vector<int>& v){
    mxArray *mx;
    mexPrintf("\n IntVector2MX: Pointer defined.");
    mx = mxCreateNumericMatrix(1, v.size(),mxINT32_CLASS, mxREAL);
    mexPrintf("\n IntVector2MX: Numeric array created.");
    std::copy(v.begin(), v.end(), mxGetPr(mx));

You are attempting to copy 32-bit integers, but mxGetPr returns a (double *) type, not a (int *) type. So I think you will end up calling the wrong copy method because it thinks it needs to copy into double memory. I can easily see this writing off the end of the array and crashing. Try this instead:

      std::copy(v.begin(), v.end(), (int *)mxGetData(mx));

Best Answer

Related Solutions

MATLAB: How to store a complex number on a binary file

MATLAB: Issue: MATLAB crashes running mex function at inconsistent points in function.

Related Question