MATLAB: Does CALLLIB to a C++ library built using Visual C++ cause MATLAB 7.6 (2008a) to crash when used with a .def file

cdecldeclspecdllMATLABstdcall

I am having problems with CALLLIB crashing MATLAB 7.6 (2008a) when calling a C++ DLL. My DLL library worked correctly in MATLAB 7.5 (2007a), but fails in MATLAB 7.6 (R2008a).
To illustrate the problem, consider the simple Visual C++ subroutine that has a single pointer variable:
 __declspec(dllexport) void  mrqversion(double *DLLversion) {
     *DLLversion = 2008.0312;
 }
This, along with many other subroutines, is compiled into a DLL library called moddll.dll. This DLL library can be called in Visual Basic, Excel and Matlab 2007a.
I can define a file moddll_extras.h that contains
 #ifndef _SHELL_H_
 #define _SHELL_H_
 #ifndef EXPORT
 #define EXPORT
 #endif
 EXPORT void mrqversion ( doublePtr DLLVersion);
 #endif
Then in Matlab I can enter
loadlibrary('moddll','moddll_extras.h')
libfunctions('moddll','-full')
Functions in library moddll:
 doublePtr mrqversion(doublePtr)
In MATLAB I perform the following operations
DLLVersion = double(0.0);
ptr_DLLVersion = libpointer('doublePtr',DLLVersion);
calllib('moddll','mrqversion',ptr_DLLVersion);
When using version 2007a, Matlab correctly returns, and I can recover the value using ptr_DLLVersion.Value. When I run this same sequence of commands using Matlab 2008a, I get a segmentation fault.

Best Answer

This is due to a miss-match between how MATLAB is calling the DLL functions, and how the DLL is expecting to be called. Because a .def file was used to export the functions, MATLAB does not know what calling convention to use. MATLAB determines calling convention from the header file, but a .def file bypasses that.
This DLL library was built using Visual C++, and a .def file was used for the exports. This happens when the file is generated using cdecl calling convention (since there is a compiler switch for this, and also the subroutine definition was
__declspec(dllexport) void mrqversion(double *DLLversion).
If you change all the cdecl to stdcall all of the functions in the DLL library would work correctly.
More information about calling conventions is available at:
http://en.wikipedia.org/wiki/Calling_convention
Related Question