Not from within the mex function that I am aware of. When these errors occur in an API library routine, of course the routine doesn't return back to the mex function so there is nothing to catch there. Other that using a try-catch with the caller, I don't know what else you can do directly. For the memory allocation stuff, I suppose you could put in code to see how much contiguous memory is available before calling a mxCreateEtc function, but that would be kind of a pain.
One approach is you could hold your critical variables at the top level of your code so they stay intact when the error occurs, and have top level flags set before & after every API call to keep track of successful progress. Then in your caller catch block you could call back into the mex routine, which would immediately detect that an API function didn't return from a previous call (via your top level flags), and you could then do your clean up.
You can of course register a mexAtExit( ) function for when the mex routine gets cleared from memory, but there isn't anything to catch the runtime errors (memory allocation etc) from within the mex routine itself.
Best Answer