MATLAB: Fail Reasons exportable in the UnitTest-Framework

MATLABunittest

Hi,

when I run unit tests with the unittest framework.

The result of a test run is a vector of matlab.unittest.TestResult objects.

The objects include an information, if a test fails, but do not contain the information about the failure.

If there is a failure, I want to give this failure information to another application.

Therefore my question: Are the fail reasons exportable in the UnitTest-Framework?

thanks

Best Answer

Now with R2014a, you can use custom test runner plugins such as the Failure Diagnostics Plugin and the Output Stream or To File plugin to gather the failure reasons and export them.

http://www.mathworks.com/help/releases/R2014a/matlab/ref/matlab.unittest.plugins.html

Related Solutions

MATLAB: How to publish the “matlab.unittest.TestResult” objects to a PDF report

One solution is to create a plugin, "TestReportPlugin", that directs the "TestRunner" to produce a test result report in PDF file format. Please refer to the following link for more information on using the plugin, "TestReportPlugin":

https://www.mathworks.com/help/releases/R2016b/matlab/ref/matlab.unittest.plugins.testreportplugin-class.html

Adding the following lines before running the tests produces the test results report in PDF file format:

>> pdfFile = 'MyTestReport.pdf';
>> pluginPdf = TestReportPlugin.producingPDF(pdfFile,...
    'IncludingPassingDiagnostics',true,'IncludingCommandWindowText',true);
>> runner.addPlugin(pluginPdf);

Also, make sure that the "TestReportPlugin" is imported.

MATLAB: How to assign the Test Diagnostic provided by the Unit Testing framework in the Command Window to a variable

Hi Malek,

You can get programmatic access to this information by creating and installing a TestRunnerPlugin for the test run. This plugin can then listen to qualification failed events (like verification failures, assertion failure, etc) and this diagnostic information is contained in the TestDiagnosticResult property of the event data as a cell array of strings.

Here is a quick example that save all of the test diagnostics for failing verifications in test methods. Note this could expand to cover other cases like assertions, assumptions, etc and other scopes like TestClassSetup, but this might help get you started.

classdef TestDiagnosticSaverPlugin < matlab.unittest.plugins.TestRunnerPlugin
      properties
          TestDiagnosticData
      end
      methods (Access = protected)
          function runTestSuite(plugin, pluginData) 
              plugin.TestDiagnosticData = []; % Reinitialize for each test run
              runTestSuite@...
                  matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
          end
          function testCase = createTestMethodInstance(plugin, pluginData)
              testCase = createTestMethodInstance@...
                  matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
              testName = pluginData.Name;
              testCase.addlistener('VerificationFailed', ...
                @(~,event)plugin.storeTestDiagnosticsData(event,pluginData.Name));
          end
      end
      methods (Access = private)
          function storeTestDiagnosticsData(plugin,eventData,name,failureType)
              s.Name = {name};
              s.TestDiagnostics = eventData.TestDiagnosticResult;
              plugin.TestDiagnosticData = [plugin.TestDiagnosticData; struct2table(s)];
          end
      end
  end

With the above plugin you could then install it on a runner:

>> import matlab.unittest.TestRunner;
>> import matlab.unittest.TestSuite;
>> suite = TestSuite.fromFile('mbd_ZaehlerUnitTest.m')
>> runner = TestRunner.withTextOutput;
>> plugin = TestDiagnosticSaverPlugin;
>> runner.addPlugin(plugin);
>> runner.run(suite)
>> plugin.TestDiagnosticData

Hope that helps! Here are a few links you might find interesting to help you do this:

Writing plugins overview

Example of creating a custom plugin

QualificationEventData

Best Answer

Related Solutions

MATLAB: How to publish the “matlab.un​ittest.Tes​tResult” objects to a PDF report

MATLAB: How to assign the Test Diagnostic provided by the Unit Testing framework in the Command Window to a variable

Related Question

MATLAB: How to publish the “matlab.unittest.TestResult” objects to a PDF report