MATLAB: How to assign the Test Diagnostic provided by the Unit Testing framework in the Command Window to a variable

test diagnosticunittest

While runnig tests with the Unit Testing framework the defined Test Diagnostic is displayed in the Command Window: ================================================================================ Verification failed in mbd_ZaehlerUnitTest/testTestCase01(UeberlaufAbfangen=off,InternerDatentyp=uint8,Abtastzeit=T_SAMP,ModelCoverage=No).

    ----------------
    Test Diagnostic:
    ----------------
    xxx
    ---------------------
    Framework Diagnostic:
    ---------------------
    verifyEqual failed.

In order to create an overall test report I need to assign this diagnostic text to a string variable. How can I Do this?

Best Answer

Hi Malek,

You can get programmatic access to this information by creating and installing a TestRunnerPlugin for the test run. This plugin can then listen to qualification failed events (like verification failures, assertion failure, etc) and this diagnostic information is contained in the TestDiagnosticResult property of the event data as a cell array of strings.

Here is a quick example that save all of the test diagnostics for failing verifications in test methods. Note this could expand to cover other cases like assertions, assumptions, etc and other scopes like TestClassSetup, but this might help get you started.

classdef TestDiagnosticSaverPlugin < matlab.unittest.plugins.TestRunnerPlugin
      properties
          TestDiagnosticData
      end
      methods (Access = protected)
          function runTestSuite(plugin, pluginData) 
              plugin.TestDiagnosticData = []; % Reinitialize for each test run
              runTestSuite@...
                  matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
          end
          function testCase = createTestMethodInstance(plugin, pluginData)
              testCase = createTestMethodInstance@...
                  matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
              testName = pluginData.Name;
              testCase.addlistener('VerificationFailed', ...
                @(~,event)plugin.storeTestDiagnosticsData(event,pluginData.Name));
          end
      end
      methods (Access = private)
          function storeTestDiagnosticsData(plugin,eventData,name,failureType)
              s.Name = {name};
              s.TestDiagnostics = eventData.TestDiagnosticResult;
              plugin.TestDiagnosticData = [plugin.TestDiagnosticData; struct2table(s)];
          end
      end
  end

With the above plugin you could then install it on a runner:

>> import matlab.unittest.TestRunner;
>> import matlab.unittest.TestSuite;
>> suite = TestSuite.fromFile('mbd_ZaehlerUnitTest.m')
>> runner = TestRunner.withTextOutput;
>> plugin = TestDiagnosticSaverPlugin;
>> runner.addPlugin(plugin);
>> runner.run(suite)
>> plugin.TestDiagnosticData

Hope that helps! Here are a few links you might find interesting to help you do this:

Writing plugins overview

Example of creating a custom plugin

QualificationEventData

Related Solutions

MATLAB: How to pass values from inside a TestRunner run

You can use a TestRunnerPlugin to store data from the test run. I've included some example code below to show how this might work. The basic outline is as follows:

The test logs information using a Diagnostic subclass and TestCase.log
A plugin listens to the DiagnosticLogged event and stores the logged data on itself
After running the test, you can then access the data on the plugin

Example test code:

classdef MyTest < matlab.unittest.TestCase
    properties (ClassSetupParameter)
        idx = num2cell(1:3);
    end
    properties
        MyData;
    end
    methods(TestClassSetup)
        function createData(testCase,idx)
            testCase.MyData = idx;
            testCase.log(4, DataDiagnostic(testCase.MyData));
        end
    end
    methods(Test)
        function a(testCase)
        end
    end
end

Plugin code:

classdef MyPlugin < matlab.unittest.plugins.TestRunnerPlugin
    properties(SetAccess=private)
        Data = {};
    end
    methods(Access=protected)
        function testCase = createTestClassInstance(plugin, pluginData)
            testCase = createTestClassInstance@matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
            testCase.addlistener('DiagnosticLogged', @plugin.logDiagnostic);
        end
    end
    methods(Access=private)
        function logDiagnostic(plugin, src, evd)
            if isa(evd.Diagnostic, 'DataDiagnostic')
                plugin.Data{end+1} = evd.Diagnostic.Data;
            end
        end
    end
end

Diagnostic class code:

classdef DataDiagnostic < matlab.unittest.diagnostics.Diagnostic
    properties(SetAccess=immutable)
        Data;
    end
    methods
        function diag = DataDiagnostic(data)
            diag.Data = data;
        end
        function diagnose(~)
        end
    end
end

Usage:

>> suite = testsuite('MyTest');
>> runner = matlab.unittest.TestRunner.withTextOutput;
>> plugin = MyPlugin;
>> runner.addPlugin(plugin);
>> runner.run(suite);
Running MyTest
...
Done MyTest
__________
>> plugin.Data
ans =
    1×3 cell array
      {[1]}    {[2]}    {[3]}

References:

MATLAB: How to use the unit-test framework in parallel with shared Simulink data dictionaries

The "runInParallel" function does not support all kinds of data access due to its multi-threaded nature. However, Simulink has the "Simulink.data.dictionary.setupWorkerCache" function which can be ran before the data dictionary is modified to ensure each parallel worker is only modifying their copy of the data dictionary.

To embed this function into the test suite, first create a TestRunnerPlugin that calls this function and then calls the superclass to run the actual tests. This way we ensure that the parallel pool has been initialized, the "setupWorkerCache" function called, and then the actual tests are ran. Save the below code in a file called "CacheSetupPlugin.m":

classdef CacheSetupPlugin < matlab.unittest.plugins.TestRunnerPlugin
    methods (Access = protected)
        %%overload the superclass' method
              function runTestSuite(plugin, pluginData)
                  Simulink.data.dictionary.setupWorkerCache
                    % Once setup is complete, invoke super class method to run the test suite
                    runTestSuite@matlab.unittest.plugins.TestRunnerPlugin(plugin, pluginData);
                    Simulink.data.dictionary.cleanupWorkerCache   
                end
      end
      methods (Hidden)
          function bool = supportsParallel(~) 
             bool = true; 
          end
      end
  end

Now run the test suite like so:

>> suite = matlab.unittest.TestSuite.fromClass(?testDataDictionaryInteractions)
>> runner = matlab.unittest.TestRunner.withTextOutput()
>> runner.addPlugin(CacheSetupPlugin)
>> result = runInParallel(runner,suite)

See the following links for added information:

https://www.mathworks.com/help/matlab/matlab_prog/create-custom-plugin.html

https://www.mathworks.com/help/matlab/matlab_prog/customizing-a-testrunner-with-plugins.html

Best Answer

Related Solutions

MATLAB: How to pass values from inside a TestRunner run

MATLAB: How to use the unit-test framework in parallel with shared Simulink data dictionaries

Related Question