IEC Certification Kit

Simulink® Design Verifier™

ISO 26262 Tool Qualification Package

R2015a
How to Contact MathWorks
Latest news: www.mathworks.com
Sales and services: www.mathworks.com/sales_and_services
User community: www.mathworks.com/matlabcentral
Technical support: www.mathworks.com/support/contact_us
Phone: 508-647-7000

The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA 01760-2098
IEC Certification Kit: Simulink® Design Verifier™ ISO 26262 Tool Qualification Package
© COPYRIGHT 2011–2015 by The MathWorks, Inc.
The software described in this document is furnished under a license agreement. The software may be used or copied only under
the terms of the license agreement. No part of this manual may be photocopied or reproduced in any form without prior written
consent from The MathWorks, Inc.
FEDERAL ACQUISITION: This provision applies to all acquisitions of the Program and Documentation by, for, or through the
federal government of the United States. By accepting delivery of the Program or Documentation, the government hereby agrees
that this software or documentation qualifies as commercial computer software or commercial computer software documentation
as such terms are used or defined in FAR 12.212, DFARS Part 227.72, and DFARS 252.227-7014. Accordingly, the terms and
conditions of this Agreement and only those rights specified in this Agreement, shall pertain to and govern the use, modification,
reproduction, release, performance, display, and disclosure of the Program and Documentation by the federal government (or
other entity acquiring for or through the federal government)and shall supersede any conflicting contractual terms or conditions.
If this License fails to meet the government’s needs or is inconsistent in any respect with federal procurement law, the
government agrees to return the Program and Documentation, unused, to The MathWorks, Inc.

Trademarks
MATLAB and Simulink are registered trademarks of The MathWorks, Inc. See www.mathworks.com/trademarks for a
list of additional trademarks. Other product or brand names may be trademarks or registered trademarks of their respective
holders.

Patents
MathWorks products are protected by one or more U.S. patents. Please see www.mathworks.com/patents for more
information.
Revision History
March 2011 New for Version 1.3.1 (Applies to Release 2010bSP1)
April 2011 Revised for Version 1.4 (Applies to Release 2011a)
September 2011 Revised for Version 2.0 (Applies to Release 2011b)
March 2012 Revised for Version 2.1 (Applies to Release 2012a)
September 2012 Revised for Version 3.0 (Applies to Release 2012b)
March 2013 Revised for Version 3.1 (Applies to Release 2013a)
September 2013 Revised for Version 3.2 (Applies to Release 2013b)
March 2014 Revised for Version 3.3 (Applies to Release 2014a)
October 2014 Revised for Version 3.4 (Applies to Release 2014b)
March 2015 Revised for Version 3.5 (Applies to Release 2015a)
Contents
1 Introduction ...................................................................................................................................... 1-1
1.1 Project Identification ............................................................................................................... 1-2
1.2 Tool Overview and Identification ........................................................................................... 1-3
1.3 Tool Qualification Artifacts Summary .................................................................................... 1-4
2 Software Tool Criteria Evaluation Report ........................................................................................ 2-1
2.1 Tool Environment ................................................................................................................... 2-2
2.2 Tool Configuration .................................................................................................................. 2-3
2.3 Reference Workflow ............................................................................................................... 2-4
2.4 Tool Use Cases ........................................................................................................................ 2-5
[SLDV_UC1] Generating Test Cases That Satisfy Structural Coverage Objectives ................... 2-5
[SLDV_UC2] Generating Customized Test Cases ...................................................................... 2-5
[SLDV_UC3] Applying the Generated Test Cases ...................................................................... 2-5
2.5 Generic Tool Classification ..................................................................................................... 2-6
2.5.1 Potential Malfunctions or Erroneous Outputs ................................................................ 2-6
[SLDV_E1] Test Case Generation – False Negative .............................................................. 2-6
[SLDV_E2] Test Case Generation – False Positive ................................................................ 2-6
[SLDV_E3] Test Case Generation – Non Interference Error .................................................. 2-6
[SLDV_E4] Test Case Generation – Usage of Incorrect Tool Inputs ..................................... 2-6
[SLDV_E5] Test Case Generation – Misinterpretation of Tool Outputs ................................ 2-6
[SLDV_E6] Test Case Generation – Incorrect Tool Usage .................................................... 2-7
[SLDV_E7] Simulink Design Verifier- Incorrect or Modified Installation ............................ 2-7
2.5.2 Error Prevention and Detection Measures ..................................................................... 2-7
[M1] Usage of Compatibility Checker .................................................................................... 2-7
[M2] Usage of Model Coverage to Assess Completeness and Adequacy of Generated Test
Cases ....................................................................................................................................... 2-7
Tool Classification Summary .................................................................................................. 2-8
3 Software Tool Qualification Report ................................................................................................. 3-1
3.1 Requirement for Tool Qualification ........................................................................................ 3-2
3.2 Voluntary Tool Qualification .................................................................................................. 3-3
4 Confirmation Review of Tool Classification and Qualification ....................................................... 4-1
4.1 Requirement for Confirmation Review ................................................................................... 4-2
4.2 Validity of Generic Tool Classification................................................................................... 4-3
4.3 Validity of Generic Tool Qualification ................................................................................... 4-4
4.4 Conformance with Reference Workflow ................................................................................. 4-5

v
vi
1 Introduction

This document constitutes the ISO 26262 Tool Qualification Package for the Simulink ® Design
Verifier™ product. This document is intended for use in the ISO 26262 tool classification and
qualification process for software tools. It contains templates for the ISO 26262 tool
qualification work products (see ISO 26262-8, Section 11).

The applicant shall review the templates for applicability to the project under consideration, and
then tailor and complete them as necessary.

See also:

 IEC Certification Kit: User’s Guide, R2015a

 ISO 26262-8, Section 11

ISO 26262-8, Clause 11 provides provisions for software tools that are used to tailor activities or
tasks required by ISO 26262. The standard outlines a two-step approach to establish the
required confidence in the tools:

 Tool classification determines the required level of confidence in the software tool.
 Depending on the result of the tool classification, you might need to carry out a formal
tool qualification.

When applying this approach to a software tool, the applicant must create the following work
products (see ISO 26262-8, 11.5):

 A software tool criteria evaluation report documenting the tool classification.

 A software tool qualification report documenting the tool qualification, if required.

Note The applicant needs to review this template for applicability to the project under
consideration and insert missing information.
1.1 Project Identification
Applicant: <Insert information>
Project under consideration: <List project under consideration>

1-2
1.2 Tool Overview and Identification
Simulink Design Verifier allows users to generate test cases for Simulink ® models. The
generated test cases provide simulation inputs that exercise functionality captured in the model
structure and specified by the test objectives. The test cases, together with the test objectives, are
used to verify the model or code running in software-in-the-loop (SIL) or processor-in-the-loop
(PIL) modes.

Tool Identification
Software Tool Version (Release) Tool Vendor

Simulink Design Verifier 2.8 (R2015a) The MathWorks, Inc.

3 Apple Hill Drive
Natick, MA,
01760-2098 USA

1-3
1.3 Tool Qualification Artifacts Summary
For the Simulink Design Verifier product, the following table lists:

 Prerequisites (see ISO 26262-8, 11.3.1)

 Supporting information (see ISO 26262-8, 11.3.2)
 Tool qualification work products (see ISO 26262-8, 11.5)

The tool qualification artifacts listed in the table are mapped to sections in this document and
artifacts found elsewhere.

Artifact Corresponding Documents / Artifacts

Safety plan <Insert document title, version, and filename / link>
<Insert software lifecycle phase(s)>
Applicable prerequisites of the lifecycle
phases where software tool is used  <Insert prerequisite(s)>

Predetermined maximum ASIL <Insert ASIL>

 Simulink Design Verifier: User's Guide
R2015a
sldv_ug.pdf
Software tool documentation
 Simulink Design Verifier: Release Notes
R2015a
rn.pdf

 MathWorks® bug report system at

Environment and constraints of the www.mathworks.com/support/bugreports/
software tool
 <Insert information>

1-4
Artifact Corresponding Documents / Artifacts
 Customized and completed “Software Tool Criteria
Evaluation Report” of Simulink Design Verifier ISO 26262
Tool Qualification Package (this document)
certkitiec_sldv_tqp.docx
 Simulink Design Verifier Reference Workflow
R2015a
certkitiec_sldv_workflow.pdf
Software tool criteria evaluation report
 Certificate Z10 11 12 67052 013
December 2011
certkitiec_sldv_certificate.pdf
 Report to the certificate Z10 11 12 67052 013
November 2014
certkitiec_sldv_certreport.pdf

 Customized and completed “Software Tool Qualification

Report” of Simulink Design Verifier ISO 26262 Tool
Qualification Package (this document)
certkitiec_sldv_tqp.docx
 Customized and completed Simulink Design Verifier
Conformance Demonstration Template
certkitiec_sldv_cdt.docx
Software tool qualification report
 Certificate Z10 11 12 67052 013
December 2011
certkitiec_sldv_certificate.pdf
 Report to the certificate Z10 11 12 67052 013
November 2014
certkitiec_sldv_certreport.pdf

Customized and completed “Confirmation Review of Tool

Classification and Qualification” of Simulink Design Verifier
Confirmation review of qualification of a ISO 26262 Tool Qualification Package (this document)
software tool
certkitiec_sldv_tqp.docx

1-5
1-6
2 Software Tool Criteria Evaluation
Report
2.1 Tool Environment
It is assumed that the Simulink Design Verifier will be used in the following environment (see
ISO 26262-8, 11.4.4.1d):

<Insert operating system and other pertinent environment information>

2-2
2.2 Tool Configuration
It is assumed that the Simulink Design Verifier will be used with the following tool
configuration (see ISO 26262-8, 11.4.4.1b).

Configuration Parameter Setting

Design Verifier Pane
Analysis options - Mode Test generation
<Insert project-specific settings> <Insert project-specific settings>
Design Verifier > Block Replacements Pane
<Insert project-specific settings> <Insert project-specific settings>
Design Verifier > Parameters Pane
<Insert relevant configuration parameter names> <Insert project-specific settings>
Design Verifier > Test Generation Pane
<Insert relevant configuration parameter names> <Insert project-specific settings>
Design Verifier > Design Error Detection
<Insert relevant configuration parameter names> <Insert project-specific settings>
Design Verifier > Property Proving
<Insert relevant configuration parameter names> <Insert project-specific settings>
Design Verifier > Results Pane
<Insert relevant configuration parameter names> <Insert project-specific settings>
Design Verifier > Report Pane
<Insert relevant configuration parameter names> <Insert project-specific settings>

2-3
2.3 Reference Workflow
It is assumed that Simulink Design Verifier will be used as described in the reference workflow
documented in Simulink Design Verifier Reference Workflow.

To access the reference workflow document, on the MATLAB command line, type
certkitiec. The reference workflow document is in Simulink Design Verifier > r2015a.

2-4
2.4 Tool Use Cases
It is assumed that Simulink Design Verifier will be used as described by one or more of the
following use cases (see ISO 26262-8, 11.4.4.1c). Additional information about the assumed
usage of Simulink Design Verifier can be found in the reference workflow document Simulink
Design Verifier Reference Workflow.

[SLDV_UC1] Generating Test Cases That Satisfy

Structural Coverage Objectives
The Simulink Design Verifier tool is being used to generate test cases that satisfy structural
coverage objectives for a model, including:

 Decision coverage
 Condition coverage
 Modified condition and decision coverage (MC/DC)

The model used for test case generation can be an executable specification, a model used for
production code generation, or other interim models created during the model elaboration phase.

Blocks or functions for customizing test cases are not used.

[SLDV_UC2] Generating Customized Test Cases

The Simulink Design Verifier, in combination with customization blocks (Test Objective, Proof
Objective, Assumption, Test Condition) or functions (sldv.test, sldv.condition), is
being used to generate test cases to satisfy customer objectives.

The model used for test case generation can be an executable specification, a model used for
production code generation or other interim models created during the modeling elaboration
phase.

[SLDV_UC3] Applying the Generated Test Cases

The generated test cases are being used to test a model or generated code running in software-in-
the-loop (SIL) or processor-in-the-loop (PIL) modes.

The test object can be the model used to generate the test cases, other interim models created
during the model elaboration phase, or generated code executing in SIL or PIL mode.

2-5
2.5 Generic Tool Classification
The tool classification for Simulink Design Verifier was performed in a generic manner,
independently from the development of a particular safety-related item or element.

For the generic tool classification, the reference use cases listed in the section “Tool Use Cases”
have been taken into account. The tool classification is based on the potential malfunctions or
erroneous outputs and Error Prevention and Detection Measures listed in the following
corresponding sections.

Additional information about the assumed error prevention and detection measures can be found
in the reference workflow document Simulink Design Verifier Reference Workflow.

2.5.1 Potential Malfunctions or Erroneous Outputs

The following potential malfunctions or erroneous outputs were taken into account as part of the
tool classification process:

[SLDV_E1] Test Case Generation – False Negative

The test case generation capability incorrectly reports test objectives that are not covered by the
generated test cases as covered.

[SLDV_E2] Test Case Generation – False Positive

The test case generation capability incorrectly reports test objectives that are covered by the
generated test cases as uncovered.

[SLDV_E3] Test Case Generation – Non Interference Error

The test case generation capability is malfunctioning or produces erroneous output, but the
model to be analyzed does not invoke the malfunctioning portion of the tool.

[SLDV_E4] Test Case Generation – Usage of Incorrect Tool Inputs

The tool user invokes the test case generation capability on incorrect or inconsistent tool inputs.

[SLDV_E5] Test Case Generation – Misinterpretation of Tool

Outputs
The tool user misinterprets correct tool outputs.

2-6
 [SLDV_E6] Test Case Generation – Incorrect Tool Usage
The tool user does not follow established procedures when using the tool.

[SLDV_E7] Simulink Design Verifier- Incorrect or Modified

Installation
The tool user does not follow established procedures when installing the tool, installs the tool in
an incorrect operational environment, or modifies an installation.

2.5.2 Error Prevention and Detection Measures

The following measures to facilitate seamless functioning of the test case generation capability
of the Simulink Design Verifier tool and to assess the completeness and adequacy of generated
test cases are referenced in the tool classification process. Additional considerations are
described in the section of the same name in Simulink Design Verifier Reference Workflow.

[M1] Usage of Compatibility Checker

Before generating test cases:

 Check the model for compatibility with Simulink Design Verifier.

 Review detected and partial incompatibilities.

[M2] Usage of Model Coverage to Assess Completeness and

Adequacy of Generated Test Cases
After generating test cases:

 Run the generated test cases against the model used for test case generation.
 Measure the model coverage1.
 Review the model coverage report.

1
The Simulink Verification and Validation product provides the Model Coverage capability.

2-7
Tool Classification Summary
Potential malfunction Use TI Justification for TI Prevention / TD Justification for TD TCL
or erroneous output cases detection
measures
[SLDV_E1] Test Case [SLDV_ TI2 Incomplete test cases [M1] Usage of TD1 A-priori compatibility checking allows to TCL1
Generation – False UC1] could result in untested Compatibility Checker detect and address (partial)
Negative [SLDV_ portions of the model or incompatibilities with the test case
UC2] generated code. [M2] Usage of Model generation capability.
[SLDV_ Untested portions of the Coverage to Assess
UC3] model or generated code Completeness and A-posteriori structural model coverage
could prevent errors Adequacy of analysis can be used to assess the
from being detected. Generated Test Cases completeness and adequacy of the
generated test cases1. Uncovered test
objectives are reported and can be
addressed.
[SLDV_E2] Test Case [SLDV_ TI1 Generated tests cover    TCL1
Generation – False UC1] defined objectives.
Positive [SLDV_
UC2]
[SLDV_
UC3]
[SLDV_E3] Test Case [SLDV_ TI1 Error in the tool does not    TCL1
Generation – Non UC1] impact generated tests.
Interference Error [SLDV_
UC2]
[SLDV_
UC3]
[SLDV_E4] Test Case [SLDV_ TI2 Incomplete test cases Configuration TD1 Revision control and configuration TCL1
Generation – Usage of UC1] could result in untested Management and management facilitate integrity of models
Incorrect Tool Inputs [SLDV_ portions of the model or Revision Control2 used for test case generation and the
UC2] generated code. Untested artifacts to be tested.
[SLDV_ portions of the model or Usage of Checksums
UC3] generated code could Using checksums allows the unique
prevent errors from identification of the artifacts being
[M1] Usage of
being detected. verified.
Compatibility Checker

A-priori compatibility checking allows to

detect and address (partial)
incompatibilities with the test case
generation capability.
[SLDV_E5] Test Case [SLDV_ TI2 Misinterpretation of tool Competency of the TD1 Training of tool users can prevent these TCL1
Generation – UC1] outputs could prevent project team3 issues.
Misinterpretation of Tool [SLDV_ errors from being
Outputs UC2] detected.
[SLDV_
UC3]

2-8
Potential malfunction Use TI Justification for TI Prevention / TD Justification for TD TCL
or erroneous output cases detection
measures
[SLDV_E6] Test Case [SLDV_ TI2 Incorrect tool usage Competency of the TD1 Training of tool users can prevent these TCL1
Generation - Incorrect UC1] could prevent errors project team3 issues.
Tool Usage [SLDV_ from being detected.
UC2]
[SLDV_
UC3]
[SLDV_E7] Simulink [SLDV_ TI2 Incorrect or modified Adherence to TD1 Adherence to installation guide TCL1
Design Verifier - UC1] installation could installation guide instructions will facilitate a seamless
Incorrect or Modified [SLDV_ prevent errors from instructions installation.
Installation UC2] being detected.
Measures to facilitate Verification of the installed tool version
Installation Integrity will prevent these issues.
and Release
Compatibility4
1 The Simulink Verification and Validation product provides the Model Coverage capability.
2 See “Configuration Management and Revision Control” of the Simulink Design Verifier Reference Workflow.
3 See “Competency of the Project Team” of the Simulink Design Verifier Reference Workflow.
4 See “Installation Integrity and Release Compatibility” of the Simulink Design Verifier Reference Workflow.

Based on the preceding analysis, the maximum tool impact of the Simulink Design Verifier use
cases taken into account is TI2.

Applying the prevention and detection measures previously described provides a high degree of
confidence that a malfunction or an erroneous output of the test vector generation capability of
Simulink Design verifier can be prevented or detected. The resulting maximum required tool
confidence level is TCLMAX1.

TÜV SÜD reviewed the generic tool classification and confirmed the above results in Report to
the certificate Z10 11 12 67052 013.

2-9
2-10
3 Software Tool Qualification Report
3.1 Requirement for Tool Qualification
Given the maximum required tool confidence level TCLMAX1 (see “Generic Tool
Classification”), the test vector generation capability of Simulink Design Verifier does not
require formal tool qualification methods (see ISO 26262-8, 11.4.6.1).

3-2
3.2 Voluntary Tool Qualification
MathWorks carried out tool qualification methods for Simulink Design Verifier on a voluntary
basis to provide additional confidence.

TÜV SÜD reviewed the voluntary generic tool qualification methods for the test case generation
capability of Simulink Design Verifier and confirmed the results in Report to the certificate Z10
11 12 67052 013.

MathWorks also generically qualified the model coverage analysis capability of the Simulink
Verification and Validation product that can be leveraged to carry out error detection method
[M2] Usage of Model Coverage to Assess Completeness and Adequacy of Generated Test
Cases.

TÜV SÜD reviewed the generic tool qualification methods for the model coverage capability of
Simulink Verification and Validation and confirmed the results in Report to the certificate Z10
11 12 67052 013.

3-3
3-4
4 Confirmation Review of Tool
Classification and Qualification
4.1 Requirement for Confirmation Review
The tool classification (see “Software Tool Criteria Evaluation Report”) was carried out
independently from the development of the project under consideration. Therefore, the resulting,
predetermined tool confidence level shall be confirmed by the applicant prior to Simulink
Design Verifier being used for the development of a particular safety-related item or element in
the project under consideration (see ISO 26262-8, 11.4.2, 11.4.10).

Provided that the predetermined maximum tool confidence level TCLMAX1 is being confirmed,
tool qualification and therefore confirmation of the tool qualification are not required.

The generic tool classification is based on the assumption that Simulink Design Verifier is being
used as described in the reference workflow documented in Simulink Design Verifier Reference
Workflow. Therefore, conformance with the reference workflow in the project under
consideration shall be confirmed by the applicant.

4-2
4.2 Validity of Generic Tool Classification
Applicable Tool Confidence Level: <Insert TCL>

<Insert results of confirmation review or reference to confirmation review documentation>

4-3
4.3 Validity of Generic Tool Qualification
Not applicable.

4-4
4.4 Conformance with Reference Workflow
<Insert reference to customized and completed Conformance Demonstration Template>.

4-5