3.  The ASPICE-Processes (VDA Scope)

3.8 SWE.6 – Software Qualification Test

What is the purpose?

Software qualification tests ensure that the integrated software meets the software requirements.

What is the benefit?

These tests are carried out during the development of a system before the entire software is linked with the hardware and the mechanical components. This is to ensure that the software works as expected and meets the software requirements.

What is the content?

  • The software test strategy for the tests of the overall software is defined (BP1). This supports all parties involved (testers, test managers, requirements engineers, project management,…) to develop a common understanding of how to accomplish the work (BP1).
  • It must be prevented that changes to the software have negative side effects on already successfully tested requirements. For this purpose a regression strategy needs to be provided (BP1).
  • The tests that demonstrate the implementation of the software requirements are to be specified in detail (BP2).
  • Test cases are selected for execution in line with the test strategy, the regression strategy, and the release plan (BP3). Tests are performed and results are recorded (BP4).
  • There needs to be bidirectional traceability between software requirements, test cases, and test results (BP5). The software requirements and the test cases need to be consistent (BP6). Consistency requires also that the traceability links are correct and complete. This is typically checked through reviews.
  • A test summary report is produced and disseminated to the relevant people (like project management and customer) (BP7).
Bidirectional traceability between software requirements and software qualification tests
Bidirectional traceability between software requirements and software qualification tests

Experiences, problems and hints

  • It is very beneficial (but not mandatory) to have one common test strategy for all test processes to avoid gaps and to create synergies.
  • The software qualification test is closely connected to the software requirements activities (SWE.1). The requirements engineers develop the verification criteria which comprise their expertise and hints on what should be observed when developing the tests.
  • Traceability between requirements and tests is essential to be able to demonstrate coverage of requirements by tests and to monitor the test progress. This requires tool-based traceability using a good integration of the requirements tool and the test tool.
  • Consistency checking is often underestimated and poorly implemented. This requires reviews which check whether the tests are correct and the requirements are covered completely.
  • Software qualification tests can be combined with any other type of test as long as one can demonstrate that the software requirements have been tested. Often the software qualification test is performed on the target hardware in conjunction with the system qualification test.
  • If the project uses platform software, the tests against platform requirements and those against the project-specific requirements should be seamlessly adjusted.

Automotive SPICE text of Software Qualification Test (SWE.6)

The purpose of the Software Qualification Test Process is to ensure that the integrated software is tested to provide evidence for compliance with the software requirements.

BP1: Develop software qualification test strategy including regression test strategy. Develop a strategy for software qualification testing consistent with the project plan and the release plan. This includes a regression test strategy for re-testing the integrated software if a software item is changed.

BP2: Develop specification for software qualification test. Develop the specification for software qualification test including test cases based on the verification criteria, according to the software test strategy. The test specification shall be suitable to provide evidence for compliance of the integrated software with the software requirements.

BP3: Select test cases. Select test cases from the software test specification. The selection of test cases shall have sufficient coverage according to the software test strategy and the release plan.

BP4: Test integrated software. Test the integrated software using the selected test cases. Record the software test results and logs.

NOTE 1: See SUP.9 for handling of non-conformances.

BP5: Establish bidirectional traceability. Establish bidirectional traceability between software requirements and test cases included in the software qualification test specification. Establish bidirectional traceability between test cases included in the software qualification test specification and software qualification test results.

NOTE 2: Bidirectional traceability supports coverage, consistency and impact analysis

 BP6: Ensure consistency. Ensure consistency between software requirements and test cases included in the software qualification test specification.

NOTE 3: Bidirectional Consistency is supported by bidirectional traceability and can be demonstrated by review records.

 BP7: Summarize and communicate results. Summarize the software qualification test results and communicate them to all affected parties.

NOTE 4: Providing all necessary information from the test case execution in a summary enables other parties to judge the consequences.

Output Work Products

Test plan, Test specification, Test result, Traceability record, Review record, Communication record, Strategy

If you wish to continue to the next chapter click here: