5.3 How are assessments conducted?
How is an assessment carried out?
An assessment consists of different phases. The VDA Automotive SPICE Guidelines define the following phases which are generally recognized.
The assessment preparation is mainly about the planning of the assessment based on the assessment scope, the objectives and other factors that influence the assessment. The result is an assessment plan with all necessary data for the assessment as well as an assessment agenda, which is coordinated between the lead assessor, the sponsor and the organization to be assessed. An important part of this is the assessment scope definition which contains mainly: which processes, which Capability Levels, which instances, which parts of the product, which teams and locations.
The actual assessment takes place on site and in person. In exceptional cases, individual process interviews can be carried out via telephone conference and remote access to the documents.
The performance phase usually begins with a short introductory session. The lead assessor provides an introductory presentation (introducing Automotive SPICE, the assessment team, the conditions of the assessment, the agenda). This is followed by a representative of the organization assessed presenting the product, the project, and the company.
The interviews and documentation reviews are then carried out according to the agenda. If Level 3 is assessed, the interviews usually start with a thorough review of the organization’s standard processes. The actual interviews on processes are started with project management (MAN.3), quality assurance (SUP.1) and configuration management (SUP.8) as they provide a good insight into the project and at the same time form the basis for the evaluation of Level 2. This is followed by the engineering processes according to their natural sequence. This sequence is important because evaluation one process requires understanding its inputs. For example, to check the traceability of the system tests to the system requirements one need to have previously understood the structure of the system requirements. Afterwards the remaining support processes and supplier monitoring will follow.
Between the interviews the assessment team performs short consolidation sessions in which the evidences recorded are reviewed and consolidated, and a preliminary rating is carried out. These consolidations and their results are confidential. The main reason is that ratings may need to be revised later due to new evidences. Therefore, too early feedback could lead to misunderstandings in the assessed organization.
Towards the end of the assessment the results are presented.
ISO/IEC 33002 standard has requirements on the contents of an assessment report, the most important of which are:
- The capability levels
- The process attribute ratings plus traceability to the objective evidence used for the rating
- The relationship between the base practices / generic practices and their associated objective evidence
The report may also contain more or less detailed explanations of the strengths and weaknesses found, and improvement suggestions.
There is also a special template available by the VDA for exchanging results with OEMs. It comprises the rating of capability levels, process attributes, base practices, and generic practices. Additional information is the name of the supplier as well as a classification of the developed system (e.g., infotainment system). However, there are no detailed findings. This VDA exchange format is generated on request by the assessment team in addition to a more detailed report. However, like any other report, it is not passed on to third parties unless agreed by all parties.
If you wish to continue to the next chapter click here: