Boosting standardization with Open Source - Quality Checker Framework

The Challenge of Ensuring Quality in Simulation Environments

Simulation environments are essential for developing and testing automotive software, as they allow engineers to model and analyze various scenarios without the need for physical prototypes. However, these environments rely on input files from different tool vendors and data suppliers, each with its own configuration and standards. This input diversity can lead to challenges in simulation projects.

To target this, Marcel Langer (Product Owner Simulation & Testing at CARIAD), Andreas Kern (Software Technologies Lead at CARIAD), and their team developed the Quality Checker Framework to reliably evaluate simulation environments against ASAM e.V. (Association for Standardization of Automation and Measuring Systems) standards. ASAM e.V. is a non-profit organization that promotes standardization in automotive development.

Introducing the Quality Checker Framework

The CARIAD team has shared the framework with the Open Source community. Building on this Open Source contribution, ASAM e.V. has funded a project to develop a standard-agnostic Quality Checker Framework applicable to all their standards. This effort includes creating the first set of official checks for the standards ASAM OpenDRIVE, ASAM OpenSCENARIO XML and informative examples on how to check ASAM OTX Extensions. By adhering to these standards, the Quality Checker Framework ensures that simulation assets remain interchangeable, reusable, and compatible with a variety of tools and processes.

Over the past few months, experts from 24 companies across nine countries have joined the project, contributing their expertise and code to expand the tool's functionality.

 Key Features of the Solution

• Formal Correctness: The Framework focuses on ensuring that all input files adhere to formal correctness standards, which is the first step in assessing their quality. This includes checking for syntax errors, complying with data formats, and adhering to predefined schemas.
• Data Integrity: Beyond formal correctness, the framework also provides rules to verify the integrity of the data within the files. This means ensuring that the files contain the right data and meet specific requirements for test benches or projects.
• Standardized Checks: The Quality Checker Framework includes a standardized set of checks that can be configured and applied to various simulation functions. These checks are designed to assess different aspects of the simulation assets, from basic compliance to advanced functionality.
• Extensibility: Include additional proprietary rule sets in the configuration and run them together with the official checks. They can be implemented in any programming language.
• Result Generation and Reporting: The Framework provides a consistent process for generating and reporting the results of the checks. This ensures transparency and allows for easy identification and resolution of any issues.

The Benefits of Open Sourcing

By making the Quality Checker available to the Open Source community, CARIAD aims to increase speed and efficiency in simulation projects for all parties involved. The requirements for quality and interoperability are the same across the industry, making the framework an ideal candidate for Open Sourcing. The Open Source approach fosters a homogeneous and transparent environment for all stakeholders and enables continuous improvement through shared resources.