By Florent Mathieu, CEO of EikoSim
In the world of scientific simulations, ensuring accuracy, reliability, and credibility is of utmost importance. To achieve this, scientists and engineers employ a rigorous process called Verification, Validation, and Uncertainty Quantification (VVUQ).
This blog article explores what VVUQ entails, its applications across various software product domains, and its role in the design specification development cycle for the development phase of software architecture larger simulation quality frameworks.
What is VVUQ, and how is it used?
VVUQ is a systematic approach to assessing the quality and reliability of computational simulations. It involves a series of activities aimed at verifying functional testing and validating the models, algorithms, and software used in simulations, while also quantifying and managing uncertainties associated with the results.
By employing VVUQ, scientists and engineers aim to gain confidence in simulation predictions, improve decision-making, and enhance the overall reliability of computational models.
VVUQ is a series of practices; let’s look at them in more detail.
Verification
Verification focuses solely on ensuring that the simulation software accurately solves the model equations and the desk checks that the implemented algorithms are correctly implemented.
It involves tasks such as code verification checks and code review, debugging system and nonfunctional testing, code comparisons, nonfunctional testing, and method convergence analysis. Verification systems and nonfunctional testing help identify and rectify coding errors, ensuring the simulation code faithfully represents the mathematical model.
Verification tasks may include:
- Code debugging: Identifying and fixing errors or bugs in the simulation code.
- Convergence analysis: Assessing the convergence of numerical methods to ensure accurate solutions.
- Code comparison: Comparing the results of the simulation code against known analytical or benchmark solutions for simple problems.
- Consistency checks: Verifying that mathematical equations and physical principles are implemented correctly.
Verification is an internal process usually carried out by simulation engineers (or simulation verification methods engineers if this role exists within the software development team and the software meets all the specified requirements of the organization testing team’s business requirements.
Validation
Validation, on the other hand, assesses the accuracy and reliability of simulation predictions by comparing them to experimental or observed data.
This process involves experimental design, data collection, statistical analysis, validation verification, and validation verification, and model calibration.
Validation allows engineers on the simulation engineering side to take test cases to determine if the simulation adequately captures the behavior of the real-world system or process under test.
Common validation tasks include:
- Experimental design: Planning and executing experiments to collect data for comparison with simulation results.
- Data collection: Gathering experimental measurements or observations, and processing the data to allow comparison between experimental and simulation data.
- Statistical analysis: Applying statistical methods to compare simulation outputs with experimental data.
Model calibration: Adjusting simulation parameters to improve agreement with experimental results.
Struggling with this step? Check out how it’s done in EikoTwin Digital Twin
Uncertainty Quantification
Uncertainty Quantification deals with managing uncertainties associated with simulation test results.
It involves identifying and quantifying sources of uncertainties, propagating them through the simulation model, and analyzing their impact on the results.
Uncertainty Quantification enables decision-makers to understand the quality assurance reliability of simulation predictions and make informed choices based on the level of difference between tests and simulations.
Uncertainty Quantification tasks include:
- Sensitivity analysis: Determining the influence of input parameter variations on simulation outputs.
- Error propagation: Assessing how uncertainties in input data propagate through the simulation model to affect output predictions.
- Model validation under uncertainty: Assessing the reliability of simulation predictions considering uncertainties in the model and input data.
Where does VVUQ apply?
VVUQ is a versatile approach that finds applications across various domains.
Some of the key fields where VVUQ is employed can be spotted from the ASME (the American Society of Mechanical Engineers) VVUQ subcommittee list.
The ASME subcommittees’ role is to propose standards system requirements design specifications for VVUQ practices business requirements in various domains, with notable examples:
- Mechanical engineering (VVUQ 10 and 20 standards): VVUQ plays a crucial role in engineering simulations, such as structural analysis, fluid dynamics, and electromagnetics.
- It ensures the accuracy of simulations for designing and optimizing complex systems, such as aircraft, bridges, and power plants (VVUQ 30 standard).
- Medical devices integration testing (ASME VVUQ 40): by performing these validation testing tasks, medical device manufacturers can gain confidence in the simulation predictions of their medical devices.
- Aerospace and Defense (standard under preparation): VVUQ is crucial in aerospace and defense applications, including aerodynamics, missile simulations, and radar system design.
It helps validate the accuracy of simulations used for performance analysis, cycle safety assessments of the final product, and optimization of complex systems.
Simulation Quality Frameworks – leveraging VVUQ activities
VVUQ is often integrated into larger simulation quality frameworks to ensure comprehensive assessment, complete design, functional testing, and improvement of simulation reliability.
These various verification frameworks provide a systematic approach to ensuring verification testing of the quality of source code and computational models in software products. Some of the most well-known frameworks for quality assessment include:
- Sandia’s PCMM (Predictive Capability Maturity Model)
- NASA’s CAS (Credibility Assessment Scale)
- ETH Zurich’s UQLab (Uncertainty Quantification Laboratory)
These frameworks provide various tools and resources to help engineers and scientists ensure the quality of their computational models.
Need to know more?
VVUQ is an important process for ensuring the accuracy and reliability of computational models and software applications.
It is used in various domains, and several larger simulation quality frameworks can help engineers and scientists ensure the quality of their models and software application.
At EikoSim, we provide tools that participate in applying VVUQ techniques in a user-friendly and efficient way.
Check out our products to know more!
If you are interested in learning more about verification and validation, many resources are available online. You can also find more information at the ASME VVUQ conference if this is a topic of interest.
We hope this blog article has given you a better understanding of VVUQ. If you have any questions, please feel free to leave a comment below.