What is VVUQ (Verification, Validation, and Uncertainty Quantification)?
By Florent Mathieu, CEO of EikoSim
In the world of scientific simulations, ensuring accuracy, reliability, and credibility is of utmost importance. To achieve this, scientists and engineers employ a rigorous process called Verification, Validation, and Uncertainty Quantification (VVUQ).
This article explains what VVUQ involves, where it applies, and how it fits into larger simulation quality frameworks.
What is VVUQ, and how is it used?
VVUQ is a systematic approach to assessing the quality and reliability of computational simulations. It brings together activities that verify and validate the models, algorithms and software used in simulations, while quantifying and managing the uncertainties associated with the results.
By employing VVUQ, scientists and engineers aim to gain confidence in simulation predictions, improve decision-making, and enhance the overall reliability of computational models.
VVUQ brings together three families of practices; let’s look at them in more detail.
Verification
Verification ensures that the simulation software accurately solves the model equations and that the algorithms are correctly implemented.
It involves tasks such as code verification and code review, debugging, code comparisons and convergence analysis. These tasks help identify and fix coding errors, so that the simulation code faithfully represents the mathematical model.
Verification tasks may include:
- Code debugging: Identifying and fixing errors or bugs in the simulation code.
- Convergence analysis: Assessing the convergence of numerical methods to ensure accurate solutions.
- Code comparison: Comparing the results of the simulation code against known analytical or benchmark solutions for simple problems.
- Consistency checks: Verifying that mathematical equations and physical principles are implemented correctly.
Verification is an internal process, usually carried out by simulation engineers (or by simulation methods engineers, when this role exists in the team).
Validation
Validation, on the other hand, assesses the accuracy and reliability of simulation predictions by comparing them to experimental or observed data.
This process involves experimental design, data collection, statistical analysis and model calibration.
Validation lets simulation engineers use test cases to determine whether the simulation adequately captures the behavior of the real-world system or process.
Common validation tasks include:
- Experimental design: Planning and executing experiments to collect data for comparison with simulation results.
- Data collection: Gathering experimental measurements or observations, and processing the data to allow comparison between experimental and simulation data.
- Statistical analysis: Applying statistical methods to compare simulation outputs with experimental data.
Model calibration: Adjusting simulation parameters to improve agreement with experimental results.
Struggling with this step? Check out how it’s done in EikoTwin Digital Twin
Uncertainty Quantification
Uncertainty Quantification deals with the uncertainties that affect simulation and test results.
It involves identifying and quantifying sources of uncertainties, propagating them through the simulation model, and analyzing their impact on the results.
Uncertainty Quantification enables decision-makers to understand how reliable simulation predictions are, and to make informed choices based on the level of difference between tests and simulations.
Uncertainty Quantification tasks include:
- Sensitivity analysis: Determining the influence of input parameter variations on simulation outputs.
- Error propagation: Assessing how uncertainties in input data propagate through the simulation model to affect output predictions.
- Model validation under uncertainty: Assessing the reliability of simulation predictions considering uncertainties in the model and input data.
Where does VVUQ apply?
VVUQ is a versatile approach that finds applications across various domains.
Some of the key fields where VVUQ is employed can be spotted from the ASME (the American Society of Mechanical Engineers) VVUQ subcommittee list.
The ASME subcommittees develop standards for VVUQ practices in various domains, for example:
- Mechanical engineering (VVUQ 10 and 20 standards): VVUQ plays a crucial role in engineering simulations, such as structural analysis, fluid dynamics, and electromagnetics.
- It ensures the accuracy of simulations for designing and optimizing complex systems, such as aircraft, bridges, and power plants (VVUQ 30 standard).
- Medical devices (ASME VVUQ 40): by performing these validation tasks, medical device manufacturers can gain confidence in the simulation predictions of their medical devices.
- Aerospace and Defense (standard under preparation): VVUQ is crucial in aerospace and defense applications, including aerodynamics, missile simulations, and radar system design.
It helps validate the accuracy of simulations used for performance analysis, safety assessments of the final product, and optimization of complex systems.
Simulation Quality Frameworks – leveraging VVUQ activities
VVUQ is often integrated into larger simulation quality frameworks, which provide a comprehensive way to assess and improve simulation reliability.
These frameworks provide a systematic approach to assessing the quality of simulation code and computational models. Some of the best known are:
- Sandia’s PCMM (Predictive Capability Maturity Model)
- NASA’s CAS (Credibility Assessment Scale)
- ETH Zurich’s UQLab (Uncertainty Quantification Laboratory)
These frameworks provide various tools and resources to help engineers and scientists ensure the quality of their computational models.
Need to know more?
VVUQ is an important process for ensuring the accuracy and reliability of computational models and software applications.
It is used in many domains, and several simulation quality frameworks help engineers and scientists ensure the quality of their models and software.
At EikoSim, we develop tools that make VVUQ techniques easier and more efficient to apply.
Check out our products to know more!
If you are interested in learning more about verification and validation, many resources are available online. You can also find more information at the ASME VVUQ conference if this is a topic of interest.
We hope this article has given you a better understanding of VVUQ. Any questions? Get in touch.
Frequently asked questions
What does VVUQ stand for and what is its purpose?
VVUQ stands for Verification, Validation and Uncertainty Quantification. It is a systematic approach to assessing the quality and reliability of computational simulations — verifying the models, algorithms and software, and quantifying the associated uncertainties — to build confidence in predictions and improve decision-making.
How does verification differ from validation in VVUQ?
Verification ensures the software accurately solves the model equations and that algorithms are correctly implemented, through debugging, convergence analysis, code comparison and consistency checks; it is an internal task. Validation assesses prediction accuracy by comparing simulation outputs to experimental or observed data to check whether the simulation captures the real-world behavior.
What tasks are involved in the validation step?
Typical validation tasks include experimental design, data collection and processing, statistical analysis comparing simulation outputs with measurements, and model calibration to improve agreement with experimental results.
What does Uncertainty Quantification address?
Uncertainty Quantification identifies and quantifies sources of uncertainty, propagates them through the simulation model and analyzes their impact, via tasks such as sensitivity analysis and error propagation. It lets decision-makers understand the reliability of predictions based on the level of difference between tests and simulations.
In which domains is VVUQ applied, and which standards support it?
VVUQ is applied in mechanical engineering (ASME VVUQ 10, 20, 30), medical devices (ASME VVUQ 40) and aerospace and defense (a standard in preparation), covering structural analysis, fluid dynamics, electromagnetics and more. Well-known frameworks include Sandia’s PCMM, NASA’s CAS and ETH Zurich’s UQLab.