Abstract
The repeatability and replicability of experiments are central notions of the scientific method. Physical experiments were the backbone of science and engineering. In today's development of complex, engineered systems, experiments are increasingly peformed in-silico, that is, through simulation. In order to reduce effort and expense, engineers use (computer) simulations of "models" of complex systems to investigate what-if scenarios and identify design flaws earlier.
Modern engineered systems rely heavily on modelling and simulation. Over the past few decades, significant advances have been made in multi-formalism, multi-abstraction, and multi-view modelling. However, these developments have not been matched by comparable progress in the foundational methods for constructing and managing simulation experiments, nor in the systematic treatment of model validity.
Without precise insight in the validity range of a model, no guarantees can be provided about the utility of the simulation results, which can often lead to costly mistakes. This is especially critical in the context of digital twins, where real-time decision-making relies on the validity of the simulated model to represent its real-world (cyber-physical) counterpart.
Validity of models is ascertained by performing experiments in the real and simulated world and comparing their results. Repeatability and replicability are highly desirable of experiments experiments, ensuring that findings remain robust across independent trials and withstand the test of time. On further investigation, it became clear that repeatability and replicability require the ability to create, manage, and share reusable descriptions/specifications/models of said experiments. However, a general framework to model experiments (real or virtual), in order to reason about their properties like repeatability and replicability, and ultimately, the validity of models, is currently lacking.
These reusable experiment models, that we call experiment specifications, serve as a basis for logical reasoning about the properties and conditions of (validation) experiments. They also enable reasoning about the contextual validity of models, where the validity of a model is not a single universal truth value, but instead a function of the experimental conditions specified in the experiment specification, called a validity frame. Furthermore, reusable experiment specifications can enhance efficiency by reducing the need to perform identical or similar experiments multiple times, allowing resources to be allocated toward exploring new experimental conditions.
This project develops the foundations for specification, execution and (re-)use of experiment and validity frames.
Researcher(s)
Research team(s)
Project type(s)