Causal Falsification of Digital Twins

About this event

We consider how to assess the accuracy of a digital twin using real-world data. We formulate this problem within the framework of causal inference, which leads to a precise definition of what it means for a twin to be "correct" that seems appropriate for many applications. Unfortunately, fundamental results from the causal inference literature mean observational data cannot be used to certify that a twin is correct in this sense unless potentially tenuous assumptions are made, such as that the data are free of unmeasured confounding. To avoid these assumptions, we propose an assessment strategy that instead aims to find situations in which the twin is not correct, and present a general-purpose statistical procedure for doing so. Our approach yields reliable and actionable information about the twin under only the assumption of an i.i.d. dataset of observational trajectories, and in particular remains sound regardless of whether or not the data are confounded. We demonstrate the effectiveness of our methodology through a large-scale, real-world case study involving sepsis modelling within the Pulse Physiology Engine, which we assess using the MIMIC-III dataset of ICU patients.