PHOENIX, ARIZONA - Change is inevitable, so it is unwise to plan for only one rainy day. Alternatively, it is always wise to have a dynamic plan in place that outlines procedures for how to best cope with and adapt to rainy days when they do arrive. Whether talking about literal precipitation, financial stability, or the anticipation of novel input data to a clinical machine learning (ML) model, the same wisdom applies. 

As mentioned in prior articles, designing and executing an end-to-end clinical ML environment is no walk in the park. One must also do the following: scope the goals of stakeholders to identify a project’s goal, identify which data match those goals, get permission to access data, curate, assess, and describe the data, pre-process the data, and build a model and validate it model, to name a few of the myriad important tasks one must consider. 

Suppose one does surmount all the difficulties that come with executing a clinical ML model. This all seems great, since performance is good, stakeholders are happy, and value is seemingly added to the clinic as a whole. Lurking in the ether, however, is an insidious, familiar mastermind that is adrift, interfering with everything that seemed great with our new model: change. The clinical input data slowly change, drift, and morph into a slightly different animal. 

Depending on the use case, such data drift and concept drift - changes of the relationships between inputs and outputs, i.e., between the data and labels - of the data can decimate the performance of models and lead to inaccurate predictions, which can also potentially lead to suboptimal care for patients. Data drift is just one of many issues relating to the post-integration phase of clinical artificial intelligence (AI). Thus, the proper maintenance and continual