Date: 10th February 2021
In most medical specialities imaging is critical to treatment decisions and it has become one of the most data-rich components of the electronic health record (EHR). A cardiologist for example will have to analyse thousands of images for a single echocardiogram, along with all the other critical data such as laboratory results, vital signs and data from other imaging tools. Now, scientists have used artificial intelligence (AI) to provide superior predictions of one-year all-cause mortality from echocardiograms, outperforming other clinically used predictors and specialists.
Recent advances in deep learning technologies is driving a new generation of AI-based diagnostic tools, such as to diagnose autism from maternal biomarkers, diagnosis of early-stage breast cancer, or COVID-19 associated pneumonia. A natural progression for machine learning in medicine is the prediction of future clinical events, for example it is being used to predict the dynamics of brain networks in response to microstimulation, or to predict cancer killing drug combinations.
Now, scientists from Geisinger, US, led by Brandon Fornwalt have developed a deep learning algorithm using echocardiogram videos of the heart to predict mortality within a year.
The team started by training a deep neural network (DNN) model on 812,278 echocardiogram videos collected from 34,362 Geisinger patients spanning back over the last ten years. The model was then tested and evaluated on two new and distinct groups of participants and compared the results to four expert cardiologists’ predictions.
The DNN model had an accuracy of 78% whilst the aggregated cardiologists score was 63%. The model’s predictions also outperformed two benchmark clinical risk models – the widely used pooled cohort equations and the Seattle Heart Failure risk score.
Next, the team wanted to determine whether cardiologists could use the model to improve the performance of their predictions. Cardiologists’ predictions were collected, then immediately after they were presented with the same study along with the machine prediction score. The cardiologists correctly changed >10% of their predictions and improved the sensitivity by 13%, supporting the use of the model as a tool to aid clinicians. Although it was noted ~4% of predictions were changed incorrectly.
Conclusions and future applications
The team here have developed a methodology and architecture for extracting clinically relevant predictive information from medical videos with a DNN, and shown the potential for it to assist physicians with the task of predicting 1 year all-cause mortality.
The team’s goal is to develop computer algorithms to improve patient care, and with this in mind they plan on assessing the feasibility of this model to predict mortality in other independent healthcare systems. Whilst here, the data had inherent heterogeneity, outside this healthcare system the model will have to be tested for its robustness. By providing accurate predictions it is hoped that it will support better clinical decisions regarding treatments and interventions.
AI is starting to have a high impact in the medical and healthcare field. The work here, starts to build on the nascent body of work that is using AI to assess risk factors and future outcomes such as we have recently seen with biomarkers identified as a risk factors of autism. How we deal with the outcomes of such predictions from an ethical perspective and how this translates to the patient will have to be carefully considered.
For more information please see the press release from Geisinger
Ulloa Cerna, A. E., L. Jing, C. W. Good, D. P. vanMaanen, S. Raghunath, J. D. Suever, C. D. Nevius, G. J. Wehner, D. N. Hartzel, J. B. Leader, A. Alsaid, A. A. Patel, H. L. Kirchner, J. M. Pfeifer, B. J. Carry, M. S. Pattichis, C. M. Haggerty and B. K. Fornwalt (2021). “Deep-learning-assisted analysis of echocardiographic videos improves predictions of all-cause mortality.” Nature Biomedical Engineering.
https://doi.org/10.1038/s41551-020-00667-9
