DEGGENDORF, GERMANY - Surgeries are one of the main cost factors of healthcare systems. To reduce the costs related to diagnoses and surgeries, a system based on machine learning (ML) has been built to automatically segment body parts like livers or lesions in medical images. This model is based on convolutional neural networks, which show promising results in real computed tomography scans.

It is also part of a larger system that aims to support doctors by visualizing the segments in a Microsoft HoloLens, a type of augmented reality device. Our approach allows doctors to intuitively look at and interact with the holographic data, rather than using 2D screens, which in the end enables them to provide better healthcare.

Need for virtual reality, augmented reality in radiology

According to Goldman Sachs, the combined virtual reality (VR) and augmented reality (AR) market is forecast to reach USD80 billion in revenue by 2025.¹ The expectation is that new markets will be created, and existing ones will be disrupted. The United States spent 17 percent of its 2015 gross domestic product on healthcare, with approximately 3 percent of such expenditures related to surgery.² As such, there is a strong demand for increasing efficiency and effectiveness in operating rooms to both reduce costs and improve patient care, and the best way of addressing this is by taking advantage of the latest technological developments.

Despite advancements in medical imaging, the current pipeline used by practitioners is still not automated, and so the quality of results often depends on the limited resources and subjectivity related to the human factor in processing images. Recently, however, more resources are being devoted to looking into less time-consuming and more automated solutions which do not rely on the human factor, like using ML to