AI for cancer diagnosis in a secure, usable and scalable environment
Highly scalable platform that allows the incorporation of new data providers
Healthcare professionals can now make better decisions supported by AI based-solution
At T-Systems, we are a team of around 28,000 people worldwide, making us one of the world's leading providers of integrated end-to-end solutions. We develop hybrid cloud solutions, artificial intelligence and promote the digital transformation of business, industry, the public sector and ultimately society as a whole.
Catalan Health Service (CatSalut) delivers comprehensive and high-quality public health coverage for all citizens of Catalonia. CatSalut is a public insurer that offers a number of specialized services, including planning, assessments and inspections of healthcare organizations contracted through the system in the region.
CatSalut, the TIC Salut Social Foundation and the Department of Health of the Generalitat de Catalunya are working on the Health/IA Program to promote and develop Artificial Intelligence (AI) in the Catalan healthcare system. One of the initiatives being developed is the INCISIVE project, supported by the European Commission. This project aims to develop and validate a set of AI tools based on medical imaging for the diagnosis, prognosis and monitoring of four types of cancer: lung, breast, colorectal and prostate.
The INCISIVE project aims to create an interoperable pan-European federated repository of medical imaging and other clinical data as well as AI models that will enable secure donation and sharing of data in compliance with ethical standards, legal and privacy demands, increasing accessibility to datasets and enabling experimentation of AI-based solutions, towards large-scale adoption of such solutions in cancer diagnosis, prediction and monitoring.
The program seeks to create an easy-to-adopt solution that can be applied in current medical practice for diagnosis and prediction, as well as follow-up of a patient's cancer.
Data in the service of health
In 2022, CatSalut and TIC Salut Foundation partnered with Cloudera to develop a secure platform that would allow for governed storage of their data, as well as efficient data management, during the design period of the federated platform. TIC Salut demanded an AI platform that would guarantee the security of health data from the different European hospital centers participating in the project, as well as the possibility of training the different algorithms from the multiple research centers and technology companies in the consortium.
The organization required a flexible and scalable solution that allowed for easy data loading and a simple structure and organization for historical data uploaded to the platform. In addition, it also had to enable data governance and efficient data management to integrate with the data processing centers of the Generalitat de Catalunya, managed by T-Systems. Finally, they chose Cloudera Data Platform (CDP) based on the experience other councils in the Government of Catalonia had with the platform for projects with similar requirements.
“Cloudera Data Platform best suited our needs. We drew on the positive experiences that other councils of the Government of Catalonia shared with us,” stated Carlos Gallego, head of Medical Imaging Systems, Pathological Anatomy, Precision Oncology of Catalonia and director of the Salut/AI program.
In addition, since most of the data providers were healthcare professionals, the Foundation needed easy data uploading and a simple structure and organization for historical data uploaded to the platform
AI for cancer diagnosis
Trust in Cloudera has allowed them to advance through the first phase of the project, which has involved uploading past images of the types of cancers being analyzed to the platform. The goal is to address the diagnostic side using AI algorithms on past medical images that healthcare professionals will be able to access in order to study and compare similar cases.
The second phase includes the training of about 20 algorithms dedicated to the diagnosis, prognosis and treatment of the four types of cancer by combining different imaging modalities and clinical data. In order to do that, they are using Cloudera Data Science Workbench, which is helping them integrate data science notebooks to the platform so that they can quickly and directly access the governed data. This solution allows them to speed up their ability to design, scale, and deploy machine learning and advanced analytics.
In addition, not only will the multiple algorithms act as decision support for healthcare professionals, but will incorporate tools with the capability of explain so that the users of these tools understand the mechanisms that have led to a given conclusion. This ability of the algorithm to provide explanations is meant to boost the confidence of the professionals using it. Thanks to a model equipped with an interpretive system, healthcare professionals can know why a particular diagnosis is being formulated. Although final decisions will be made based on their own knowledge, the fact that professionals know what variables have influenced a diagnosis will help them make better decisions.
“Our work has been made much easier thanks to the simplicity and the agile learning curve offered by Cloudera technology. Accessing the data through a console would have required individuals with professional profiles that included that kind of knowledge,” says Didier Domínguez, Data Scientist at CatSalut.
The platform has served multiple profiles, from radiologists and oncologists to AI algorithm developers with great flexibility. This has made it unnecessary to hire specialized technical personnel due to the usability of the platform. It is also a highly scalable platform that allows the incorporation of new data providers to the project, as well as different developers located in different European countries.
At its conclusion, the INCISIVE project will result in an interoperable federated data repository of thousands of clinical images of breast, lung, prostate, and colorectal cancer to enable improved diagnosis, prediction and care of these cancers for the residents of Catalonia. The repository, built on Cloudera Data Platform, will allow the secure and GDPR-compliant sharing of health data.
"Thanks to Cloudera Data Platform, CatSalut and the TIC Salut Social Foundation have made progress in their use of AI for cancer diagnosis in a secure, usable and scalable environment, absolutely essential in healthcare" Carlos Gallego, head of Medical Imaging Systems, Anatomic Pathology, Precision Oncology of Catalonia and director of the Salut/IA program.