Projects |

MISTAI: Medical Intelligence & Skills Training in AI

Wed, 01 Jul 2026 00:00:00 +0000

The war in Ukraine has severely disrupted medical research and teaching. Digital pathology and AI offer a way to keep both going without depending on physical infrastructure. MISTAI is a bilateral cooperation between the University Hospital Bonn and Dnipro State Medical University (DSMU), funded by the Ministry of Culture and Science of North Rhine-Westphalia (MKW NRW).

Goals

Shared data infrastructure: digitise and validate at least 100 H&E and IHC whole-slide images from a DSMU cohort of muscle-invasive bladder cancer (MIBC), on a joint secure platform.
AI diagnostic pipeline: fine-tune state-of-the-art pathology foundation models on the combined Bonn–Dnipro data, including spatial analysis of tumour-infiltrating lymphocytes from routine H&E slides, inspired by the UC-TIL approach.
Clinical tool: a QuPath plugin that lets pathologists run the trained model directly in their existing workflow.
Capacity building: weekly remote collaboration, a one-week exchange visit to Bonn, and a Computational Pathology teaching module for the DSMU curriculum.
Follow-on work: a joint DFG/BMBF proposal outline and a MICCAI challenge concept.

Partners

Bonn contributes high-resolution slide scanners, AI infrastructure and expertise in computational medical imaging (Prof. Shadi Albarqouni) and translational lung and tumour pathology (Prof. Verena Tischler). DSMU contributes a clinically distinct bladder-cancer cohort and long-standing pathology expertise (Assoc. Prof. Nina Bondarenko, Department of Pathological Anatomy).

Symposium on AI in Medicine & Food Science

Mon, 01 Dec 2025 00:00:00 +0000

Time: All times correspond to local Bonn time (CET)

Day 1 — Tuesday, 9 December 2025

Arrival & Check-in
📍 Youth Hostel Bonn, Haager Weg 42, 53127 Bonn

Day 2 — Wednesday, 10 December 2025

Symposium on AI in Medicine
📍 Baunscheidtstraße 17, 53113 Bonn

Time	Session	Speaker
09:30–10:00	Registration	—
10:00–10:30	Welcome Speech
10:30–11:45	Keynote Speech: Seeing, Sensing, Securing: AI in Modern Medicine
11:45–12:00	Therapeutic & Surgical Simulation in Medicine
12:00–12:15	AI-Driven Segmentation for Early Eye Disease Detection
12:15–12:30	“Ocean modeling in the Eastern Mediterranean: insights from physical based modelling, data assimilation and machine learning.
12:30–14:00	Lunch Break	—
14:00–14:15	FIDAL: Diversity-Aware Federated Active Learning Under Real-World Distribution Shifts
14:15–14:30	MRIQT: Physics-Aware Diffusion Model for Image Quality Transfer in Neonatal Ultra-Low-Field MRI
14:30–14:45	Rapid and affordable detection of Giant Cell Arteritis using B-mode ultrasound
14:45–15:15	Break	—
15:15–15:30	Teleradiology System Integrating DICOMweb and AI	Molka Yermani
15:30–15:45	Lightweight Bayesian Deep Learning for Point-of-Care Breast Ultrasound	Zeravan Ali
15:45–16:00	Modality‑Aware Deep Learning for Human Bone Fracture Recognition on HBFMID	Omar Al-Zakholi
16:00–16:15	Embedding BI-RADS Foundational Concepts for Cross-Domain and Cross-Modality Generalization in Breast Imaging
16:15–16:30	ML for Histopathological Classification of Conjunctival Melanocytic Intraepithelial Lesion
16:30–16:45	Affordable AI for Retinal Disease Detection
16:45–17:00	Wrap-up Discussion

Day 3 — Thursday, 11 December 2025

Symposium on Food Science
📍 Käthe-Kümmel-Str. 1, 53115 Bonn

Time	Session	Speaker
14:25–14:30	Welcome Speech	&
14:30–15:00	Proteins Under the Microscope: Extraction, Modification and Characterisation
15:00–15:10	Effects of processing on protein-bound amino acids
15:10–15:30	Innovative plant protein sources: Common bean and hempseed press cake as raw material for protein ingredient production	&
15:30–16:00	Break	—
16:00–16:15	From Questions to Discoveries: Highlights from My Food Science Research Career
16:15–16:30	Driving Excellence in Nutrition and Food Science: LAU’s NFS Department’s Mission and Work
16:30–16:40	Micro and Nanoplastics in table salt: occurence and exposure	Hanin El Hajj
16:40–16:50	Unmasking gluten contamination in gluten-free products	Roa Echtay
16:50–17:10	Wrap-up Discussion	Shadi & Ismail
17:10–17:30	Lab Tour

We have the symposium a bit late as one of the key guest’s expertise in Food Science will only arrive in the afternoon

Day 4 — Friday, 12 December 2025

Roundtable discussion in our lab + Excursion Day (Self-Organized)** 📍 Baunscheidtstraße 17, 53113 Bonn

For the Excurssion, here are some suggested visits: Bonn Christmas Market, Cologne Cathedral, Chocolate Museum Köln.
📍 Bonn / Cologne region

Day 5 — Saturday, 13 December 2025

Departure Day
📍 Youth Hostel Bonn

Summer School on Biomedical Imaging with Deep Learning (BILD)

Sun, 01 Jun 2025 00:00:00 +0000

AI in Radiology – From Pixels to Decisions (joint teaching with GUC)

Wed, 01 Jan 2025 00:00:00 +0000

This joint teaching activity was developed within a BMFTR-funded international program to strengthen academic exchange and capacity building in artificial intelligence for medical imaging. The course combines lectures, hands-on exercises, and project-based learning, enabling students to work on clinically motivated imaging problems using AI methods.

Perspective

The initiative contributes to sustainable academic exchange between Germany and Egypt and provides a basis for continued joint teaching activities, student exchange, and integration of research-led topics into international curricula.

Selected Interdisciplinary Research with Methodological AI Contributions

In addition to my core research in artificial intelligence for medical imaging and clinical applications, I have contributed to a small number of exploratory interdisciplinary projects in materials science and food science. These activities focused on transferring methodological expertise in data-driven modeling and machine learning to neighboring domains and align with the University of Bonn’s Transdisciplinary Research Areas Sustainable Futures, Modelling, and Life and Health. Together, they demonstrate the transferability of AI methodologies and my engagement in transdisciplinary research initiatives without detracting from my primary research focus.

Application of AI and Advanced MRI for a Better Prediction of Disease Progression in Neurological Disorders

Wed, 01 Jan 2025 00:00:00 +0000

TBA

Strategic Arab-German Network for Affordable and Democratized AI (SANAD)

Wed, 01 Jan 2025 00:00:00 +0000

In recent years, the transformative potential of Artificial Intelligence (AI) in revolutionizing global healthcare has become evident, offering promising advancements in diagnostic precision, personalized treatment, and healthcare delivery. However, integrating AI into low-resource environments, especially in low- and middle-income countries (LMICs), remains constrained by significant barriers such as limited computational resources, inadequate local data, and financial constraints. The Strategic Arab-German Network for Affordable and Democratized AI in Healthcare (SANAD- سند) seeks to address these challenges by fostering collaboration, empowering local expertise, and advancing Affordable AI and Collaborative Federated Learning solutions tailored to underserved regions. The word “SANAD” means support in Arabic, which is what this project is about.

SANAD aims to bridge the healthcare gap between resource-rich and resource-limited settings, particularly the Global North and Global South disparity. Many existing AI models are developed using centralized datasets from well-funded institutions, which often fail to generalize across diverse populations and contexts. SANAD proposes a strategic, inclusive approach to democratizing access to AI by leveraging interdisciplinary expertise and promoting knowledge-sharing between Arab and German stakeholders

Measures:

, Tunisia
, Germany

Summer School on Affordable AI (SAAI)

Wed, 01 May 2024 00:00:00 +0000

Bias-aware foundation models for equitable breast cancer screening

Mon, 01 Jan 2024 00:00:00 +0000

This project addresses fairness and generalization challenges of foundation models for breast mammography, where AI systems trained predominantly on data from high-income countries often underperform in underrepresented populations. Together with international clinical partners, we assembled and analyzed a heterogeneous collection of mammography datasets from Europe, North America, the Middle East, and Asia, including a biopsy-confirmed in-house cohort of approximately 1,500 patients from Lebanon. Methodologically, we conducted a systematic evaluation of foundation models under cross-dataset shift and demonstrated that naïve dataset aggregation improves average performance but does not eliminate subgroup-specific biases. Fairness-aware optimization strategies substantially reduced performance disparities across age groups, breast density categories, and geographic regions while maintaining competitive diagnostic accuracy.

Perspective

The collaboration constitutes an ongoing international research effort at the intersection of medical imaging, fairness-aware AI, and global health. Building on current results, the project is being expanded to additional clinical partners in Palestine and Egypt, supported by the Global One Health Research Academy (GLOHRA), and is planned to scale further through the submitted EU Horizon Europe project SHE-Scan, contingent on funding.

INTERACT-CMIL: Interpretable multi-task grading of conjunctival melanocytic lesions

Mon, 01 Jan 2024 00:00:00 +0000

Collaborators: University Hospital Tübingen, University Hospital Erlangen

This project addresses the grading of conjunctival melanocytic intraepithelial lesions (CMIL), rare precursors of conjunctival melanoma characterized by complex, interdependent histopathological criteria and substantial inter-observer variability. Together with clinical partners, we curated a multi-center, expert-annotated histopathology dataset comprising 486 conjunctival biopsy image patches labeled along five clinically defined grading axes. Methodologically, we developed INTERACT-CMIL, a multi-task deep learning framework modeling inter-criterion dependencies via selective multi-head supervision and an inter-task consistency loss.

The proposed approach outperformed CNN and pathology foundation-model baselines (macro-F1 up to 85%), producing coherent and interpretable predictions aligned with expert grading standards. Methodological development and evaluation were coordinated within my group in close collaboration with the clinical partners.

Perspective

The project established a reproducible computational benchmark for CMIL grading and contributed to the standardization of digital ocular pathology. A related follow-up project on AI-assisted eyelid tumor analysis received a University of Bonn Prototype Grant, supporting translational development and clinical feasibility studies.

MRIQT: Image-quality transfer from ultra-low-field to high-field neonatal MRI

Mon, 01 Jan 2024 00:00:00 +0000

This project addresses the limited diagnostic usability of portable ultra-low-field MRI (0.064 T) in neonatal neuroimaging, where reduced signal-to-noise ratio constrains clinical interpretation despite bedside accessibility. Together with the clinical partners, we curated a unique in-house cohort comprising 50 paired ultra-low-field and high-field 3D neonatal brain MRI scans, complemented by 230 additional unpaired scans across nine pathologies. Building on this paired–unpaired data structure, we developed MRIQT, a 3D conditional diffusion framework for structure-preserving image quality transfer from ultra-low-field to high-field MRI. The method integrates physics-consistent k-space degradation, classifier-free guided diffusion, and an SNR-weighted volumetric perceptual loss.

MRIQT outperformed CNN- and GAN-based baselines, achieving up to 15.3% improvement in PSNR, while expert clinicians rated 85% of generated images as diagnostically useful with preserved pathology.

Perspective

The project provides a methodological foundation for automated MRI quality enhancement and modality-agnostic quality control. Building on the current results, the work is prepared for translational follow-up and prospective validation, with a planned submission to the Go-Bio Initial funding program.

Robust surgical tool segmentation with state-space vision models

Mon, 01 Jan 2024 00:00:00 +0000

This exploratory collaboration investigated state-space vision models for robust surgical tool segmentation under realistic visual corruptions. In collaboration with Kyung Hee University, we participated in the MICCAI EndoVis 2024 SegSTRONG-C challenge, benchmarking robustness to non-adversarial corruptions such as smoke, bleeding, and illumination changes. The proposed Vision Mamba–UNet approach achieved competitive performance across corruption types and ranked 7th overall among international submissions.

Perspective

The project supported early exploration of state-space models in medical image analysis and international doctoral mentorship. A jointly submitted DAAD exchange proposal, although not funded, laid the groundwork for continued collaboration and future joint research activities.

Affordable AI and Collaborative Federated Learning for Global Healthcare (EEDA)

Fri, 01 Sep 2023 00:00:00 +0000

Federated foundation models for multi-center CT staging of pancreatic cancer

Mon, 01 May 2023 00:00:00 +0000

Collaborators: University Medical Center Göttingen, Charité – Universitätsmedizin Berlin, Peter MacCallum Cancer Centre

This project addresses preoperative staging of pancreatic ductal adenocarcinoma (PDAC), where lymph node metastasis is a major prognostic factor but CT-based assessment suffers from low sensitivity and high inter-observer variability. Together with partners across three German centers, we established a retrospective multi-center cohort of 546 patients to enable evaluation under real-world heterogeneity while preserving data privacy. Methodologically, we combined CT foundation model transfer learning (pre-training on 148,000 volumetric CT scans) with heterogeneity-aware federated optimization. The resulting system improved federated performance by +12.6% balanced accuracy over FedAvg and recovered most of the centralized reference performance (centralized BA 0.601; DOR 3.45).

Perspective

The project establishes a sustainable multi-center collaboration for privacy-preserving oncologic imaging in Germany. Ongoing work extends the framework toward multimodal integration and more robust harmonization strategies, and a joint DFG proposal with the clinical partners has been submitted to further advance federated foundation models for PDAC staging and translation.

Deep learning for clinically significant prostate cancer in 3D mpMRI and MR Fingerprinting

Sun, 01 Jan 2023 00:00:00 +0000

Collaborators: Peter MacCallum Cancer Centre, Melbourne

This project addresses the detection and characterization of clinically significant prostate cancer (csPCa) in MRI, where diagnostic performance is limited by inter-reader variability and the complexity of multi-parametric, volumetric imaging. Together with the clinical partners, we curated an initial in-house 3D MRI cohort of 106 patients with histopathological reference, including bi-parametric MRI and quantitative MR Fingerprinting (MRF), which was subsequently extended with external and public datasets to form a multi-center cohort of approximately 500 cases. Methodologically, we developed fully 3D deep learning models that explicitly incorporate anatomical prostate zone information, enabling interpretable and biomarker-aware csPCa detection and segmentation beyond slice-wise analysis.

A dedicated sub-project evaluated transformer-based 3D U-Net architectures for lesion segmentation. On public (PI-CAI) and in-house datasets, transformer-based models achieved Dice scores above 0.60 on detected lesions and demonstrated improved robustness under reduced training data compared to CNN-based baselines, while highlighting persistent challenges in false-positive reduction.

Perspective

The project established a scalable research line on interpretable, anatomy-aware AI for prostate cancer imaging and contributed to early-career training through a completed Bachelor’s thesis and an ISMRM abstract. Ongoing work extends this line toward vision–language foundation models integrating 3D imaging, quantitative MRI biomarkers, and anatomical priors, forming a basis for continued multi-center collaboration and future third-party funding.

Deep learning to estimate aging from chest imaging

Sun, 01 Jan 2023 00:00:00 +0000

Chronologic age is an important risk factor for many chronic diseases. However, as we all age at different rates chronological age is an imperfect measure of aging and more accurate estimates of one’s true biological age are desirable. The proposed study will develop and test artificial intelligence-based measures of aging from chest imaging data and will explore whether these measures can improve chronologic age-based clinical guidelines for prevention of cardiovascular disease and cancer. Over recent decades life expectancy has continuously increased, however, the years of life without significant morbidity have remained stable over the last 5 decades resulting in significant socioeconomic burden to society and the healthcare system. Therefore, delaying the onset of morbidity is a promising way to improve healthy aging. In particular, prevention and early detection of diseases of high socioeconomic importance such as cancer and cardiovascular disease are a priority to increase health and longevity. Biological age is a concept to estimate the differences in rates of aging not captured by chronologic age. Several measures of biological age have been developed in recent years including blood, functional and physiological approaches. However, there relevance in clinical routine is limited. Medical imaging may be an effective way to estimate aging by measuring age-related changes based on anatomical alterations visible in the image such as degenerative changes of the spine, dilation of the heart and vasculature and changes in lung parenchyma. Unlike molecular measures, image-based reflections of aging can be calculated opportunistically using data acquired during routine clinical care. Furthermore, these measures can estimate aging of specific organ systems, which may improve disease-specific preventive measures over general estimates of biological age. The image-based estimates of aging using deep learning proposed in this study will address this unmet need and may help to personalize clinical decision-making for chronic disease screening and prevention. My preliminary data show that deep learning can estimate a chest x-ray age from a chest radiograph image and that this chest x-ray age predicts longevity better than chronologic age in large multicenter clinical trial data. The proposed project will build on these promising initial findings with the potential to broaden the application to other imaging modalities and to leverage multi-omics data to infer causes of aging.

Collaborators:

PD. Dr. , UniKlinikum Freiburg

Outcomes

This exploratory project examined the feasibility of imaging-based phenotyping and biological aging estimation from chest CT and MRI. The work focused on conceptual development and clinical hypothesis generation rather than large-scale validation and resulted in a clinical abstract presented at the European Congress of Radiology.

Perspective

The project provided methodological insights into opportunistic imaging-based risk modeling and informed the design of subsequent projects with clearer data availability, validation strategies, and funding perspectives.

Open Medical Inference (OMI)

Sun, 01 Jan 2023 00:00:00 +0000

Outcomes

The Open Medical Inference (OMI) initiative is a BMFTR-funded national research infrastructure aimed at improving reproducibility, transparency, and comparability of machine learning methods in medical imaging and clinical AI. Within OMI, I contributed to the development and evaluation of standardized benchmarking protocols, open evaluation pipelines, and reference datasets for medical AI, with a particular focus on robustness, generalization, and fair comparison across institutions and data distributions. These activities complement my research on federated learning and trustworthy AI by addressing methodological gaps in evaluation practices and supporting community-wide standards for medical machine learning research.

Our role

Contributing researcher (Scientific Lead) with responsibility for methodological input on benchmarking design, evaluation protocols, and reproducibility aspects, in close collaboration with national partners.

Perspective

OMI provides a sustained national framework for open and reproducible medical AI research. The experience and infrastructure developed within OMI inform ongoing and future projects on federated learning, multi-center evaluation, and trustworthy AI, and strengthen national coordination across academic and clinical institutions. These activities align with and complement my work on federated learning benchmarks and multi-center evaluation (see also Sections and ).

Utilising Artificial Intelligence in Cancer Imaging To Improve Patient Outcomes

Sun, 01 Jan 2023 00:00:00 +0000

Cachexia is a devastating tumour-induced wasting disease affecting 30% of all cancerpatients. This initiative brings together clinical, computational (artifi cial intelligence),and laboratory research partners to work on the delivery of improved outcomes forpatients with cancer cachexia. Aims of the collaboration: Funding will provideopportunities to broaden digital research links and submit joint grant and conjoint PhDproposals.

Long-term objectives:

1. Developing non-invasive screening tools for cancercachexia, using AI-assisted imaging analysis approaches.
1. Developing a greaterunderstanding of molecular processes of cancer cachexia using animal models andpatient samples.
1. Increase knowledge and awareness about cachexia througheducational events for clinicians, patients and families on cachexia diagnosis andtreatment.

This multidisciplinary and diverse initiative builds on the expertise andexisting collaborations between researchers/clinicians across both universities (Bonn/Melbourne). The group consists of experts in radiology/ image analysis (Ko,Attenberger), computational science /AI (Albarqouni) and cancer biology/ metabolism(Cheng). This team of clinicians and researchers with deep knowledge of the fi eld ofcancer cachexia are ideally placed to educate clinicians, patients and families oncachexia and increase its awareness in the general community.

Partners:

, Australia
, Germany

Outcomes

This collaboration was supported by institutional funding to foster international exchange at the intersection of artificial intelligence, cancer imaging, and translational oncology. Core outcomes included two jointly organized international workshops held in Bonn and Melbourne, reciprocal research visits, and the identification of shared research directions. The collaboration further supported concrete synergies in ongoing projects, including pancreatic cancer staging and multiparametric prostate MRI.

Perspective

The project established a durable international partnership between the University of Bonn and the University of Melbourne, providing a platform for continued joint research, doctoral training, and future collaborative funding applications.

Federated learning for musculoskeletal segmentation in CT

Tue, 01 Nov 2022 00:00:00 +0000

This collaboration explored federated learning for musculoskeletal segmentation in CT imaging across institutions in Germany and Japan. Using labeled CT data from NAIST and an unlabeled whole-body CT cohort from University Hospital Bonn, we evaluated cross-site generalization and domain transfer under realistic scanner and population shifts. In parallel, a secure federated learning infrastructure was deployed and ethical approval obtained at University Hospital Bonn. While the project did not progress to publication-ready federated training due to annotation constraints and deployment complexity, it provided valuable methodological and organizational insights.

Perspective

The collaboration strengthened institutional expertise in federated learning under real-world clinical constraints and informed subsequent national and international projects with more harmonized data availability.

Affordable deep learning for inherited and common retinal diseases

Wed, 01 Jun 2022 00:00:00 +0000

This international collaboration addresses the diagnosis of inherited retinal diseases (IRDs) and common retinal disorders using affordable color fundus photography, motivated by limited access to genetic testing and advanced imaging in low-resource settings. Together with partners in Germany and Lebanon, we curated and analyzed heterogeneous multi-center retinal imaging datasets combining in-house cohorts and public data, spanning diverse demographics and disease spectra. Methodologically, we developed lightweight deep learning models for multi-class retinal disease classification covering 21 retinal conditions, including 13 IRDs. The approach achieved strong diagnostic performance for prevalent IRDs (e.g., F1 score of 0.86 for retinitis pigmentosa) while remaining computationally efficient and interpretable.

Perspective

The collaboration established a sustainable international research link between ophthalmology groups in Germany and Lebanon and contributed to interdisciplinary student training. Ongoing work explores privacy-preserving and federated learning strategies to improve robustness and equitable deployment across populations further.

Deep learning for fetal diaphragmatic hernia detection in ultrasound

Wed, 01 Jun 2022 00:00:00 +0000

Congenital diaphragmatic hernia (CDH) is a rare but severe fetal malformation associated with high neonatal morbidity and mortality, where early detection is critical for risk stratification, parental counseling, and perinatal management. Although ultrasound is the primary imaging modality in prenatal screening, CDH detection remains challenging due to limited acoustic windows, overlapping anatomical structures, and strong operator dependence.

The initial objective of this project was to develop deep learning–based methods for automated CDH detection in fetal ultrasound. Due to changes in the clinical collaboration context, the project could not be continued in its originally planned data-driven form. As a result, the focus shifted toward consolidating the existing evidence base and systematically analyzing methodological and clinical barriers for AI-assisted fetal ultrasound diagnostics.

As a central outcome, we co-authored a comprehensive systematic review on artificial intelligence in obstetric and gynecologic ultrasound, critically assessing methodological rigor, data availability, validation practices, and clinical readiness across 189 published studies. The review identified fetal malformation detection, including CDH, as a clinically high-impact yet underexplored application area, highlighting the need for standardized datasets, multi-center validation, and interpretable AI approaches.

Perspective

The insights gained from this project provided a methodological foundation for subsequent ultrasound- and MRI-based research with sustained clinical partnerships and informed the design of later projects with improved data availability and validation strategies. Building on this groundwork, renewed efforts toward CDH detection are envisaged within a national multi-center collaboration, leveraging harmonized data collection and shared clinical expertise.

Development of a Deep Learning Toolkit for MRI-Guided Online Adaptive Radiotherapy

Sat, 01 Jan 2022 00:00:00 +0000

In Germany close to 500,000 new patients are diagnosed with cancer every year. More than half of the patients undergo radiotherapy as part of their treatment. In-room imaging, in combination with modern external beam radiotherapy techniques, enables tight adjustment of the delivered dose to the target. However, the full potential of these techniques is currently not exploited. One of the reasons for this is the presence of anatomical changes. For the vast majority of patients in today’s radiotherapy, anatomical alterations are considered by introducing safety margins around the tumor volume during treatment planning, which increases the irradiated volume, the dose burden to organs at risk (OARs), and eventually limits the applicable dose.

An improved treatment can be realized by online adaptive radiation therapy (ART): instead of applying the same irradiation plan throughout the entire course of treatment, the treatment is optimized at each irradiation session on basis of the daily anatomy in treatment position, as inferred from in-room imaging. Great efforts have been made to realize the integration of magnetic resonance imaging (MRI) as in-room imaging modality, but only during the last few years integrated MR-linacs became clinically available at few academic institutions. The superior soft-tissue contrast allows for accurate visualization of targets and OARs, which has allowed basic online ART workflows to be implemented. The time overhead associated with the generation of a synthetic-CT from MRI and a re-delineation of the patient for treatment plan optimization greatly limits applicability to all patients.

The goal of this project is to tackle the main bottlenecks of online plan adaptation: synthetic CT generation and organ delineation. By combining our respective expertise in radiotherapy and artificial intelligence, we aim at proposing deep-learning-based solutions to build a toolkit to tackle both of these challenges. In the case of synthetic CT generation, we will use generative adversarial networks with hybrid paired-unpaired training paradigms, additionally exploiting novel embedding loss functions to ensure geometric fidelity. Alternatively, disentangled representations will be investigated. For segmentation we aim at exploiting the information from previous fractions by performing spatio-temporal segmentation using longitudinal spatial memory networks. Finally, we aim at evaluating the benefits from these approaches by performing an evaluation of their clinical impact in terms of plan optimization, including physician grading of delineations and robustness assessment. The time benefits, including potential corrections to segmentations and synthetic CT, will also be quantified. Further details can be found .

Collaborators:

Prof. , Department of Radiation Oncology, LMU University Hospital
Dr. , Department of Radiation Oncology, LMU University Hospital
Prof. , CAMP, CIT School, TU Munich

Outcomes

MRI-guided online adaptive radiotherapy requires fast and reliable image segmentation and synthesis to support treatment adaptation under strict clinical time constraints. In this DFG-funded project, we addressed key methodological challenges related to patient-specific segmentation and uncertainty-aware modeling under inter-observer and inter-institutional variability. Together with the partners, we curated a longitudinal multi-center MRI cohort comprising planning and fraction scans across multiple institutions, enabling systematic evaluation under realistic clinical heterogeneity. Methodologically, the project resulted in two complementary contributions: CloverNet, a patient-specific 3D segmentation architecture leveraging prior planning annotations, and ProSona, a prompt-guided personalization framework modeling inter-expert variability via a continuous latent space. The proposed methods achieved a significant improvement over standard baselines and received a Best Paper Award at a MICCAI workshop.

Perspective

The project established a strong national research network at the interface of medical imaging, radiation oncology, and AI, and contributed to doctoral and master’s training. Ongoing work extends these approaches toward scalable foundation models for personalized and uncertainty-aware radiotherapy workflows, with a focus on clinical integration and multi-center validation.

GCA-US-AI: Deep learning–based ultrasound diagnosis of Giant Cell Arteritis

Sat, 01 Jan 2022 00:00:00 +0000

In close interdisciplinary cooperation with clinical rheumatology, this project addresses the highly operator-dependent and time-critical ultrasound diagnosis of Giant Cell Arteritis (GCA), a vasculitis associated with irreversible vision loss. Together with our partners, we curated and annotated an in-house dataset of 3,800 ultrasound examinations from 344 patients, constituting one of the largest clinically validated datasets for ultrasound-based GCA diagnosis. Methodologically, we developed artery-aware deep learning models that integrate anatomical context into ultrasound image analysis, achieving robust performance improvements over artery-agnostic baselines (AUROC up to ~0.9 for larger arteries). Uncertainty estimation and visual explainability were incorporated to support reliable clinical interpretation.

Perspective

The collaboration is ongoing and continues to serve as a basis for joint publications, supervised theses, and continued dataset extension (in-house already ~15,000 images) toward a multi-center cohort. Current work explores vision–language models integrating imaging and clinical data and forms the basis of a planned joint DFG proposal coordinated together with the clinical partners.

Large vision models for robust and trustworthy medical image understanding

Sat, 01 Jan 2022 00:00:00 +0000

This collaboration investigates the applicability of large vision models to medical imaging, addressing robustness, generalization, and trustworthiness under domain shift and limited supervision. The work examined the behavior of foundation-scale models transferred from natural-image pretraining to medical imaging tasks. Methodological contributions include empirical analyses of cross-domain generalization, sensitivity to spurious correlations, and limitations of naïve scaling, alongside proposed evaluation protocols and architectural adaptations to improve robustness and reliability in medical contexts.

Perspective

The collaboration supported doctoral mentorship and strengthened national expertise at the intersection of foundation models and medical imaging. The insights gained inform ongoing work on principled evaluation and responsible deployment of large vision models in medical applications.

FLamby: Federated learning benchmarks for realistic healthcare settings

Tue, 01 Jun 2021 00:00:00 +0000

Collaborators: EPFL (Switzerland), INRIA (France), University of Southern California (USA), Institut Polytechnique de Paris (France), University of California, Berkeley (USA)

A central bottleneck in federated learning research is the lack of realistic, multi-center benchmarks reflecting the heterogeneity of clinical data. Within this international collaboration, I contributed to the development of FLamby, a benchmark suite comprising seven real-world medical datasets with natural institutional splits across histopathology, radiology, dermatology, and tabular clinical data. Published at NeurIPS 2022 (Datasets and Benchmarks Track), FLamby demonstrated that standard federated learning approaches often underperform pooled training and highlighted the need for heterogeneity-aware optimization methods.

Our role

Contributing collaborator with responsibility for dataset preparation, benchmark design, and evaluation of federated learning methods.

Perspective

FLamby established a widely used community reference for cross-silo federated learning in healthcare. Its open-source release continues to inform benchmarking practices and supports subsequent federated learning studies and methodological developments.

BigPicture Project

Mon, 01 Feb 2021 00:00:00 +0000

A new consortium of the EU Innovative Medicines Initiative (IMI) will establish the biggest database of pathology images to accelerate the development of artificial intelligence in medicine.

To take AI development in pathology to the next level, a European consortium combining leading European research centres, hospitals as well as major pharmaceutical industries, is going to develop a repository for the sharing of pathology data. The 6-year project called , will herald a new era in pathology.

Background

Pathology is the cornerstone of the workup of many diseases such as cancer, autoimmune diseases, of the follow up after transplantation and is also critical for the evaluation of the safety of drugs. It’s based on the examination of tissue samples (slides) under the microscope. However, despite its pivotal role, it still relies heavily on the qualitative interpretation by a qualified pathologist.

While the microscope symbolizes the profession, the digitalisation of slides in recent years ignited a revolution: not only images can now be shared and accessed from distant locations, they can also be processed by computers. This opens the door for artificial intelligence (AI) applications to assist the pathologist and help study diseases, find better treatments and contribute to the 3Rs (replace, reduce, and refine animal use in research). However, the development of robust AI applications requires large amounts of data, which in the case of pathology means a huge collection of digital slides and the medical data necessary for their interpretation. Sharing these has so far remained challenging due to the data storage capacity required to host a sufficiently large collection and to concerns regarding the confidential character of the medical information.

To allow the fast development of AI in pathology, the project aims to create the first European, ethical and GDPR-compliant (General Data Protection Regulation), quality-controlled platform, in which both large-scale data and AI algorithms will coexist. The platform will be developed in a sustainable and inclusive way by connecting communities of pathologists, researchers, AI developers, patients, and industry parties.

TU Munich

In this project, , and from , together with will be leading and contributing to the development of Federated Deep Learning algorithms leveraging massive amounts of data, distributed in multiple sources, in a privacy-preserved fashion. This will enable deep learning models to be trained using sensitive data that cannot be made publicly available due to GPDR or sensitivity, e.g. rare diseases. Please visit the website of for further details.

Intended results

The project is divided into four main aspects that concern the large-scale collection of data. First, an infrastructure (hardware and software) must be created to store, share and process millions of images that can be gigabytes each. Second, legal and ethical constraints must be put in place to ensure adequate usage of data while fully respecting patient’s privacy and data confidentiality. Then, an initial set of 3 million digital slides from humans and laboratory animals will be collected and stored into the repository to provide data for the development of pathology AI tools. Finally, functionalities that aid the use of the database as well as the processing of images for diagnostic and research purposes will be developed.

Eyesight to AI: Discovery of efficient corrosion modulators via predictive machine learning

Fri, 01 Jan 2021 00:00:00 +0000

Outcome publication (peer-reviewed):

FUTURE-AI: International guidelines for trustworthy medical AI

Fri, 01 Jan 2021 00:00:00 +0000

This collaboration contributes to the development of international guidelines, evaluation frameworks, and best practices for trustworthy artificial intelligence in medical imaging and clinical decision support. Within the FUTURE-AI consortium, I contributed to survey- and consensus-driven publications synthesizing methodological, clinical, and regulatory perspectives on reliability, safety, fairness, and clinical validation of AI systems. The resulting publications define actionable evaluation criteria, identify common failure modes, and propose harmonized reporting and validation standards that support responsible clinical translation.

Perspective

Engagement in the FUTURE-AI consortium supports sustained international community-building and contributes to the maturation of evaluation standards for medical AI. Ongoing activities focus on refining guidelines and aligning academic research with emerging regulatory and translational requirements.

Interpretable learning for rare disease classification in computational hematology

Fri, 01 Jan 2021 00:00:00 +0000

This research line addresses automated analysis of hematological disorders from microscopic blood images, where rare diseases, limited annotations, and cross-site variability challenge conventional deep learning approaches. Using multi-center microscopy datasets covering several rare anemia subtypes, we developed interpretable and data-efficient learning methods, including anomaly-aware multiple instance learning and unsupervised cross-domain feature extraction. The proposed approaches achieved improved generalization and instance-level interpretability compared to supervised and attention-based baselines.

Perspective

Initiated during my appointment at Helmholtz Munich and continued after transitioning to the University of Bonn, this collaboration established a durable national research partnership and contributed to doctoral training. The methodological insights continue to inform ongoing work on robust and weakly supervised learning for rare diseases.

FedDis: Federated disentangled representation learning for brain anomaly detection

Mon, 01 Jun 2020 00:00:00 +0000

This research line addressed one of the central methodological challenges in collaborative medical AI: how to learn clinically meaningful representations across institutions without sharing patient data and without relying on pixel-level pathology annotations. In the context of brain MRI, unsupervised anomaly detection is particularly attractive because lesion annotation is expensive, heterogeneous, and often unavailable, while privacy and governance constraints limit centralized multi-site learning.

Together with the collaborators, we developed FedDis, a federated disentangled representation-learning framework for unsupervised brain anomaly detection. The key methodological idea was to separate the representation space into global shape components, learned collaboratively across institutions, and local appearance components, trained locally to capture client-specific intensity and scanner characteristics. This disentanglement strategy was designed to address statistical heterogeneity across sites while preserving the privacy-preserving benefits of federated learning.

The resulting system was trained collaboratively on 1,532 healthy MRI scans from four institutions and evaluated on 538 pathological volumes from six institutions, covering multiple sclerosis, vascular lesions, and brain tumors. Compared with locally trained unsupervised models, the approach achieved substantial improvements in anomaly segmentation performance, including marked gains for multiple sclerosis, vascular lesions, and tumors, while requiring neither pathology annotations nor raw data sharing. The work was published in Nature Machine Intelligence and became one of the most visible methodological outputs of my collaborative research in federated medical imaging.

Perspective

This project established an important methodological foundation for my later work on federated and collaborative learning, benchmark development, and multi-institutional validation in medical imaging. Conceptually, it shaped my subsequent research agenda by demonstrating that privacy-preserving learning across hospital networks must address not only optimization and communication constraints, but also representation-level heterogeneity across institutions. The insights from this project continue to inform my current and planned work on collaborative validation, foundation models under distribution shift, and trustworthy medical AI infrastructures such as OMI.

Deep Federated Learning in Healthcare

Wed, 01 Apr 2020 00:00:00 +0000

Deep Learning (DL) has emerged as a leading technology for accomplishing many challenging tasks showing outstanding performance in a broad range of computer vision and medical applications. However, this success comes at the cost of collecting and processing a massive amount of data, which often are not accessible, in Healthcare, due to privacy issues. Federated Learning (FL) has been recently introduced to allow training DL models without sharing the data. Instead, DL models at local hubs, i.e. hospitals, share only the trained parameters with a centralized DL model, which is, in return, responsible for updating the local DL models as well.

Our golas in this project is to develop novel models and algorithms for a ground-breaking new generation of deep FL, which can distill the knowledge from local hubs, i.e. hospitals, and edges, i.e. wearable devices, to provide personalized healthcare services.

The principal challenges, to overcome, concern the nature of medical data, namely data heterogeneity; severe class-imbalance, few amounts of annotated data, inter-/intra-scanners variability (domain shift), inter-/intra-observer variability (noisy annotations), system heterogeneity, and privacy issues (see the example below).

Collaboration:

Digital staining with confidence estimation for hematological imaging

Wed, 01 Jan 2020 00:00:00 +0000

Collaborators: PD Dr.\ Slobodan Ilic (Siemens AG)

This project investigated the reliability of generative image translation for digital staining of hematological microscopy images, aiming to reduce variability and processing time in laboratory workflows while preserving diagnostically relevant structures. Using paired unstained and chemically stained microscopy images from 24 patients, we developed a style–structure disentangled generative framework with an associated confidence estimation that quantifies output reliability via latent perturbations. The confidence score correlated strongly with perceptual image quality and enabled the identification of unreliable synthetic samples.

Our role

Contributing collaborator with responsibility for methodological development and uncertainty estimation; doctoral supervision and mentorship conducted in close collaboration with academic and industrial partners.

Perspective

The project contributed to the successful doctoral training of an external PhD candidate and established methodological foundations for later work on uncertainty estimation and trust in medical image generation. It exemplifies sustained national collaboration between academia and industry.

Modelling Uncertainty in Deep Learning for Medical Applications

Wed, 01 Jan 2020 00:00:00 +0000

Deep Learning has emerged as a leading technology for accomplishing many challenging tasks showing outstanding performance in a broad range of applications in computer vision and medical applications. Despite its success and merit in recent state-of-the-art methods, DL tools still lack in robustness hindering its adoption in medical applications. Modeling uncertainty, through Bayesian Inference and Monte-Carlo dropout, has been successfully introduced to computer vision for better understanding the underlying deep learning models. In this proposal, we investigate modeling the uncertainty for medical applications given the well-known challenges in medical image analysis, namely severe class-imbalance, few amounts of labeled data, domain shift, and noisy annotations.

Collaboration:

Prof. , , .

Prof. , ,

Uncertainty Aware Methods for Camera Pose Estimation and Relocalization

Wed, 01 Jan 2020 00:00:00 +0000

Camera pose estimation is the term for determining the 6-DoF rotation and translation parameters of a camera. It is now a key technology in enabling multitudes of applications such as augmented reality, autonomous driving, human computer interaction and robot guidance. For decades, vision scholars have worked on finding the unique solution of this problem. Yet, this trend is witnessing a fundamental change. The recent school of thought has begun to admit that for our highly complex and ambiguous real environments, obtaining a single solution is not sufficient. This has led to a paradigm shift towards estimating rather a range of solutions in the form of full probability or at least explaining the uncertainty of camera pose estimates. Thanks to the advances in Artificial Intelligence, this important problem can now be tackled via machine learning algorithms that can discover rich and powerful representations for the data at hand. In collaboration, TU Munich and Stanford University plan to devise and implement generative methods that can explain uncertainty and ambiguity in pose predictions. In particular, our aim is to bridge the gap between 6DoF pose estimation either from 2D images/3D point sets and uncertainty quantification through multimodal variational deep methods. Further details can be found here: .

Collaboration:

, , Stanford University

, , , Technical University of Munich

LYSTO: Benchmark and hackathon for lymphocyte assessment

Tue, 01 Jan 2019 00:00:00 +0000

Reliable assessment of tumor-infiltrating lymphocytes in immunohistochemistry is clinically relevant for prognosis and immunotherapy decision-making but remains time-consuming and subject to inter-observer variability. Within the international LYSTO initiative, I contributed to the development and consolidation of a benchmark initially introduced as a MICCAI 2019 hackathon. The benchmark comprises multi-center, multi-organ IHC datasets with over 170,000 cell-level annotations and formulates lymphocyte assessment as a weakly supervised learning problem. Multiple methods achieved pathologist-level agreement across institutions and organs, and extended analyses incorporating community feedback culminated in a comprehensive peer-reviewed publication in 2024.

Our role

Contributing collaborator with responsibility for methodological analysis, evaluation, and consolidation of benchmark results.

Perspective

LYSTO emphasizes the importance of sustained, community-driven benchmarking that extends beyond time-limited challenges. The experience informed subsequent work on multi-center robustness, evaluation, and trustworthy assessment of medical AI.

CAD model-based detection and tracking of large objects

Sun, 01 Jan 2017 00:00:00 +0000

Industry project with Siemens AG on CAD model-based detection and tracking of large objects in RGB(D) video (2017–2019).

Optimisation of X-ray images with respect to image quality and dose

Fri, 01 Jan 2016 00:00:00 +0000

Industry project with Siemens Healthineers on learning-based X-ray image-quality and dose optimisation (2016–2019).

Telemedicine in Palestine

Thu, 01 Jan 2009 00:00:00 +0000

Telemedicine in Palestine from Shadi Nabil Albarqouni