Disentangled, fair and discriminative representations for ophthalmic imaging

Industry project with Carl Zeiss Meditec on representation learning for medical imaging — microaneurysm segmentation for diabetic-retinopathy screening, interpretable disentangled representations, and representations that are invariant to sensitive attributes for fair and privacy-preserving AI.

Abstract

Data-driven models only reach clinical practice if what they learn is discriminative enough for subtle findings, compact and interpretable enough to be inspected, and free of information it should not rely on. Together with Carl Zeiss Meditec, the project addressed all three through representation learning, starting from retinal imaging for diabetic-retinopathy screening.

What we did

Finding the smallest lesions. Microaneurysms are the earliest visible sign of diabetic retinopathy and occupy only a few pixels of a fundus image. A two-stage approach segments them on multiple scales with a region-proposal fully convolutional network, then refines the candidates with a patch-wise network trained with an embedding triplet loss and selective sampling of hard patches — a 30% relative improvement over a plain fully convolutional network (MICCAI 2019; extended in IET Image Processing).
Interpretable latent spaces. An adversarial variational autoencoder with a total-correlation constraint enforces independent latent factors while keeping reconstruction fidelity, improving disentanglement by 81.5% and making the representation useful for clustering and for classification with few labels (DART @ MICCAI 2019).
Fairness by orthogonal disentanglement. Splitting the representation into a target code and a sensitive code, enforcing orthogonality between them and maximising the entropy of the target code with respect to the sensitive attribute yields representations that are invariant to it — state of the art on three of five benchmarks (ECCV 2020) — and carries over to privacy-preserving medical applications such as autism detection and healthcare insurance (DART/DCL @ MICCAI 2020).

Our role

Scientific lead of the project as postdoc at TUM CAMP under Nassir Navab, and co-supervision — with Abouzar Eslami (Carl Zeiss Meditec) — of Hasan Sarhan’s industrial PhD, completed at TU Munich in 2024.

Perspective

The work on disentangled and orthogonal representations became a building block of our later research on fairness, privacy and federated learning in medical imaging.