Disentangled, fair and discriminative representations for ophthalmic imaging

Mon, 01 Jan 2018 00:00:00 +0000

Data-driven models only reach clinical practice if what they learn is discriminative enough for subtle findings, compact and interpretable enough to be inspected, and free of information it should not rely on. Together with Carl Zeiss Meditec, the project addressed all three through representation learning, starting from retinal imaging for diabetic-retinopathy screening.

What we did

Finding the smallest lesions. Microaneurysms are the earliest visible sign of diabetic retinopathy and occupy only a few pixels of a fundus image. A two-stage approach segments them on multiple scales with a region-proposal fully convolutional network, then refines the candidates with a patch-wise network trained with an embedding triplet loss and selective sampling of hard patches — a 30% relative improvement over a plain fully convolutional network ( ; extended in ).
Interpretable latent spaces. An adversarial variational autoencoder with a total-correlation constraint enforces independent latent factors while keeping reconstruction fidelity, improving disentanglement by 81.5% and making the representation useful for clustering and for classification with few labels ( ).
Fairness by orthogonal disentanglement. Splitting the representation into a target code and a sensitive code, enforcing orthogonality between them and maximising the entropy of the target code with respect to the sensitive attribute yields representations that are invariant to it — state of the art on three of five benchmarks ( ) — and carries over to privacy-preserving medical applications such as autism detection and healthcare insurance ( ).

Our role

Scientific lead of the project as postdoc at TUM CAMP under Nassir Navab, and co-supervision — with Abouzar Eslami (Carl Zeiss Meditec) — of Hasan Sarhan’s industrial PhD, completed at TU Munich in 2024.

Perspective

The work on disentangled and orthogonal representations became a building block of our later research on fairness, privacy and federated learning in medical imaging.

Zeiss |

Disentangled, fair and discriminative representations for ophthalmic imaging

What we did

Our role

Perspective