completed2021 – 2022

FLamby: Federated learning benchmarks for realistic healthcare settings

A central bottleneck in federated learning research is the lack of realistic, multi-center benchmarks reflecting the heterogeneity of clinical data.

Project teaser
Images: Juliana Casagrande Tavoloni Braga et al., Wikimedia Commons, CC BY 4.0; Mikael Häggström, M.D, Wikimedia Commons, CC0; Mikael Häggström. When using this image in external works et al., Wikimedia Commons, CC0.

Abstract

Collaborators: EPFL (Switzerland), INRIA (France), University of Southern California (USA), Institut Polytechnique de Paris (France), University of California, Berkeley (USA)

A central bottleneck in federated learning research is the lack of realistic, multi-center benchmarks reflecting the heterogeneity of clinical data. Within this international collaboration, I contributed to the development of FLamby, a benchmark suite comprising seven real-world medical datasets with natural institutional splits across histopathology, radiology, dermatology, and tabular clinical data. Published at NeurIPS 2022 (Datasets and Benchmarks Track), FLamby demonstrated that standard federated learning approaches often underperform pooled training and highlighted the need for heterogeneity-aware optimization methods.

Our role

Contributing collaborator with responsibility for dataset preparation, benchmark design, and evaluation of federated learning methods.

Perspective

FLamby established a widely used community reference for cross-silo federated learning in healthcare. Its open-source release continues to inform benchmarking practices and supports subsequent federated learning studies and methodological developments.

Partner institutions

Publications 1

Flamby: Datasets and benchmarks for cross-silo federated learning in realistic healthcare settings

Jean Ogier Du Terrail, Samy-Safwan Ayed, Edwige Cyffers, Felix Grimberg, Chaoyang He, Regis Loeb, Paul Mangold, Tanguy Marchand, Othmane Marfoq, Erum Mushtaq, Boris Muzellec, Constantin Philippenko, Santiago Silva, Maria Telenczuk, Shadi Albarqouni, Salman Avestimehr, Aurélien Bellet, Aymeric Dieuleveut, Martin Jaggi, Sai Praneeth Karimireddy, Marco Lorenzi, Giovanni Neglia, Marc Tommasi, Mathieu Andreux

Advances in Neural Information Processing Systems, 2022