completed2019 – 2024

LYSTO: Benchmark and hackathon for lymphocyte assessment

Reliable assessment of tumor-infiltrating lymphocytes in immunohistochemistry is clinically relevant for prognosis and immunotherapy decision-making but remains time-consuming and subject to inter-observer variability.

Project teaser
Figure from Jiao et al., IEEE JBHI 2023, CC BY 4.0. Images: Jensflorian, Wikimedia Commons, CC BY-SA 4.0.

Abstract

Reliable assessment of tumor-infiltrating lymphocytes in immunohistochemistry is clinically relevant for prognosis and immunotherapy decision-making but remains time-consuming and subject to inter-observer variability. Within the international LYSTO initiative, I contributed to the development and consolidation of a benchmark initially introduced as a MICCAI 2019 hackathon. The benchmark comprises multi-center, multi-organ IHC datasets with over 170,000 cell-level annotations and formulates lymphocyte assessment as a weakly supervised learning problem. Multiple methods achieved pathologist-level agreement across institutions and organs, and extended analyses incorporating community feedback culminated in a comprehensive peer-reviewed publication in 2024.

Our role

Contributing collaborator with responsibility for methodological analysis, evaluation, and consolidation of benchmark results.

Perspective

LYSTO emphasizes the importance of sustained, community-driven benchmarking that extends beyond time-limited challenges. The experience informed subsequent work on multi-center robustness, evaluation, and trustworthy assessment of medical AI.

Partner institutions

Publications 1

LYSTO: The lymphocyte assessment hackathon and benchmark dataset

Yiping Jiao, Jeroen van der Laak, Shadi Albarqouni, Zhang Li, Tao Tan, Abhir Bhalerao, Shenghua Cheng, Jiabo Ma, Johnathan Pocock, Josien P.W. Pluim, Navid Alemi Koohbanani, Raja Muhammad Saad Bashir, Shan E Ahmed Raza, Sibo Liu, Simon Graham, Suzanne Wetstein, Syed Ali Khurram, Xiuli Liu, Nasir Rajpoot, Mitko Veta, Francesco Ciompi

IEEE journal of biomedical and health informatics, 2024