Self-Supervised Learning for Early Disease Detection in Low-Label Medical Image Datasets

Abstract

Getting reliable labeled data for medical image analysis is genuinely difficult. Radiologists, pathologists, and other specialist clinicians are expensive, slow, and not universally available — particularly in low-resource clinical environments across developing nations. Unlike natural image benchmarks such as ImageNet, which provide millions of meticulously labeled training examples, most publicly accessible medical imaging repositories offer only a few hundred to a few thousand annotated scans. Supervised deep learning models trained on such limited annotations are prone to severe overfitting and generalize poorly to new patient populations, imaging devices, or clinical protocols. Self-supervised learning (SSL) offers a principled solution to this problem by enabling neural networks to learn rich, transferable visual representations from large volumes of unlabeled data, without requiring any human annotation during the representation learning phase. By constructing pretext tasks that generate training signals directly from the raw image data — through augmentation invariance, masked patch reconstruction, or self-distillation — SSL methods acquire structured feature representations that can subsequently be fine-tuned effectively using only a small labeled set. This paper presents a comprehensive evaluation of four prominent SSL frameworks — SimCLR, MoCo v2, Masked Autoencoders (MAE), and DINO — applied to early disease detection across three clinically significant medical imaging modalities: chest radiographs, retinal fundus photographs, and histopathological tissue slides. We conduct systematic benchmarking under low-label conditions with label fractions of 1%, 5%, and 10% of the available training data. Beyond benchmarking, we introduce MedSSL, a novel hybrid SSL framework specifically designed for medical imaging that integrates contrastive instance discrimination with masked image reconstruction through a modality-adaptive weighted joint loss. Experimental results across all three datasets demonstrate that MedSSL consistently outperforms every single-objective SSL baseline in all nine evaluation conditions (three datasets × three label fractions), achieving a mean AUROC improvement of 4.7% over the best competing SSL method. At the extreme low-label setting of 1%, MedSSL outperforms supervised training from random initialization by over 16 AUROC points. The incorporation of domain-specific augmentation pipelines — tailored separately for chest radiography, retinal imaging, and histopathology — provides an additional 2.1% AUROC gain over generic augmentation strategies. These findings collectively establish SSL as a viable, practical, and scalable pre-training paradigm for clinical AI systems that are constrained by limited annotation budgets.