Not Every Label-Efficient Method Solves the Same Problem

Where Core-Set Selection Fits - and Where It Doesn't

Part 3 of a short series on label-efficient learning in medical imaging.

Most methods for cutting labeling costs in medical imaging share a hidden assumption: the dataset is fixed, and the task is to label it more cheaply or squeeze more from it. Core-set selection makes a different move. It asks which images are worth labeling in the first place, and keeps only a small, representative subset of the raw data.

It belongs to the broader family of sample-selection methods, and it is worth separating from active learning, which is often mentioned in the same breath but works on a different principle. Active learning selects the samples a model is currently most uncertain about. Core-set selection selects a subset that best represents the data as a whole, chosen for coverage and diversity rather than volume. The two carry different failure modes, a distinction that matters later.

The idea is to funnel a large, redundant, imbalanced dataset down to a small set of images chosen to cover it, so that a model trained on the subset generalizes much like one trained on everything. Why does that matter?

First, cost. In medical imaging the dominant expense is not tooling or compute but the time of a radiologist or pathologist for each annotation. Weak-labeling methods make each annotation cheaper; core-set selection reduces how many are needed at all, a more durable saving. The figures in the literature are not marginal: in the Jin et al. survey, a skin-lesion model reached an AUC of 0.75 using roughly twenty-two percent of the images (Gal et al., 2017), and a colon detection task reached an AUC of 0.96 using about two percent of the data (Zhou et al., 2021).

Second, defensibility. Core-set selection changes which images enter training and then hands the rest of the process to ordinary supervised learning. Many alternatives bolt on less predictable machinery - pseudo-labels that can be wrong, generative models that can collapse, meta-learning loops - each adding parameters and new ways to fail quietly. In a program heading toward FDA or CE approval, a simpler pipeline is an asset, and a documented record of which images the model was trained on, and why, is far easier to defend to a reviewer than behavior buried inside a model.

Third, control over the data distribution. Selecting deliberately, for coverage and diversity, makes it possible to keep the rare, hard cases that decide whether a product works in practice while dropping redundant, near-identical ones. That is a direct answer to the imbalanced datasets that undermine medical models.

On a pneumothorax detection task - a small, life-threatening finding in a heavily imbalanced chest X-ray dataset - a core-set selection strategy was compared against random sampling at fixed labeling budgets. The gap was widest where it matters most: at 5% of the data, F1 was 0.62 versus 0.57, and at 1% it was 0.46 versus 0.44. By 50% of the data the two converged to 0.73 and 0.70. Selection does not make the model smarter; it removes the redundant majority that random sampling keeps re-buying, and its advantage shrinks as the budget grows large enough to cover the distribution anyway. See details here

It pays off when the dataset is redundant or imbalanced and when deployment data drifts from training data — which is precisely the medical-imaging reality - and it is roughly a coin-flip against random sampling when those conditions are absent. For the data drift case follow the link.

Fourth, it composes. A dataset reduced to its most informative core can then be handed to semi-supervised learning, weak labeling, or self-supervised pre-training. Core-set selection is not a rival to those methods; it runs ahead of them.

None of this makes it free of trade-offs, and two deserve to be stated plainly.

The first is a cold-start problem. Selecting the most useful images requires an initial model or a learned representation to guide the choice. Self-supervised learning has the edge here, since it can start from a completely unlabeled pile. The dependency is real and needs planning around.

The second is selection bias, and this is where the distinction from active learning pays off. Active learning leans on model uncertainty, and a model's confidence is not a reliable measure of its correctness: modern networks are frequently overconfident, assigning high confidence to predictions that are wrong (Guo et al., 2017). A confidently wrong case is never flagged for review, and if the pipeline auto-labels from those predictions, the errors are written straight into the training set.

Core-set selection sidesteps that particular trap, because it selects for representativeness rather than uncertainty. But it has a bias of its own: it judges similarity in a learned feature space, so a representation that already underweights a rare presentation can lead the selector to discard those images as redundant. The quality of the representation, and an explicit effort to keep rare cases, matter as much as the selection algorithm.

The summary is simple. Other methods take a fixed dataset and make it cheaper to label or richer to learn from. Core-set selection changes the dataset, reducing the labeled set to a representative subset, which lowers the most expensive input, keeps the model simpler to defend, and leaves room for everything else to build on what it selects. In a field where the labeling bill is a clinician's hours and regulators expect a clear account of how a model was built, that is the sensible place to begin, provided the selecting is done with care.

At FiveBrane, this is where the work starts: finding the high-value subset of an imaging dataset so regulatory-grade accuracy can be reached on a fraction of the labels, using data that never has to leave the hospital.