Not Every Label-Efficient Method Solves the Same Problem

Part 1 of a short series on label-efficient learning in medical imaging.

Teams building AI for medical imaging are offered a long list of ways to spend less on labeling: semi-supervised learning, self-supervised pre-training, multi-instance learning, few-shot learning, and weak annotation. Each is usually described the same way — as a method that reduces the annotation burden. On the surface that makes them look interchangeable, and it raises a fair question: if they all cut the cost of labeling, does the choice between them matter?

It does, and the reason is easy to miss. These methods do not reduce the same cost. Separating them by what they actually save is the fastest way to see why one of them, core-set selection, belongs at the very start of the process rather than in the interchangeable pile.

The most thorough reference on the landscape is a 2023 survey by Jin and colleagues, which reviewed more than three hundred label-efficient methods in medical imaging (arXiv:2303.12484). A simple pattern can be seen throughout.

Semi-supervised and self-supervised learning reduce the number of labels required. They lean on large amounts of unlabeled data to do more with fewer annotations. But they keep every image in the dataset, and they usually add computation rather than remove it: an extra pre-training run, a second model, additional training objectives.

Weak annotation reduces the cost of each individual label. Instead of a careful pixel-level outline, an annotator places a dot, draws a box, or adds a tag. Every image is still labeled, but each label is faster to produce.

Multi-instance learning reduces the level of detail a label requires, letting a single label cover an entire slide instead of every region within it.

Active learning chooses images according to the current model's uncertainty, on the assumption that the samples the model is least sure about are the most informative. That assumption has a well-documented weakness: a model's confidence is not a reliable measure of its correctness. Modern deep networks are frequently miscalibrated and overconfident, assigning high confidence to predictions that are in fact wrong (Guo et al., 2017).

Few-shot learning reduces the labels needed to handle new categories, though it assumes a strong model already exists to build on.

Every one of these methods takes the training set as fixed. Their job is to extract more from that set, or to label it more cheaply. None of them asks the prior question: is this the right set of images to be labeled?

That is the question core-set selection answers. Instead of treating every image as a given, it changes the composition and size of the labeled set, choosing a small, representative subset of the raw data worth annotating at all. It is a different move on a different variable, and it is why the choice of method is not cosmetic.

The distinction has real consequences in medicine, where the dominant cost is not compute but a radiologist's or pathologist's time for each annotation. A method that makes each label cheaper helps at the margin. A method that reduces how many expert labels are needed changes the economics of the whole program.

It also matters for what the model learns. A dataset full of near-identical images teaches a model to memorize the common case and stay blind to the rare one, and most label-saving methods inherit whatever imbalance and redundancy the data already contains. Curating the set first shapes everything downstream.

None of this makes the other methods useless. It reframes their relationship. Core-set selection is not a competitor to semi-supervised learning or weak annotation; it runs ahead of them. Choose the right subset first, then apply whichever label-saving method was already planned, now to a leaner, better-chosen dataset.

The next two pieces in this series look closer: first at why self-supervised learning, the method most often held up as the strongest, has weaknesses that are hard to see; then at how core-set selection actually works, and where it can go wrong.