Conclusion#
A common challenge in deep learning is working with very limited data. Training a convolutional network from scratch on a small dataset is often impractical due to overfitting. Fortunately, transfer learning provides a way around this limitation. A pretrained model can generalize well even with relatively little data, especially when the task is simple and the model is well-regularized. The effectiveness of transfer learning is due to the fact that the features learned by the convolutional backbone are often general, and can be used to solve a wide range of problems, not just the ones they were trained on.
Choosing a strategy#
Transfer learning is often the best starting point for a small vision dataset, but there is no single strategy that always wins. Use the following questions as a practical guide.
Do the pretrained labels already solve the task?
Use the model directly with its matching preprocessing and evaluate it on representative data.
Is the target dataset small and visually similar to the source data?
Start with feature extraction.
Extracting and caching features is fastest when deterministic preprocessing is sufficient.
Does the frozen model underfit or face moderate domain shift?
Fine-tune a few late blocks with small learning rates.
Is there abundant labeled data or a large domain shift?
Compare broader fine-tuning against training from scratch.
Pretraining is still a useful baseline, but it may no longer be decisive.
Recap questions#
For each scenario below, choose a starting strategy and explain what evidence would make you change it:
800 labeled photographs of two common animal species;
20,000 grayscale microscope images with subtle class differences;
one million labeled street-scene images from a new camera system.
For your chosen scenario, specify the source weights and transforms, trainable layers, learning rates, and one diagnostic for domain shift. There is not always one correct choice. The goal is to make assumptions explicit and to reason about how to validate them.