Transfer Learning#
In a previous tutorial, we trained a cats-versus-dogs classifier from randomly initialized weights. Despite using a relatively small dataset, the classifier achieved a reasonable accuracy of 80–84%. In this chapter, we will explore transfer learning, a powerful technique that allows us to use a model that has already been trained on a large dataset, such as ImageNet, and adapt it to our cats-versus-dogs classification task.
Transfer learning works because different visual tasks share useful structure. Early convolutional layers often respond to edges, colors, and textures. Deeper layers combine these patterns into more abstract features. Not every feature transfers equally well, especially when the source and target images are very different, but pretrained weights usually provide a better starting point than random weights.
Two transfer-learning strategies are commonly used.
Feature extraction keeps the pretrained backbone fixed and trains a new head for the target labels. The backbone acts as a reusable image-to-feature converter. This is often effective when the target dataset is small or similar to the source dataset.
Fine-tuning starts from pretrained weights, replaces the task-specific head, and then updates some or all of the backbone with small gradient steps. This lets the representation adapt to the target data, which is beneficial when the target dataset is larger or different from the source dataset.
In this chapter, we first use pretrained models as they are, then apply feature extraction and fine-tuning to cats versus dogs. The same train/validation/test discipline still applies: validation data guides our choices, while test data is reserved for the final estimate.
