Build a CNN

Build a CNN#

(Click the button in the top right corner to download the lab.)

A multilayer perceptron can classify images after flattening them into vectors, but it has no built-in notion that nearby pixels form local patterns or that the same pattern may appear in many positions. A convolutional network introduces exactly those assumptions. Before training a CNN, we will first show the effects of convolution on an ordinary photograph. The objective is to connect the operation of Conv2d to a visible image effect, then use the same operation as a learned component of a classifier.

NOTE: The following tutorials are useful when you need API details.

Tutorial	Use in this lab
CNN — Network Architecture	Review of the overall CNN idea
MLP — Training a Neural Network	Review of the training cycle
MLP — Evaluating a Neural Network	Review of evaluation loop

1. Convolution#

A convolutional layer is easiest to understand when its output can be inspected visually. We will begin with a real photograph bundled with Matplotlib. The photograph is not part of dataset used later for classification. It is simply an image with enough structure (object boundaries, clothing, facial features, and background changes) to make local edge detection visible.

Load an image#

We load a grayscale image with PIL, then apply a transformation to convert it to a float tensor with values in the range [0, 1]. After this transformation, the tensor has shape (channels, height, width). We then add a batch dimension to the tensor so that it can be passed to a convolutional layer. Run the cell below and inspect the result.

Checkpoint#

The following properties are assumptions used by every convolution in this section.

# TODO: assert that photo_tensor has shape (1, 1, 600, 512)
# TODO: assert that photo_tensor is floating point
# TODO: assert that all pixel values lie between 0 and 1

Edge detectors#

A learned convolutional layer eventually discovers its own kernels. For the moment, we will set the weights ourselves so that we know what the layer is looking for. The two kernels below are Sobel-style edge detectors. One responds strongly to changes from left to right; the other responds strongly to changes from top to bottom.

vertical_kernel = torch.tensor(
    [[-1.0, 0.0, 1.0],
     [-2.0, 0.0, 2.0],
     [-1.0, 0.0, 1.0]]
)

horizontal_kernel = torch.tensor(
    [[-1.0, -2.0, -1.0],
     [ 0.0,  0.0,  0.0],
     [ 1.0,  2.0,  1.0]]
)

Exercise 1#

A Conv2d layer with one input channel and two output channels contains two complete kernels. You will copy the vertical detector into the first kernel and the horizontal detector into the second. Remember that the weight attribute of a convolutional layer is a tensor of shape: (out_channels, in_channels, kernel_height, kernel_width)

edge_detector = nn.Conv2d(
    in_channels=1,
    out_channels=2,
    kernel_size=3,
    bias=False,
)

with torch.no_grad():
    # TODO: copy vertical_kernel into the first output kernel
    # TODO: copy horizontal_kernel into the second output kernel
    pass

with torch.inference_mode():
    edge_maps = edge_detector(photo_tensor)

print("Edge-map shape:", edge_maps.shape)

The convolution produces signed responses. Reversing an intensity transition reverses the sign of the response. For visualization, the magnitude is often more convenient because it shows where the detector responds strongly regardless of orientation.

Analysis#

Inspect several visible boundaries in the photograph.

Which structures produce strong responses in the vertical map?
Which structures appear more strongly in the horizontal map?
Explain why the output is still an image-like grid.
Explain weight sharing and local connectivity.

Convolution settings#

The kernel determines what local pattern is measured. Padding and stride determine where and how densely the measurement is made.

We will keep the vertical edge detector fixed and change only these settings. This lets us attribute visible differences to the convolution geometry rather than to a different filter.

Exercise 2 #

For a 600 × 512 image and a 3 × 3 kernel, predict the spatial output shape for each configuration before running the code.

Padding	Stride	Predicted height × width
0	1
1	1
1	2

Then, complete the function below so that it constructs a convolution, copies the fixed kernel into the layer, and returns the resulting feature map.

def apply_fixed_kernel(image, kernel, padding=0, stride=1):

    layer = nn.Conv2d(1, 1, 3, stride, padding, bias=False)

    with torch.no_grad():
        # TODO: copy kernel into layer.weight[0, 0]
        layer.weight[0, 0] = None # YOUR CODE HERE
        pass

    with torch.inference_mode():
        # TODO: apply the layer to image
        response = None # YOUR CODE HERE

    return response


#--- Test ---#

print("Original photo shape:", tuple(photo_tensor.shape))

setting_results = []
for padding, stride in [(0, 1), (1, 1), (1, 2)]:
    response = apply_fixed_kernel(photo_tensor, vertical_kernel, padding, stride)
    setting_results.append((padding, stride, response))
    print(f"padding={padding}, stride={stride} → {tuple(response.shape)}")

Run the cell below to see the actual outputs.

Analysis#

Compare the three feature maps. Explain what information is lost at the boundary without padding and why stride two produces a lower-resolution representation. The kernel weights did not change, so what exactly changed about the computation?

2. Convolutional layer#

In a CNN, the convolutional kernel values are ordinary model parameters and backpropagation adjusts them to reduce the classification loss. A convolutional layer usually learns several kernels at once. Each kernel produces one output feature map, so the number of kernels is the number of output channels.

Several learned filters#

Consider a layer with one grayscale input channel and eight output channels.

learned_conv = nn.Conv2d(
    in_channels=1,
    out_channels=8,
    kernel_size=3,
    padding=1,
)

print("Weight shape:", tuple(learned_conv.weight.shape))
print("Bias shape:", tuple(learned_conv.bias.shape))

Convolution blocks#

Convolution remains an affine operation, so it is normally followed by a nonlinear activation. Pooling can then reduce the spatial resolution while preserving the number of channels.

block = nn.Sequential(
    nn.Conv2d(1, 8, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(kernel_size=2),
)

Exercise 3#

Starting from (1, 1, 600, 512), predict the shape after the convolution, after ReLU, and after max pooling. Also calculate the number of trainable parameters in each layer. Fill in the table below.

Operation	Predicted shape	Trainable parameters
Conv2d
ReLU
MaxPool2d

Then, pass the photograph through each operation and record the shape.

activation = photo_tensor
block_shapes = []

for layer in block:
    # TODO: apply layer to activation
    activation = None # YOUR CODE HERE
    block_shapes.append((layer.__class__.__name__, tuple(activation.shape)))

block_shapes

Checkpoint#

Verify that the parameter counts match your predictions. If they do not, check your calculations and the convolution settings.

# TODO: assert the total number of trainable parameters in Conv2d
# TODO: assert MaxPool2d contributes no trainable parameters
conv_parameters = sum(parameter.numel() for parameter in block[0].parameters())
pool_parameters = sum(parameter.numel() for parameter in block[2].parameters())

3. Build a CNN#

We have now seen convolution act directly on a photograph. The next step is to let the filters be learned for an actual classification task. FashionMNIST contains 28 × 28 grayscale images from ten clothing categories. The input already has the spatial form expected by a CNN, so we will keep the channel, height, and width dimensions intact until the final classifier.

Prepare the data#

The original training partition contains 60,000 examples. As in the earlier labs, we will reserve 6,000 for validation and use the official FashionMNIST test set only after model selection.

DATA_DIRECTORY = ".data"

transform = v2.Compose([v2.ToImage(), v2.ToDtype(torch.float32, scale=True)])

train_source = datasets.FashionMNIST(root=DATA_DIRECTORY, train=True,  transform=transform, download=True)
test_dataset = datasets.FashionMNIST(root=DATA_DIRECTORY, train=False, transform=transform, download=True)

split_generator = torch.Generator().manual_seed(SEED)
training_ds, validation_ds = random_split(train_source, [54_000, 6_000], split_generator)

class_names = train_source.classes

The dataloaders are constructed with a batch size of 128. The training dataloader shuffles the data, while the validation and test dataloaders do not.

BATCH_SIZE = 128

def make_training_loader(seed=SEED):
    return DataLoader(training_ds, BATCH_SIZE, shuffle=True, generator=torch.Generator().manual_seed(seed))

def make_unshuffled_loader(ds, seed=SEED):
    return DataLoader(ds, BATCH_SIZE, shuffle=False, generator=torch.Generator().manual_seed(seed))


training_loader = make_training_loader()
validation_loader = make_unshuffled_loader(validation_ds)
test_loader = make_unshuffled_loader(test_dataset)

images, targets = next(iter(training_loader))
print("Image batch:", tuple(images.shape))
print("Target batch:", tuple(targets.shape))

Assemble the CNN#

The model follows the architecture developed in the lesson. Two convolutional blocks construct spatial feature maps, after which the final 7 × 7 maps are flattened and converted to ten logits.

Exercise 4#

Complete the SmallCNN class below. It should contain two convolutional blocks, each with a convolutional layer, a ReLU activation, and a max pooling layer. The final output should be flattened and passed through a linear layer to produce ten logits.

class SmallCNN(nn.Module):
    def __init__(self):
        super().__init__()

        self.features = nn.Sequential(
            # TODO: 1 → 16 channels, 3×3 kernel, preserve spatial size
            nn.ReLU(),
            nn.MaxPool2d(2),

            # TODO: 16 → 32 channels, 3×3 kernel, preserve spatial size
            nn.ReLU(),
            nn.MaxPool2d(2),
        )

        self.classifier = nn.Sequential(
            nn.Flatten(),
            # TODO: map the final 32×7×7 representation to 10 logits
        )

    def forward(self, images):
        features = self.features(images)
        logits = self.classifier(features)
        return logits

Before training, trace one batch through the model. Predict each shape first, then run the code.

Checkpoint#

Verify the shape of parameter weights.

# TODO: assert that the first convolution has weight shape (16, 1, 3, 3)
# TODO: assert that the second convolution has weight shape (32, 16, 3, 3)
# TODO: assert that the final layer has weight shape (10, 1568)

Analysis#

Identify the point at which the representation stops being spatial.

4. Train and Inspect the CNN#

Nothing special is required to optimize convolutional parameters. They participate in the same differentiable computation as the weights of a linear layer. The training workflow therefore remains the one developed earlier: batches produce losses, backpropagation calculates gradients, the optimizer updates parameters, validation tracks generalization, and the best validation checkpoint is restored.

Training workflow#

The functions below are supplied infrastructure. Their purpose here is to keep attention on the CNN rather than on rebuilding the training loop.

Exercise 5#

Train the CNN with cross-entropy loss and Adam with learning rate 1e-3. Five epochs are enough for this lab: the purpose is to observe the architecture in a complete workflow, not to exhaustively tune FashionMNIST.

torch.manual_seed(SEED)
cnn = SmallCNN().to(device)

loss_fn = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(cnn.parameters(), lr=1e-3)

EPOCHS = 5

cnn_training_loader = make_training_loader(SEED)

# TODO: train cnn with fit(...), using cnn_training_loader
cnn_history, cnn_checkpoint = None # YOUR CODE HERE

print(f"Selected CNN checkpoint\n{'\n'.join(f' - {k}: {v}' for k, v in cnn_checkpoint.items())}")

Plot the resulting histories.

Analysis#

Did optimization progress?
How strong was the training fit?
Did validation performance improve with training?
Which epoch was retained by the validation-loss checkpoint?

First convolution kernels#

At the beginning of the lab, we inserted edge detectors by hand. The first CNN layer has instead learned sixteen 3 × 3 kernels from the classification loss. Let’s visualize them after training.

Now apply the first convolution and ReLU to one FashionMNIST image.

Analysis#

Explain the difference between the sixteen learned kernels and the sixteen feature maps produced for this particular image. Which one is a learned parameter of the model, and which one depends on the current input? Avoid assigning a precise human meaning to every channel unless the visual evidence supports it.

5. Compare with an MLP#

Let’s compare the CNN with a multilayer perceptron (MLP). The CNN and an MLP can use the same loss function, optimizer, batches, validation procedure, and checkpoint rule. The architectural difference is how they represent the image.

Train the MLP#

The MLP flattens the image immediately. Its hidden units receive position-specific weights for all 784 pixels.

class FashionMLP(nn.Module):
    def __init__(self, hidden_width=128):
        super().__init__()
        self.network = nn.Sequential(
            nn.Flatten(),
            nn.Linear(28 * 28, hidden_width),
            nn.ReLU(),
            nn.Linear(hidden_width, 10),
        )

    def forward(self, images):
        return self.network(images)

Exercise 6#

Train the MLP using the same training parameters used for the CNN.

torch.manual_seed(SEED)
mlp = FashionMLP(hidden_width=128).to(device)

mlp_optimizer = torch.optim.Adam(mlp.parameters(), lr=1e-3)

mlp_training_loader = make_training_loader(SEED)

# TODO: train mlp with the same fit(...) workflow, using mlp_training_loader
mlp_history, mlp_checkpoint = None # YOUR CODE HERE

Evaluate the restored CNN and MLP checkpoints on the validation set and compare their parameter counts.

Analysis#

Compare the validation results and parameter counts.

Did the CNN provide stronger evidence of generalization under this training protocol?
How does the opening edge-detection experiment help explain why convolution is an appropriate architectural choice for images?

Evaluate once on test#

Now, we’ll select the best model based on validation loss and evaluate it once on the untouched test set.

candidates = [
    ("MLP", mlp, mlp_valid),
    ("CNN", cnn, cnn_valid),
]

selected_name, selected_model, selected_validation = min(candidates, key=lambda item: item[2]["loss"])

selected_test = evaluate(selected_model, test_loader, loss_fn, device)

print("Selected model:", selected_name)
print(f"Validation accuracy: {selected_validation['accuracy']:.1%}")
print(f"Test accuracy: {selected_test['accuracy']:.1%}")

Build a CNN

Contents

Build a CNN#

1. Convolution#

Load an image#

Checkpoint#

Edge detectors#

Exercise 1#

Analysis#

Convolution settings#

Exercise 2 #

Analysis#

2. Convolutional layer#

Several learned filters#

Convolution blocks#

Exercise 3#

Checkpoint#

3. Build a CNN#

Prepare the data#

Assemble the CNN#

Exercise 4#

Checkpoint#

Analysis#

4. Train and Inspect the CNN#

Training workflow#

Exercise 5#

Analysis#

First convolution kernels#

Analysis#

5. Compare with an MLP#

Train the MLP#

Exercise 6#

Analysis#

Evaluate once on test#