Convolutional Networks

Use this page to quickly reference the tensor shapes, convolution and downsampling operations, parameter counts, receptive fields, and shape calculations needed to construct convolutional networks.

Image Tensors

PyTorch image models normally use the channels-first convention.

Object	Shape
Grayscale image	(1, H, W)
RGB image	(3, H, W)
Image with C channels	(C, H, W)
Image batch	(N, C, H, W)

Legend: N = batch size, C = channels, H = height, W = width.

Convolution

Conv2d

Create a two-dimensional convolutional layer with Conv2d.

conv = torch.nn.Conv2d(in_channels=C_in, out_channels=C_out, kernel_size=K)

The layer transforms an input tensor of shape (N, C_in, H, W) to an output tensor of shape (N, C_out, H_out, W_out). Each output channel corresponds to one learned filter applied across all input channels.

Warning

The in_channels argument of a convolution must match the channel dimension of its input tensor.

The most frequently used arguments are the following.

Argument	Meaning	Default
in_channels	Number of input channels	Mandatory
out_channels	Number of learned filters and output channels	Mandatory
kernel_size	Spatial size of the filters (int or tuple)	Mandatory
stride	Distance between evaluated spatial positions	1
padding	Values added around the input boundary	0
bias	Whether the layer learns one bias per output channel	True

Learnable Parameters

For a Conv2d layer with C_in input channels, C_out output channels, and a kernel of height K_h and width K_w, the weight tensor and bias tensor have shapes:

weight: (C_out, C_in, K_h, K_w)
bias:   (C_out,)

With bias enabled, the total number of learnable parameters is therefore

\[ \textsf{Parameter Count} = \underbrace{C_{\text{out}} C_{\text{in}} K_h K_w}_{\text{weights}} + \underbrace{C_{\text{out}}}_{\text{biases}} \]

The parameter tensors can be accessed directly with:

conv = torch.nn.Conv2d(C_in, C_out, (K_h, K_w))

conv.weight
conv.bias

Spatial Dimensions

The output size \(O\) of a convolutional layer can be calculated from the input size \(I\), kernel size \(K\), padding \(P\), and stride \(S\). For one spatial dimension, the formula is:

\[ O = \left\lfloor\frac{I + 2P - K}{S} + 1\right\rfloor. \]

Notable special cases for odd kernel sizes.

• If stride=1 and padding=(kernel_size-1)/2, the output size equals the input size.

• If stride=S and padding=(kernel_size-1)/2, the output size is \(\lceil I/S \rceil\).

The formula assumes no dilation. The official PyTorch documentation includes dilation and other edge cases.

Downsampling

MaxPool2d

Max pooling reduces spatial dimensions without changing the number of channels.

pool = torch.nn.MaxPool2d(kernel_size=K)

Pooling layers do not have learnable parameters. They perform a fixed local aggregation of the input. For example, MaxPool2d outputs the maximum value in each local region of the input. Other pooling operations, such as average pooling, compute different local statistics.

The most frequently used arguments are the following.

Argument	Meaning	Default
kernel_size	Spatial size of the local region where pooling is applied	Mandatory
stride	Distance between evaluated spatial positions	kernel_size
padding	Values added around the input boundary	0

The output size \(O\) of a pooling layer can be calculated from the input size \(I\), kernel size \(K\), padding \(P\), and stride \(S\). For one spatial dimension, the formula is the same as convolution:

\[ O = \left\lfloor\frac{I + 2P - K}{S} + 1\right\rfloor. \]

With default values for stride and padding, the output size is \(\lfloor I/K \rfloor\).

Strided Convolution

A convolution can also perform downsampling by using stride > 1. Both pooling and strided convolution reduce spatial resolution, but they perform different operations.

• MaxPool2d(...) performs a fixed local aggregation of the input.

• Conv2d(..., stride = ...) performs a learned transformation and downsampling of the input.

Convolutional Networks

A convolutional network combines convolutions, nonlinearities, and downsampling operations into a sequence of layers. Intermediate representations preserve the spatial organization of the input, while the number of channels and spatial resolution are progressively transformed as data moves through the layers.

Feature Extractor

A sequence of convolutional operations can be grouped into a feature extractor:

features = nn.Sequential(
    nn.Conv2d(1, 16, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2),

    nn.Conv2d(16, 32, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2),
)

Each layer must be designed to receive a tensor compatible with the output of the preceding layer. In this example, the first convolution produces 16 channels. ReLU and max pooling preserve the channel count, so the second convolution is set to receive 16 input channels.

The feature extractor transforms a batch of grayscale images from shape (N, 1, H, W) to a spatial representation with shape (N, 32, H//4, W//4). What happens to this representation afterward depends on the task. A downstream component may preserve the spatial dimensions, aggregate them, flatten them, or otherwise transform the representation.

Shape Tracing

It is important to trace the tensor shape through each operation when constructing a convolutional network. This verifies that consecutive layers are compatible and makes changes in spatial resolution explicit.

For the feature extractor above, a batch of 28 × 28 grayscale images is transformed as follows.

Layer	Output shape
Input	(N,  1, 28, 28)
Conv2d(1, 16, 3, padding=1)	(N, 16, 28, 28)
ReLU	(N, 16, 28, 28)
MaxPool2d(2)	(N, 16, 14, 14)
Conv2d(16, 32, 3, padding=1)	(N, 32, 14, 14)
ReLU	(N, 32, 14, 14)
MaxPool2d(2)	(N, 32,  7,  7)

Convolution may change both the channel count and the spatial dimensions. Element-wise activations such as ReLU preserve the complete shape. Pooling preserves the channel count while reducing the spatial dimensions.

Receptive Field

As convolution and pooling layers are composed, each activation can depend on an increasingly large region of the original input. The receptive field of an activation is the region of the original input that can influence it. The effective stride measures the spacing between neighbouring activations relative to the original input.

\[\begin{split} \begin{aligned} r_\ell &= \textsf{size of the receptive field after layer $\ell$}\\ j_\ell &= \textsf{effective stride after layer $\ell$} \end{aligned} \end{split}\]

For a layer with kernel size \(k_\ell\) and stride \(s_\ell\), the receptive field and effective stride can be calculated recursively from the previous layer, with \(r_0 = j_0 = 1\) at the input layer.

\[\begin{split} \begin{aligned} r_\ell &= r_{\ell-1} + (k_\ell - 1) j_{\ell-1}\\ j_\ell &= j_{\ell-1} s_\ell \end{aligned} \end{split}\]

For example, consider the following sequence of convolution and pooling layers.

Stage	Kernel	Stride	Receptive field	Effective stride	Spatial Dimensions
Input	—	—	1	1	\(H \times W\)
Conv	3	1	3	1	\(H \times W\)
Pool	2	2	4	2	\(H/2 \times W/2\)
Conv	3	1	8	2	\(H/2 \times W/2\)
Pool	2	2	10	4	\(H/4 \times W/4\)

Flattening

If the final spatial representation must be converted into one vector per example, flattening from the first non-batch dimension transforms a tensor of shape (N, C, H, W) into a tensor of shape (N, C * H * W). This is commonly done before connecting to a standard linear layer.

For example, the final feature maps with shape (N, 32, 7, 7) can be flattened to (N, 1568) before connecting to a linear layer. If the task is classification, the convolutional feature extractor can be followed by a classification head that maps the flattened features to class scores.

classifier = nn.Sequential(
    nn.Flatten(),
    nn.Linear(32 * 7 * 7, 10),
)

Flattening is only one possible way to consume the final feature maps of a convolutional feature extractor. Other architectures may preserve or aggregate their spatial structure instead.