# Convolutional Networks

Use this page to quickly reference the tensor shapes, convolution and downsampling operations, parameter counts, receptive fields, and shape calculations needed to construct convolutional networks.

## Image Tensors

PyTorch image models normally use the channels-first convention.

| Object | Shape |
| --- | --- |
| Grayscale image | `(1, H, W)` |
| RGB image | `(3, H, W)` |
| Image with `C` channels | `(C, H, W)` |
| Image batch | `(N, C, H, W)` |

*Legend: `N` = batch size, `C` = channels, `H` = height, `W` = width.*



## Convolution

### `Conv2d`

Create a two-dimensional convolutional layer with `Conv2d`.

```python
conv = torch.nn.Conv2d(in_channels=C_in, out_channels=C_out, kernel_size=K)
```

The layer transforms an input tensor of shape `(N, C_in, H, W)` to an output tensor of shape `(N, C_out, H_out, W_out)`. Each output channel corresponds to one learned filter applied across all input channels.

:::{warning}
The `in_channels` argument of a convolution must match the channel dimension of its input tensor.
:::

The most frequently used arguments are the following.

| Argument       | Meaning | Default |
| -------------- | --- | --- |
| `in_channels`  | Number of input channels | *Mandatory* |
| `out_channels` | Number of learned filters and output channels | *Mandatory* |
| `kernel_size`  | Spatial size of the filters (int or tuple) | *Mandatory* |
| `stride`       | Distance between evaluated spatial positions | 1 |
| `padding`      | Values added around the input boundary | 0 |
| `bias`         | Whether the layer learns one bias per output channel | `True` |


### Learnable Parameters

For a `Conv2d` layer with `C_in` input channels, `C_out` output channels, and a kernel of height `K_h` and width `K_w`, the weight tensor and bias tensor have shapes:

```text
weight: (C_out, C_in, K_h, K_w)
bias:   (C_out,)
```

With bias enabled, the total number of learnable parameters is therefore

$$
\textsf{Parameter Count} = \underbrace{C_{\text{out}} C_{\text{in}} K_h K_w}_{\text{weights}} + \underbrace{C_{\text{out}}}_{\text{biases}}
$$

The parameter tensors can be accessed directly with:

```python
conv = torch.nn.Conv2d(C_in, C_out, (K_h, K_w))

conv.weight
conv.bias
```

### Spatial Dimensions

The output size $O$ of a convolutional layer can be calculated from the input size $I$, kernel size $K$, padding $P$, and stride $S$. For one spatial dimension, the formula is:

$$
O = \left\lfloor\frac{I + 2P - K}{S} + 1\right\rfloor.
$$

Notable special cases for odd kernel sizes.
- If `stride=1` and `padding=(kernel_size-1)/2`, the output size equals the input size.
- If `stride=S` and `padding=(kernel_size-1)/2`, the output size is $\lceil I/S \rceil$.

The formula assumes no dilation. The official PyTorch documentation includes dilation and other edge cases.


## Downsampling

### `MaxPool2d`

Max pooling reduces spatial dimensions without changing the number of channels.

```python
pool = torch.nn.MaxPool2d(kernel_size=K)
```

Pooling layers do not have learnable parameters. They perform a fixed local aggregation of the input. For example, `MaxPool2d` outputs the maximum value in each local region of the input. Other pooling operations, such as average pooling, compute different local statistics.

The most frequently used arguments are the following.

| Argument       | Meaning | Default |
| -------------- | ------- | ------- |
| `kernel_size`  | Spatial size of the local region where pooling is applied | *Mandatory* |
| `stride`       | Distance between evaluated spatial positions | `kernel_size` |
| `padding`      | Values added around the input boundary | 0 |

The output size $O$ of a pooling layer can be calculated from the input size $I$, kernel size $K$, padding $P$, and stride $S$. For one spatial dimension, the formula is the same as convolution:

$$
O = \left\lfloor\frac{I + 2P - K}{S} + 1\right\rfloor.
$$

With default values for stride and padding, the output size is $\lfloor I/K \rfloor$.


### Strided Convolution

A convolution can also perform downsampling by using `stride > 1`. Both pooling and strided convolution reduce spatial resolution, but they perform different operations.

- `MaxPool2d(...)` performs a fixed local aggregation of the input.
- `Conv2d(..., stride = ...)` performs a learned transformation and downsampling of the input.


## Convolutional Networks

A convolutional network combines convolutions, nonlinearities, and downsampling operations into a sequence of layers. Intermediate representations preserve the spatial organization of the input, while the number of channels and spatial resolution are progressively transformed as data moves through the layers.

### Feature Extractor

A sequence of convolutional operations can be grouped into a feature extractor:

```python
features = nn.Sequential(
    nn.Conv2d(1, 16, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2),

    nn.Conv2d(16, 32, kernel_size=3, padding=1),
    nn.ReLU(),
    nn.MaxPool2d(2),
)
```

Each layer must be designed to receive a tensor compatible with the output of the preceding layer. In this example, the first convolution produces 16 channels. ReLU and max pooling preserve the channel count, so the second convolution is set to receive 16 input channels.

The feature extractor transforms a batch of grayscale images from shape `(N, 1, H, W)` to a spatial representation with shape `(N, 32, H//4, W//4)`. What happens to this representation afterward depends on the task. A downstream component may preserve the spatial dimensions, aggregate them, flatten them, or otherwise transform the representation.

### Shape Tracing

It is important to trace the tensor shape through each operation when constructing a convolutional network. This verifies that consecutive layers are compatible and makes changes in spatial resolution explicit.

For the feature extractor above, a batch of `28 × 28` grayscale images is transformed as follows.

| Layer                          | Output shape      |
| ------------------------------ | ----------------- |
| Input                          | `(N,  1, 28, 28)` |
| `Conv2d(1, 16, 3, padding=1)`  | `(N, 16, 28, 28)` |
| `ReLU`                         | `(N, 16, 28, 28)` |
| `MaxPool2d(2)`                 | `(N, 16, 14, 14)` |
| `Conv2d(16, 32, 3, padding=1)` | `(N, 32, 14, 14)` |
| `ReLU`                         | `(N, 32, 14, 14)` |
| `MaxPool2d(2)`                 | `(N, 32,  7,  7)` |

Convolution may change both the channel count and the spatial dimensions. Element-wise activations such as ReLU preserve the complete shape. Pooling preserves the channel count while reducing the spatial dimensions.

### Receptive Field

As convolution and pooling layers are composed, each activation can depend on an increasingly large region of the original input. The *receptive field* of an activation is the region of the original input that can influence it. The *effective stride* measures the spacing between neighbouring activations relative to the original input.

$$
\begin{aligned}
r_\ell &= \textsf{size of the receptive field after layer $\ell$}\\
j_\ell &= \textsf{effective stride after layer $\ell$}
\end{aligned}
$$

For a layer with kernel size $k_\ell$ and stride $s_\ell$, the receptive field and effective stride can be calculated recursively from the previous layer, with $r_0 = j_0 = 1$ at the input layer.

$$
\begin{aligned}
r_\ell &= r_{\ell-1} + (k_\ell - 1) j_{\ell-1}\\
j_\ell &= j_{\ell-1} s_\ell
\end{aligned}
$$

For example, consider the following sequence of convolution and pooling layers.

| Stage | Kernel | Stride | Receptive field | Effective stride | Spatial Dimensions | 
| ----- | ------ | ------ | --------------- | ---------------- | ------------------ |
| Input | —      | —      | 1               | 1                | $H \times W$       | 
| Conv  | 3      | 1      | 3               | 1                | $H \times W$       |
| Pool  | 2      | 2      | 4               | 2                | $H/2 \times W/2$   |
| Conv  | 3      | 1      | 8               | 2                | $H/2 \times W/2$   |
| Pool  | 2      | 2      | 10              | 4                | $H/4 \times W/4$   |

### Flattening

If the final spatial representation must be converted into one vector per example, flattening from the first non-batch dimension transforms a tensor of shape `(N, C, H, W)` into a tensor of shape `(N, C * H * W)`. This is commonly done before connecting to a standard linear layer. 

For example, the final feature maps with shape `(N, 32, 7, 7)` can be flattened to `(N, 1568)` before connecting to a linear layer. If the task is classification, the convolutional feature extractor can be followed by a classification head that maps the flattened features to class scores.

```python
classifier = nn.Sequential(
    nn.Flatten(),
    nn.Linear(32 * 7 * 7, 10),
)
```

Flattening is only one possible way to consume the final feature maps of a convolutional feature extractor. Other architectures may preserve or aggregate their spatial structure instead.