Residual Connections: Deep Network Training

Hidden layers

Activation

Batch normalization

Batch size

Optimizer

Learning rate

Training updates

Ready

step 0

Plain Network

Residual Network

MSE loss

—

—

Function learned by each network

Target function Plain network Residual network

Training loss