How to Work on Projects#
The projects ask you to apply deep learning to larger computer-vision problems. You will be required to decompose a problem into components, make design choices, evaluate their consequences, and decide how to proceed based on experimental results. The objective is to conduct an applied investigation.
Decomposing a Problem#
Deep-learning systems contain many interacting components:
data preparation,
model architecture,
loss functions,
training,
evaluation,
inference.
Start by reducing the problem to the simplest version that can be made to work. Break the task into components and determine which parts already have suitable solutions. Many elements can often be reused from the tutorials, standard PyTorch components, pretrained models, or existing implementations. Your effort should focus on the parts that are specific to the project.
Important
Build the system incrementally. Verify individual components when possible before combining them, and introduce additional complexity only after the simpler version behaves as expected.
Baselines and Experiments#
Before trying to obtain the best possible result, establish a baseline: a complete system whose performance can be measured. A baseline might use a small model, a straightforward loss function, a pretrained network without fine-tuning, or another simple approach appropriate to the task. Its purpose is not necessarily to perform well. It provides a reference against which later changes can be evaluated.
Once a baseline is available, treat improvements as experiments rather than as a sequence of arbitrary modifications. Each experiment should address a reasonably precise question. Whenever possible, change one important aspect of the system at a time and keep the evaluation procedure fixed.
A useful experimental cycle is:
Observe the behaviour of the current system.
Identify a limitation or question.
Formulate a modification intended to address it.
Evaluate the modified system under comparable conditions.
Interpret the result before deciding what to try next.
Note
Not every experiment needs to improve performance. A negative result can still be informative if it rules out a plausible approach or reveals something about the problem.
Evaluation and Error Analysis#
Evaluation is part of model development, not only something performed after training is complete. Measurements provide evidence about whether a change helped, while qualitative analysis helps explain why. Use metrics appropriate to the task and compare them across experiments. A single aggregate score, however, rarely gives a complete picture of model behaviour. Inspect predictions as well.
Look for recurring patterns in the failures. A classifier may confuse visually similar categories. A localization model may work for large objects but fail for small ones. A retrieval model may emphasize colour while ignoring semantic similarity. A counting model may systematically underestimate dense scenes. These observations can suggest what to investigate next and reveal limitations not visible in aggregate metrics.
Documenting Your Work#
As you work through a project, document the important stages of the investigation. For important experiments, record the relevant model and training configuration, the evaluation procedure, the results, and the interpretation of those results. Use tables, plots, metrics, and representative predictions to support your interpretation. When an experiment motivates a subsequent change, make that connection explicit.
If you use notebooks to run experiments, keep them focused on the investigation. Avoid cluttering them with implementation details. Reusable components such as datasets, models, losses, training utilities, evaluation functions, and visualization routines should generally be placed in Python modules. The notebooks should then import those modules and focus on the experimental workflow.
Reproducibility#
A project should contain enough information to reproduce the experiments that support its conclusions. Record the experimental procedure, including the data, model, loss, training, and evaluation configurations.
Avoid relying on undocumented intermediate steps. Code should run in a well-defined order, dependencies between files should be clear, and reported results should correspond to the submitted implementation.
Exact numerical results may vary because of stochastic training procedures. Reproducibility therefore does not require identical numbers, but it does require a clear and repeatable experimental procedure.
By the end of a project, you should be able to explain which approaches worked, which did not, and what are the remaining limitations. You should be able to justify your design choices and your conclusions.