Sketch Image Retrieval#
Image retrieval is the process of finding images relevant to a given query. Traditionally, this is done using metadata such as tags or descriptions. A more advanced approach is to retrieve images based on their visual content, allowing searches to be driven by characteristics like color, texture, and shape.
In this project, you will develop a content-based image retrieval system with PyTorch. You will focus on the case where the query is a hand-drawn sketch, and the goal is to find photos that match the category of the object depicted in the sketch. You will build a neural network that learns a similarity metric between sketches and photos, allowing the system to retrieve images from a photo gallery based on how well they visually correspond to the query sketch.
Difficulty |
Suggested Tutorials |
|---|---|
Moderate |
Convolutional Networks, Feature Extraction, Weak Supervision |
Grading
The project will be graded based on the following criteria. Points for each activity are awarded based on quality and completeness (partial credit possible).
Activity |
Points (max) |
|---|---|
Build a baseline image retrieval system |
9 |
Properly evaluate its performance (precision, recall) |
3 |
Transfer learning (fine-tuning) |
3 |
Deployment (model made usable) |
2 |
Presentation (clarity & demo) |
3 |
Total |
0-20 |