6. Solving optimization problems

The basic idea of an optimization problem is that you want to maximize or minimize a particular quantity. However, the situation is usually described with words, instead of using mathematical formulas. Before you can proceed with optimization, you need to identify the variables of your problem, with which to formulate the function that you intend to maximize or minimize. This falls under the category of unconstrained optimization, whose problems can be solved by gradient descend. Moreover, in most (if not all) real-world scenarios, you also have some auxiliary conditions that must be satisfied by the solution to your problem. This falls under the category of constrained optimization, whose problems can be solved by projected gradient descend. At this point in the course, you should have a good understanding of how to deal with both categories.

Contents: This chapter presents a step-by-step guide for solving various kinds of optimization problems arising from real-world scenarios.

• You will understant how to convert the description of a problem into a mathematical formulation involving any number of variables, a cost function, and possibly some constraints (if any).

• You will learn how to remove simple equality constraints from your optimization problems.

• You will practice solving optimization problems using both analytical and numerical methods.