Skip to main content
Back to top
Ctrl
+
K
Search
Ctrl
+
K
Chapters
1. Array programming
1.1. Arrays
1.2. Operations
1.3. Indexing
1.4. Linear algebra
1.5. Exercises (lab01)
1.6. Self-assessment test
2. Black-box optimization
2.1. Optimization problem
2.2. Objective function
2.3. Zero-order optimality
2.4. Global optimization
2.5. Local optimization
2.6. Stochastic optimization
2.7. Exercises (lab02)
2.8. Self-assessment test
3. Building a tetris AI
3.7. Assignment (lab03)
4. Differentiable optimization
4.1. Differentiable functions
4.2. Automatic differentiation
4.3. First-order optimality
4.4. Gradient descent
4.5. Choice of step-size
4.6. Examples
4.6.1. Example 1
4.6.2. Example 2
4.6.3. Example 3
4.6.4. Example 4
4.6.5. Example 5
4.7. Exercises (lab04)
4.8. Self-assessment test
5. Constrained optimization
5.1. Constraints
5.2. Normal cone
5.3. Geometric optimality
5.4. Orthogonal projection
5.4.1. Box
5.4.2. Affine space
5.4.3. Simplex
5.4.4. Half-space
5.4.5. L1-ball
5.4.6. Positive L1-ball
5.4.7. Convex polyhedron
5.4.8. L2-ball
5.4.9. Positive L2-ball
5.4.10. Hypersphere
5.5. Projected gradient descent
5.6. Alternating gradient descent
5.7. Examples
5.7.1. Example 6
5.7.2. Example 7
5.7.3. Example 8
5.7.4. Example 9
5.8. Exercises (lab05)
5.9. Self-assessment test
6. Solving optimization problems
6.1. Step-by-step guide
6.2. Assignment (lab06)
Appendix
Choice of step-size
Constant step-size
Maximum curvature
Variable step-size
Line search
Backtracking
Convergence analysis
Example 1
Exercises (lab07)
Training a neural network
Assignment (lab08)
.ipynb
.pdf
Backtracking
Backtracking
#
