Skip to content
Tutorials Center
My learning

Topics

Lesson 1 of 38 · Getting Started

What Is Optimization?

Meet optimization: choosing the best option under rules. See real examples, write your first objective function and find its minimum with a few lines of code.

Every day you pick the "best" option under limits. You choose the fastest route that avoids tolls. You pack a bag with the most useful items that still weigh under 10 kg. A company picks prices that earn the most profit. All of these are optimization problems: find the choice that makes some number as small or as large as possible, while following rules.

Optimization is the engine behind machine learning, route planning, power grids, finance and engineering design. This track focuses on one special and very friendly family, convex optimization, where the problem has no tricky traps and good algorithms come with guarantees. In this first lesson you will see what an optimization problem looks like and solve a few with code.

Choices, a score and rules

Every optimization problem has three ingredients.

  1. Decision variables. The things you control. Examples: the price of a product, the width of a fence, the weights of a model.
  2. An objective function. One number that scores a choice. You want it as small as possible (a minimization) or as large as possible (a maximization). Costs, errors and distances are minimized. Profit and accuracy are maximized.
  3. Constraints. Rules the choice must obey, such as "the fence is at most 40 m long" or "the weights must be non-negative".

We write a problem like this:

Text
minimize    f(x)
subject to  g(x) ≤ 0

Read it as: "among all x that satisfy the rule g(x) ≤ 0, find the one that makes f(x) smallest."

Maximizing f is the same as minimizing -f. Because of this, textbooks (and this track) usually talk only about minimization.

A first example: the cheapest box

You want an open-top box with a square base of side s (in meters) and height h. It must hold 2 m³, so s² · h = 2. The material costs the same everywhere, so the cost is proportional to the surface area: base s² plus four sides 4·s·h.

Use the volume rule to remove h: h = 2 / s². The area becomes A(s) = s² + 8 / s. Now there is one variable and no constraint left. Let us scan many values of s and look for the smallest area.

Python Scanning for the cheapest box

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

The scan is slow and approximate, but it works for any one-variable problem. Later lessons replace the scan with smarter methods that need far fewer evaluations.

Seeing the landscape

A picture makes the idea concrete. The objective is a curve, and the best choice is the lowest point on it.

Python Plot the objective

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Notice the shape: it falls, reaches one lowest point, then rises. There is only one valley. That single-valley property is exactly what "convex" will mean in the next chapters.

Maximizing is minimizing the negative

Suppose a farmer has 40 m of fence and wants a rectangular pen with the most area next to a long wall (no fence needed along the wall). With width w perpendicular to the wall, the length along the wall is 40 - 2w, and the area is w(40 - 2w). To use a minimizer, minimize the negative area.

Python A maximization turned into a minimization

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

You can check the answer by hand: the area 40w - 2w² is a downward parabola with its top at w = 10, so the pen is 10 m by 20 m with an area of 200 m².

Kinds of optimization problems

Kind Variables Example
Unconstrained any real numbers fit a line to data
Constrained must satisfy rules portfolio with weights summing to 1
Discrete (integer) whole numbers schedule 7 delivery trucks
Convex any, but a special shape least squares, linear programs

Discrete problems are often very hard. Convex problems are the "nice" ones: any local minimum is a global minimum, and efficient solvers exist. Many real tasks can be written, or approximated, as convex problems, which is why this track exists.

flowchart LR
    A["Real-world goal"] --> B["Choose variables"]
    B --> C["Write objective f(x)"]
    C --> D["Add constraints"]
    D --> E["Solve with an algorithm"]

The usual path from a real question to a solved optimization problem.

Check your understanding

Quick check · 4 questions

of 4 answered You scored / 4 · perfect!

  1. 1. Which of these is NOT one of the three main ingredients of an optimization problem?

    A problem needs variables to choose, an objective to score choices and (usually) constraints. A random seed is not part of the problem itself.
  2. 2. How can you turn 'maximize profit P(x)' into a minimization?

    The point that makes -P as small as possible makes P as large as possible.
  3. 3. In the box example, the constraint s² · h = 2 was used to

    Solving for h = 2/s² and substituting leaves the objective A(s) = s² + 8/s in one variable.
  4. 4. Why is convex optimization considered friendly?

    Convexity rules out false valleys, so a method that finds any local minimum has found the best one.

Summary

  • An optimization problem has decision variables, an objective function and (often) constraints.
  • Maximizing f equals minimizing -f, so we study minimization.
  • A scan over a grid is the simplest solver; it works but is slow.
  • Convex problems have a single "valley" and come with strong guarantees.

Next: Anatomy of an Optimization Problem