Skip to content
Tutorials Center
My learning

Topics

Lesson 31 of 37 · State Space

LQR Optimal Control

Design the linear-quadratic regulator: pick Q and R, solve the algebraic Riccati equation with SciPy and from scratch, and see how weights trade accuracy for control effort.

Pole placement asks you to choose pole locations, but nobody knows what the right poles are for a six-state aircraft. The linear-quadratic regulator (LQR) replaces that guess with a question about cost: how much do you care about state errors compared to control effort? You write that down as a number and let mathematics find the best feedback gain.

For x' = Ax + Bu LQR minimizes

TEXT
J = ∫0^∞ ( xᵀ Q x + uᵀ R u ) dt

with Q ≥ 0 weighting state deviations and R > 0 weighting control effort. The optimal control is plain state feedback, u = -K x, with

TEXT
K = R⁻¹ Bᵀ P        where P solves the algebraic Riccati equation (ARE):
AᵀP + PA - P B R⁻¹ Bᵀ P + Q = 0

The cost of starting at x₀ and running the optimal controller forever is x₀ᵀ P x₀. The solution comes with a bonus: LQR loops have guaranteed robustness: at least 60° phase margin and an infinite upward gain margin (for full-state feedback), something pole placement does not give.

Solving the Riccati equation

scipy.linalg.solve_continuous_are solves it. To see that there is no magic, we also solve it from scratch: build the Hamiltonian matrix

TEXT
H = [[ A,  -B R⁻¹ Bᵀ ],
     [-Q,  -Aᵀ       ]]

whose eigenvalues come in ± pairs. The n eigenvectors for the stable eigenvalues (negative real part) span a subspace [X₁; X₂], and P = X₂ X₁⁻¹.

Python Riccati by SciPy and by the Hamiltonian

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Both give K = [0.916, 0.614], with closed-loop poles -1.31 ± 2.05j. You never typed pole locations; they came out of the weights.

flowchart LR
    A["Choose Q, R"] --> B["Solve ARE for P"]
    B --> C["K = R^-1 B^T P"]
    C --> D["Simulate, check effort and speed"]
    D -->|"too slow / too violent"| A

LQR design is an iteration on the weights, not on the poles.

Choosing Q and R

The weights are the design interface:

  • Start with Bryson's rule: Qᵢᵢ = 1/(max acceptable xᵢ)², Rⱼⱼ = 1/(max acceptable uⱼ)². It normalizes each term to "how many times over the limit".
  • Increasing Q relative to R makes the controller more aggressive: poles move left and |u| grows.
  • Increasing R makes the controller more economical and sluggish.
  • Only the ratio matters. Multiplying both Q and R by the same constant leaves K unchanged.
  • Q is often diagonal. Penalizing a state that you care about (position error) raises its weight; zero weight on a state means "I don't care about it directly".
Python How the weights change K, poles and cost

Press Run (or Ctrl + Enter) to see the result here.

Program input · what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

Read down the rows:

  • q = 1, r = 10 (cheap errors, expensive control): K ≈ [0.01, 0.03], practically no feedback, poles near the open-loop -1 ± 2j.
  • q = 10, r = 1: moderate action, poles at -1.31 ± 2.05j.
  • q = 100, r = 1: stronger action, -2.08 ± 2.62j.
  • q = 100, r = 0.01 (control is nearly free): K ≈ [95, 15], much faster poles at -8.58 ± 5.15j.

The last two columns agree to within 0.3%: the simulated cost from integrating the dynamics matches x₀ᵀ P x₀, confirming the meaning of P as the cost-to-go. The small difference is the Euler integration error.

Extensions

  • Tracking and integral action. Add the integral of the tracking error as an extra state and apply LQR to the augmented system. You get an optimal "PI" in state-space form.
  • Output weighting. Penalizing yᵀ Qy y instead of xᵀ Q x is equivalent to Q = Cᵀ Qy C.
  • Discrete LQR. solve_discrete_are handles sampled systems (see digital control).
  • LQG. Combine LQR with a Kalman filter. By separation, design them independently. As noted in the observers lesson, the combined loop may lose the nice margins of LQR; recheck them (the LTR technique recovers them).
  • Nonlinear systems. Linearize about an operating point and use the LQR gain locally (the pendulum project later does this).

If solve_continuous_are complains, check that (A, B) is stabilizable, that Q is positive semi-definite and (A, √Q) is detectable, and that R is positive definite. Violating any of them makes the Riccati equation unsolvable or the solution non-stabilizing.

Check your understanding

Quick check · 5 questions

of 5 answered You scored / 5 · perfect!

  1. 1. What does LQR minimize?

    J = integral of (x'Qx + u'Ru) dt, a weighted sum of state error and control effort.
  2. 2. What is the optimal control law?

    K = R^-1 B' P with P from the Riccati equation, applied as u = -Kx.
  3. 3. You raise R a lot while keeping Q fixed. What happens?

    A large R penalizes control effort, so the optimal gains shrink and the poles move toward the open-loop poles.
  4. 4. What is x0' P x0?

    P is the cost-to-go matrix: the optimal infinite-horizon cost from x0 equals x0' P x0.
  5. 5. Which of the following is a typical guarantee of full-state LQR?

    Continuous-time LQR with full-state feedback has at least 60 degrees of phase margin and an infinite upward gain margin.

Summary

  • LQR minimizes ∫ (xᵀQx + uᵀRu) dt; the answer is u = -Kx with K = R⁻¹BᵀP.
  • P solves the algebraic Riccati equation (SciPy, or the stable subspace of the Hamiltonian).
  • Q versus R sets the speed-versus-effort trade-off; start with Bryson's rule.
  • x₀ᵀPx₀ is the optimal cost from x₀.
  • LQR plus a Kalman filter gives LQG; always recheck margins of the whole loop.

Next: Sampling and the Z-Transform