LQG Analysis

The LQG analysis is used to design optimal controllers that minimize a quadratic cost function while accounting for process and measurement noise. The analysis combines optimal state feedback (LQR) with optimal state estimation (steady-state Kalman filter) using the separation principle, providing a systematic approach to controller design under uncertainty.

LQG Theory and Matrix Interpretation

Cost Function

The LQG controller minimizes the infinite-horizon quadratic cost:

$$J=\mathbb{E}[{\int }_0^{\infty }{z}^T{Q}_{1}z+u^T{Q}_{2}udt]$$

Noise Model

The system is subject to process and measurement noise:

$$\begin{array}{rl}\dot{x}& =Ax+Bu+G{w}_1\\ z& =C_{z}x\\ y& =C_{y}x+w_2\end{array}$$

where w₁ ~ N(0, R₁) is process noise and w₂ ~ N(0, R₂) is measurement noise. In this implementation, $G=B$ for simplicity.

Matrix Meanings

• Q₁ (controlled output penalty): Higher values → tighter tracking of controlled outputs, more aggressive control

• Q₂ (control input penalty): Higher values → smoother control signals, reduced actuator effort

• R₁ (disturbance noise): Models uncertainty in disturbance inputs (w=u), affects Kalman filter design

• R₂ (measurement noise): Models sensor uncertainty, affects how much the estimator trusts measurements

Separation Principle

The optimal LQG controller separates into:

1. LQR design: Compute optimal state feedback gain L assuming perfect state knowledge

2. Kalman filter: Compute optimal state estimator gain K assuming the control law is given

3. Combine: Use u = -L*x̂ where x̂ is the Kalman filter estimate

Example Definition

The following example demonstrates LQG controller design for a DC motor system with load control:

analysis LQGControllerAnalysis
  extends DyadControlSystems.LQGAnalysis(
    measurement = ["y"],
    controlled_output = ["y"],
    control_input = ["u"],
    loop_openings = ["r"],
    q1_diag = [1000.0],
    q2_diag = [1.0],
    r1_diag = [1.0],
    r2_diag = [0.1],
    duration = 1.0
  )

  model = DyadExampleComponents.TestDCMotorLoadControlled()

end

using DyadExampleComponents, DyadInterface
solution = LQGControllerAnalysis()

┌ Warning: Initialization system is overdetermined. 2 equations for 0 unknowns. Initialization will default to using least squares. `SCCNonlinearProblem` can only be used for initialization of fully determined systems and hence will not be used here. To suppress this warning pass warn_initialize_determined = false. To make this warning into an error, pass fully_determined = true
└ @ ModelingToolkit ~/.julia/packages/ModelingToolkit/Z9mEq/src/systems/diffeqs/abstractodesystem.jl:1522

using Plots
fig_step = artifacts(solution, :StepResponse)

fig_gang = artifacts(solution, :GangOfFour)

fig_bode = artifacts(solution, :BodePlot)

fig_pz = artifacts(solution, :PZMaps)

We can examine the optimal controller and observer gains:

L = artifacts(solution, :ControllerGain)
K = artifacts(solution, :ObserverGain)
L, K

(2×1 DataFrame
 Row │ u1       
     │ Float64  
─────┼──────────
   1 │ 31.0731
   2 │  2.19099, 1×2 DataFrame
 Row │ x1       x2      
     │ Float64  Float64 
─────┼──────────────────
   1 │ 94.2549  179.565)

Analysis Arguments

The following arguments define an LQG analysis:

Required Arguments

• model: The system model to be controlled.

• measurement::Vector{String}: Names of measured outputs ($y$) available for state estimation.

• controlled_output::Vector{String}: Names of controlled outputs ($z$) used for performance evaluation.

• control_input::Vector{String}: Names of control inputs ($u$) that affect the plant.

Penalty Matrix Parameters

• q1_diag::Vector{Real}: Diagonal elements of Q₁ matrix penalizing controlled outputs ($z$). Length must match number of controlled outputs.

• q2_diag::Vector{Real}: Diagonal elements of Q₂ matrix penalizing control inputs ($u$). Length must match number of control inputs.

• r1_diag::Vector{Real}: Diagonal elements of R₁ matrix representing disturbance noise covariance ($w$). Length must match number of control inputs.

• r2_diag::Vector{Real}: Diagonal elements of R₂ matrix representing measurement noise covariance ($y$). Length must match number of measured outputs.

Advanced Parameters

• qQ::Real = 0.0: Loop-transfer recovery parameter for output direction. Higher values recover more robustness in the output direction.

• qR::Real = 0.0: Loop-transfer recovery parameter for observer dynamics. Higher values make the observer more aggressive.

System Parameters

• loop_openings::Vector{String} = []: Names of loop openings to break feedback loops during linearization.

• t::Real = 0.0: The time at which to perform the linearization of the system.

Discretization Parameters

• disc::String = "cont": Discretization method. Use "cont" for continuous-time, or "zoh", "tustin", "foh" for discrete-time.

• Ts::Real = -1.0: Sampling time for discretization. Required when disc != "cont". Set to -1 for automatic selection.

Integral Action Parameters

• integrator_indices::Vector{Int} = []: Indices of control inputs (matching control_input) that are augmented with integrating disturbance models for integral action.

• integrator_r1_diag::Vector{Real} = []: Diagonal elements of R₁ matrix for integrator disturbance inputs. Length must match integrator_indices.

Analysis Parameters

• wl::Real = -1: The lower frequency bound for analysis plots. Set to -1 for automatic selection.

• wu::Real = -1: The upper frequency bound for analysis plots. Set to -1 for automatic selection.

• num_frequencies::Int = 3000: The number of frequencies to be used in frequency-domain analysis.

• duration::Real = -1.0: The duration of the step-response analysis. Set to -1 for automatic selection.

LQG Algorithm

The controller design follows this systematic approach:

1. Linearization: The plant is linearized at the specified operating point with loop openings

2. Plant Augmentation: If integrator_indices is specified, the plant is augmented with integrating disturbances using add_low_frequency_disturbance to provide integral action

3. Output Partitioning: Measured outputs ($y$) and controlled outputs ($z$) are separated using explicit vector specification

4. ExtendedStateSpace Creation: The system is converted to the $w,u\to z,y$ format required for robust control design

5. Matrix Construction: Penalty matrices $Q₁,Q₂$ and noise covariance matrices $R₁,R₂$ are built from diagonal specifications

6. LQG Problem Formulation: The optimization problem is set up with optional loop-transfer recovery parameters

7. Optimal Gains Computation: State feedback gain $L$ and Kalman filter gain $K$ are computed optimally

8. Controller Synthesis: Multiple controller forms are generated (observer_controller, ff_controller, extended_controller)

9. Discretization: If discrete-time control is specified, the system is discretized before the design, resulting in a discrete-time controller

Design Considerations

Penalty Matrix Tuning

Q₁ Matrix (Controlled Output Penalty):

• Higher values: Tighter tracking, more aggressive control

• Lower values: More relaxed performance, smoother control

• Relative weighting: Balance between different controlled outputs

Q₂ Matrix (Control Input Penalty):

• Higher values: Reduced actuator effort, smoother signals

• Lower values: More aggressive control, faster response

• Physical limits: Consider actuator saturation constraints

R₁ Matrix (Disturbance Noise):

• Higher values: Less trust in model, more conservative estimation

• Lower values: More trust in model, more aggressive estimation

• Reality matching: Should reflect actual disturbance input uncertainties

R₂ Matrix (Measurement Noise):

• Higher values: Less trust in sensors, rely more on model prediction

• Lower values: More trust in sensors, aggressive state correction

• Sensor characteristics: Should match actual sensor noise levels

Loop-Transfer Recovery

qQ Parameter (Output Direction):

• qQ = 0: Standard LQG design

• qQ > 0: Recovers robustness in the output direction

• Higher values: More robust but may sacrifice performance

qR Parameter (Observer Dynamics):

• qR = 0: Standard Kalman filter

• qR > 0: Makes observer more aggressive

• Higher values: Faster state estimation but more noise sensitivity

Tutorial Video

Artifacts

An LQGAnalysis returns the following artifacts:

Standard Plots

• :StepResponse: Step response showing reference tracking, disturbance rejection, and control input response.

• :GangOfFour: Gang of four transfer functions (S, PS, CS, T) showing closed-loop sensitivity properties.

• :BodePlot: Bode plot of the open-loop transfer function showing gain and phase characteristics.

• :MarginPlot: Gain and phase margins indicating stability robustness.

• :PZMaps: Pole-zero maps of the open-loop and closed-loop systems.

Tables

• :ControllerGain: DataFrame of the optimal state feedback gain matrix L.

• :ObserverGain: DataFrame of the optimal Kalman filter gain matrix K.

Notes

• The system must be controllable and observable for LQG design to work. Use loop_opneings to disconnect parts of the model that are not controllable or observable given the specified inputs and outputs.

Further Reading

• LQG Control (Wikipedia)

• Kalman Filter (Wikipedia)

• Separation Principle (Wikipedia)

• RobustAndOptimalControl.jl Documentation

• ControlSystems.jl Documentation

• Example: Disturbance modeling and rejection with LQG controllers