Extremum-seeking control

Extremum-seeking control (ESC) is an online optimization strategy whereby a controller seeks the extremum of a measureable cost function. Consider a system on the form

\[\begin{aligned} \dot x &= f(x) + g(x)u \\ y &= h(x) \end{aligned}\]

where $y = h(x)$ is the output of an objective function to be minimized. In a practical application, $h$ may be unknown, as long as $y$ is measurable. The extremum-seeking controller manipulates the input $u$ in order to find $x^*$ such that $h(x^*)$ is optimized.

Example: Online optimization of a cost function

In this example we will optimize the function $h$ in an online fashion.

using Plots
h(u) = -3u - 2u^2 + u^3 + u^4
plot(h, -2, 2)

We do this by constructing a perturbing extremum-seeking controller ESC and indicate that the cost function input of the controller, $y$, is related to the controller output, $u$, as $y = h(u)$:

using JuliaSimControls, ModelingToolkit, OrdinaryDiffEq
@named escont = ESC(k=0.8, a=0.1, w=3, wh=1)

connections = [
    escont.y ~ h(escont.u)
]

@named closed_loop = ODESystem(connections, systems=[escont])
sys = structural_simplify(closed_loop)

x0 = []
prob = ODEProblem(sys, x0, (0, 300.0))
sol = solve(prob, Tsit5())
plot(sol, vars=[escont.u, escont.uh, escont.y])

As we can see above, the ESC explores the input space by means of a sinusoidal dither signal, and finds the global optimum of $h(1) = -3$. Had we started in $u=-1.5$ instead of $u=0$, we would have gotten stuck in the local minimum $h(-1)=1$ for much longer:

x0 = [
    escont.uh => -1.5
]
prob = ODEProblem(sys, x0, (0, 300.0))
sol = solve(prob, Tsit5())
plot(sol, vars=[escont.u, escont.uh, escont.y])

Increasing the dither amplitude solves this problem:

x0 = [
    escont.uh => -1.5
    escont.a => 0.3
    escont.b => 0.3
]
prob = ODEProblem(sys, x0, (0, 300.0))
sol = solve(prob, Tsit5())
plot(sol, vars=[escont.u, escont.uh, escont.y])

In general, increasing the dither amplitude increases the region of convergence and the convergence speed.

Example: Dynamical extremum seeking

In this example, we will use an extremum-seeking controller to control a nonlinear dynamical system with a cost function output. The system to be controlled is given by

\[\begin{aligned} \dot{x}_1(t) &= x_1(t)^2 + x_2(t) + u(t) \\ \dot{x}_1(t) &= x_1(t)^2 - x_2(t)\\ y(t) &= -1 + x_1(t)^2 - x_1(t) \end{aligned}\]

and we will use the controller PIESC. This controller offers potentially faster convergence compared to the ESC controller, at the expense of slightly harder tuning.

using JuliaSimControls: t, D

@named esc = PIESC(k=10, tau=0.1, a=10, w=100, wh=1000)

function systemmodel(; name=:systemmodel)
    @variables x1(t)=0 x2(t)=0 u(t)=0 y(t)=0
    eqs = [
        D(x1) ~ x1^2 + x2 + u
        D(x2) ~ x1^2 - x2
        y ~ -1 + x1^2 - x1
    ]
    ODESystem(eqs, t; name)
end

model = systemmodel()

connections = [
    model.y ~ esc.y
    esc.u ~ model.u
]

@named closed_loop = ODESystem(connections, t, systems=[model, esc])
sys = structural_simplify(closed_loop)

x0 = [
    # model.x1 => 0.5
    # model.x2 => 0.25
    # esc.uh => -0.5
    esc.v => 0
]

prob = ODEProblem(sys, x0, (0, 6.0))
sol = solve(prob, Tsit5())
plot(sol, vars=[model.x1, model.x2, model.y, esc.u], layout=4)