Extremum-seeking control
Extremum-seeking control (ESC) is an online optimization strategy whereby a controller seeks the extremum of a measureable cost function. Consider a system on the form
\[\begin{aligned} \dot x &= f(x) + g(x)u \\ y &= h(x) \end{aligned}\]
where $y = h(x)$ is the output of an objective function to be minimized. In a practical application, $h$ may be unknown, as long as $y$ is measurable. The extremum-seeking controller manipulates the input $u$ in order to find $x^*$ such that $h(x^*)$ is optimized.
Example: Online optimization of a cost function
In this example we will optimize the function $h$ in an online fashion.
using Plots
h(u) = -3u - 2u^2 + u^3 + u^4
plot(h, -2, 2)
We do this by constructing a perturbing extremum-seeking controller ESC
and indicate that the cost function input of the controller, $y$, is related to the controller output, $u$, as $y = h(u)$:
using JuliaSimControl, ModelingToolkit, OrdinaryDiffEq
@named escont = ESC(k=0.8, a=0.1, w=3, wh=1)
connections = [
escont.y ~ h(escont.u)
]
@named closed_loop = ODESystem(connections, systems=[escont])
sys = structural_simplify(closed_loop)
x0 = []
prob = ODEProblem(sys, x0, (0, 300.0))
sol = solve(prob, Rodas5())
plot(sol, vars=[escont.u, escont.uh, escont.y])
As we can see above, the ESC
explores the input space by means of a sinusoidal dither signal, and finds the global optimum of $h(1) = -3$. Had we started in $u=-1.5$ instead of $u=0$, we would have gotten stuck in the local minimum $h(-1)=1$ for much longer:
x0 = [
escont.uh => -1.5
]
prob = ODEProblem(sys, x0, (0, 300.0))
sol = solve(prob, Rodas5())
plot(sol, vars=[escont.u, escont.uh, escont.y])
Increasing the dither amplitude solves this problem:
x0 = [
escont.uh => -1.5
escont.a => 0.3
escont.b => 0.3
]
prob = ODEProblem(sys, x0, (0, 300.0))
sol = solve(prob, Rodas5())
plot(sol, vars=[escont.u, escont.uh, escont.y])
In general, increasing the dither amplitude increases the region of convergence and the convergence speed.
Example: Dynamical extremum seeking
In this example, we will use an extremum-seeking controller to control a nonlinear dynamical system with a cost function output. The system to be controlled is given by
\[\begin{aligned} \dot{x}_1(t) &= x_1(t)^2 + x_2(t) + u(t) \\ \dot{x}_1(t) &= x_1(t)^2 - x_2(t)\\ y(t) &= -1 + x_1(t)^2 - x_1(t) \end{aligned}\]
and we will use the controller PIESC
. This controller offers potentially faster convergence compared to the ESC
controller, at the expense of slightly harder tuning.
using JuliaSimControl: t, D
@named esc = PIESC(k=10, tau=0.1, a=10, w=100, wh=1000)
function systemmodel(; name=:systemmodel)
@variables x1(t)=0 x2(t)=0 u(t)=0 y(t)=0
eqs = [
D(x1) ~ x1^2 + x2 + u
D(x2) ~ x1^2 - x2
y ~ -1 + x1^2 - x1
]
ODESystem(eqs, t; name)
end
model = systemmodel()
connections = [
model.y ~ esc.y
esc.u ~ model.u
]
@named closed_loop = ODESystem(connections, t, systems=[model, esc])
sys = structural_simplify(closed_loop)
x0 = [
# model.x1 => 0.5
# model.x2 => 0.25
# esc.uh => -0.5
esc.v => 0
]
prob = ODEProblem(sys, x0, (0, 6.0))
sol = solve(prob, Rodas5())
plot(sol, vars=[model.x1, model.x2, model.y, esc.u], layout=4)