Collocation Methods

Introduction

Many methods for solving inverse problems involving ODE's need costly ODE solvers at each iteration to compute the loss. E.g. for L2 loss:

$\mathrm{Loss}_{\mathrm{L2}} = \left\|u - \hat{u}_{\mathrm{solver}}(u_0, t,p) \right\|_2$

Where $u$ is the observed data, $\hat{u}_{\mathrm{solver}}(u_0, t,p)$ is the ODE solver prediction of $u$, given initial conditions $u_0$, timespan $t$ and parameters $p$.

In order to reduce the computational cost, a collocation method may be used [1], [2]. Collocation methods circumvents the need for an ODE solver by estimating the derivatives of the data. The loss is computed between these estimated derivatives and the system dynamics. E.g.

$\mathrm{Loss}_{\mathrm{L2}} = \left\|\hat{u}' - f(u, t,p) \right\|_2$

Where $\hat{u}'$ is the estimated derivatives of the observed data.

JuliaSimModelOptimizer implements three different collocation methods to estimate the derivative of the data: Kernel Collocation, Spline Collocation and Noise Robust Collocation.

Kernel Collocation

JuliaSimModelOptimizer.KernelCollocation — Type
KernelCollocation(; maxiters = nothing, maxtime = nothing,  optimizer = IpoptOptimizer(; max_iter = maxiters), kernel = EpanechnikovKernel(), bandwidth = nothing, cutoff = (0.0, 0.0), kwargs...)

A kernel based collocation method for solving the inverse problem. The derivatives of the data are estimated with a local linear regression using a kernel. The loss is then generated from the error between the estimated derivatives and the time derivatives predicted by solving the ODE system. Atleast one of maxiters or maxtime is required. If both of them are given, optimization stops after either of condition is first met.

Keyword Arguments

  • maxiters: Maximum numbers of iterations when using the method with a time stepping optimizer.
  • maxtime: Maximum time for the optimization process.
  • optimizer: An Optimization.jl algorithm for the global optimization. Defaults to Ipopt.Optimizer which can be used from OptimizationMOI.
  • kernel: The kernel used to estimate the derivatives of the data. For more information, see: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2631937/.
  • bandwidth: The bandwidth used for computing kernel weights for local kernel regression.
  • cutoff: The fraction of data in the beginning and end which is removed for loss computation.
  • kwargs: Keyword arguments passed onto Optimization.solve.
source

Spline Collocation

JuliaSimModelOptimizer.SplineCollocation — Type
SplineCollocation(; maxiters = nothing, maxtime = nothing, optimizer = IpoptOptimizer(; max_iter = maxiters), interp = Datainterpolations.CubicSpline, interp_args = (), cutoff = (0.0, 0.0), kwargs...)

A spline based collocation method for solving the inverse problem. The derivatives of the data are estimated by intepolating the data with a spline. The loss is then generated from the error between the estimated derivatives and the time derivatives predicted by solving the ODE system. Atleast one of maxiters or maxtime is required. If both of them are given, optimization stops after either of condition is first met.

Keyword Arguments

  • maxiters: Maximum numbers of iterations when using the method with a time stepping optimizer.
  • maxtime: Maximum time for the optimization process.
  • optimizer: An Optimization.jl algorithm for the global optimization. Defaults to Ipopt.Optimizer which can be used from OptimizationMOI.
  • interp: The interpolation function used to estimate the derivatives of the data. For more information, see: DataInterpolations.jl.
  • interp_args: Extra arguments apart from states and time to be provided for methods like BSplineInterpolation or BSplineApprox of DataInterpolations.jl.
  • cutoff: The fraction of data in the beginning and end which is removed for loss computation.
  • kwargs: Keyword arguments passed onto Optimization.solve.
source

Noise Robust Collocation

JuliaSimModelOptimizer.NoiseRobustCollocation — Type
NoiseRobustCollocation(; maxiters = nothing, maxtime = nothing, diff_iters, dx, α, optimizer = IpoptOptimizer(; max_iter = maxiters), tvdiff_kwargs = (), cutoff = (0.0, 0.0), kwargs...)

A noise robust collocation method for solving the inverse problem. The derivatives of the data are estimated using total variational regularized numerical differentiation (tvdiff). The loss is then generated from the error between the estimated derivatives and the time derivatives predicted by the ODE solver. Atleast one of maxiters or maxtime is required. If both of them are given, optimization stops after either of condition is first met.

For more information about the tvdiff implementation, see: NoiseRobustDifferentiation.jl.

Keyword Arguments

  • maxiters: Required. Maximum numbers of iterations when using the method with a time stepping optimizer.
  • optimizer: An Optimization.jl algorithm for the global optimization. Defaults to Ipopt.Optimizer which can be used from OptimizationMOI.
  • diff_iters: Required. Number of iterations to run the main loop of the differentiation algorithm.
  • α: Required. Regularization parameter. Higher values increase regularization strength and improve conditioning.
  • tvdiff_kwargs: The keyword arguments passed on to NoiseRobustDifferentiation.tvdiff function for estimating derivatives.
  • cutoff: The fraction of data in the beginning and end which is removed for loss computation.
  • kwargs: Keyword arguments passed onto Optimization.solve.
source

CollocationData for plotting

  • 1Roesch, Elisabeth, Christopher Rackauckas, and Michael PH Stumpf. "Collocation based training of neural ordinary differential equations." Statistical Applications in Genetics and Molecular Biology 20, no. 2 (2021): 37-49.
  • 2Liang, Hua, and Hulin Wu. "Parameter estimation for differential equation models using a framework of measurement error in regression models." Journal of the American Statistical Association 103, no. 484 (2008): 1570-1583. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2631937/