# Experiments

In JuliaSimModelOptimizer, the different variations of the model to be ran are called the *experiments*. For example, one experiment may specify that the model should be solved with a driving voltage of 10V pulse, while the next experiment specifies that the driving voltage is a 5V pulse. Each experiment is then optionally tied to a dataset which, when defined in an inverse problem, specifies a multi-simulation optimization problem that the further functions (`calibrate`

, `parametric_uq`

, etc.) generate solutions for. The type of experiment is used to signify what the data corresponds to measuring, i.e. whether the experiment is used to match data of time series, or steady states, etc.

## Experiment Types

The following describes the types of experiments which can be generated.

`JuliaSimModelOptimizer.Experiment`

— Type`Experiment(data, model; kwargs...)`

The `Experiment`

describes an experiment in which `data`

was obtained. The dynamics of the investigated system are represented in `model`

as an ODESystem. The trial is used within the optimization problem, as part of `InverseProblem`

to fit the unknown `model`

parameters and initial conditions to `data`

. In the case of Global Sensitivity Analysis (GSA), the `data`

is not needed and can be assigned to `nothing`

.

A `Experiment`

can modify the defaults values for parameters and initial conditions of `model`

. This modification reflects the conditions under which the experiment was conducted. In order to modify the defaults, one can use the `u0`

keyword argument for initial conditions (e.g. `u0 = [state_name => custom_initial_value]`

) and `params`

for the parameters (e.g. `params = [p1 => specific_value, p2 => other_value]`

).

A required keyword of the `Experiment`

constructor is `tspan`

, which indicates the timespan for which the model equations are solved.

A `Experiment`

can have identifying name, which is assigned to it using the `trial_name`

keyword argument. The default name is "Experiment".

The `save_names`

keyword argument is used to specify which `model`

states are saved. The same states are extracted from `data`

.

A `Experiment`

's contribution to the cost function is computed using the `l2loss`

function by default, but this can be changed by using the `err`

keyword argument (e.g. err = (sol, data) -> compute_error). The function requires 2 arguments, the solution of the trial and the data and is expected to return a scalar value corresponding to the cost of the trial.

If an `MCMCOpt`

method is used, then a likelihood function is needed for each trial. This likelihood describes the distribution that generated the data. By default, a multivariate Normal distribution is used, centered at the solution of the trial at each saved timepoint `u`

with a standard deviation `s`

around it. This can be changed by using the `likelihood`

keyword argument (e.g. likelihood = (u, s) -> MvNormal(u, s)).

The `s`

parameter represents any observation noise around the solution of the trial at each timepoint, `u`

. A prior distribution for `s`

is needed to run an `MCMCOpt`

method. The default value for this prior distribution is an `InverseGamma(2,3)`

. This can be changed by using the `noise_priors`

keyword argument (e.g. `noise_prior = [s1 => Exponential(1), s2 => Gamma(3,2)]`

). If only one prior is given, then it is assumed that it applies to all saved states (`save_names`

). Otherwise a vector containing pairs of states and prior distributions with length equal to `saved_names`

needs to be provided, to set different priors for each state. In the example above, the model had two states `s1`

and `s2`

so two pairs were provided. The noise parameters `s`

are assumed to be unique for each trial and to be constant across timepoints for each state of each trial.

In the case of GSA, a `reduction`

function is used. The output of `reduction`

is expected to be the quantity whose sensitivity is being investigated.

If the trial is part of a collection of `SteadyStateExperiments`

, then `forward_u0=true`

signals that the trial should use the outcome of the `SteadyStateTrial`

of the same collection as its initial condition.

If additional keywords are passed, they will be forwarded to the `solve`

call. For example, one can pass `alg=Tsit5()`

to specify what solver will be used. More information about supported arguments can be found here.

`JuliaSimModelOptimizer.SteadyStateExperiment`

— Type`SteadyStateExperiment(data, model; kwargs...)`

Describes a experiment that is ran until a steady state is reached. This object can be initialized in the same way as a `Experiment`

object, with the only difference being that `data`

needs to be a `Vector`

here. The `data`

in this case represents the values of the saved states when the system has reached its steady state.

See the SciML documentation for background information on steady state problems.

`JuliaSimModelOptimizer.IndependentExperiments`

— Type`IndependentExperiments(experiments...)`

This experiment collection type indicates that each experiment can be solved individualy and that there is no interaction between them. This experiment type is automatically created it the experiments are passed as a `Vector`

(i.e. [experiment1, experiment2])

`JuliaSimModelOptimizer.SteadyStateExperiments`

— Type`SteadyStateExperiments(ss_trial, trials...; postprocess=last)`

`SteadyStateExperiments`

is an experiment collection that describes a steady state experiment (see `SteadyStateExperiment`

) (specified as the first argument) followed by subsequent experiments that can continue using the steady state by setting `forward_u0=true`

. The steady state solution that is passed on can be modified using the `postprocess`

keyword argument, which accepts a function with a single argument that represents the solution of the first trial and returns the state to be further passed on.

`JuliaSimModelOptimizer.ChainedExperiments`

— Type`ChainedExperiments(experiments...)`

`ChainedExperiments`

is an experiment collection that describes experiments that depend on each other for defining the initial conditions. By using `forward_u0=trial1`

in a definition of an experiment, `experiment2`

, we express the fact that the initial condition for `experiment2`

is obtained by solving `experiment1`

first and than applying an user defined transformation to obtain the starting point for `experiment2`

. The transformation can be modified using the `postprocess`

keyword argument, which accepts a function with a single argument that represents the solution of the required experiment and returns the state to be further passed on. By default `last`

is used.

## Simulation and Analysis Functions

To better understand and debug experiments, the experiments come with associated analysis functions to allow for easy investigation of the results in a trial-by-trial form. The following functions help the introspection of such experiments.

`JuliaSimModelOptimizer.simulate`

— Function`simulate(experiment::AbstractExperiment, prob::InverseProblem, x)`

Simulate the given `experiment`

using optimization-state point `x`

, which contains values for each parameter and initial condition that is optimized in `InverseProblem`

`prob`

.

## Loss Functions

By default, the loss function associated with a experiment against its data is the standard Euclidian distance, also known as the L2 loss. However, JuliaSimModelOptimizer provides alternative loss definitions to allow for customizing the fitting strategy.

`JuliaSimModelOptimizer.l2loss`

— Function`l2loss(sol, data)`

Squared error loss :

$\sum_{i=1}^{M} \sum_{j=1}^{N} \left( \text{sol}_{i,j} - \text{data}_{i,j} \right)^2$

where N is the number of saved timepoints and M the number of measured states in the solution

`JuliaSimModelOptimizer.ARMLoss`

— Function`ARMLoss(sol, bounds)`

Allen-Rieger-Musante (ARM) loss :

$\sum_{i=1}^{M} \sum_{j=1}^{N} \text{max} \left[ \left( \text{sol}_{i,j} - \frac{\text{u}_{i,j} + \text{l}_{i,j}}{2} \right)^2 - \left( \frac{\text{u}_{i,j} - \text{l}_{i,j}}{2} \right)^2, 0 \right]$

where N is the number of saved timepoints, M the number of measured states in the solution and `l, u`

are the lower and upper bounds of each measured state respectively.

**Reference**

Allen RJ, Rieger TR, Musante CJ. Efficient Generation and Selection of Virtual Populations in Quantitative Systems Pharmacology Models. CPT Pharmacometrics Syst Pharmacol. 2016 Mar;5(3):140-6. doi: 10.1002/psp4.12063. Epub 2016 Mar 17. PMID: 27069777; PMCID: PMC4809626.