API

ParameterSpace(lb, ub; ...) -> ParameterSpace
ParameterSpace(
    lb,
    ub,
    nsamples;
    ...
) -> ParameterSpace{S} where S<:(AbstractMatrix)
ParameterSpace(
    lb,
    ub,
    nsamples,
    alg;
    labels
) -> ParameterSpace{S} where S<:(AbstractMatrix)

Generate some parameter space within lower and upper bounds using a specified sampling algorithm.

Optional Arguments

• nsamples::Integer: number of samples to generate (defaults to $100$).
• alg<:SamplingAlgorithm: algorithm to generate samples (defaults to Latin Hypercube).
• labels::Vector{String}: names for each parameter of the sampled space (defaults to ["p_1", "p_2", ..., "p_n"] where n is the same as nsamples).

ParameterSpace(
    lb,
    ub,
    samples::AbstractMatrix;
    ...
) -> ParameterSpace{S} where S<:(AbstractMatrix)
ParameterSpace(
    lb,
    ub,
    samples::AbstractMatrix,
    alg;
    labels
) -> ParameterSpace{S} where S<:(AbstractMatrix)

Generate some parameter space using some collection of pre-existing samples.

Note that samples are in matrix form and alg = nothing since the samples already exist.

Optional Arguments

• labels::Vector{String}: names for each parameter of the sampled space (defaults to ["p_1", "p_2", ..., "p_n"] where n is the same as nsamples).

ICSpace(lb, ub; ...) -> ICSpace
ICSpace(
    lb,
    ub,
    nsamples;
    ...
) -> ICSpace{S} where S<:(AbstractMatrix)
ICSpace(
    lb,
    ub,
    nsamples,
    alg;
    labels
) -> ICSpace{S} where S<:(AbstractMatrix)

Generate some initial condition space within lower and upper bounds using a specified sampling algorithm.

Optional Arguments

• nsamples::Integer: number of samples to generate (defaults to $100$).
• alg<:SamplingAlgorithm: algorithm to generate samples (defaults to Latin Hypercube).
• labels::Vector{String}: names for each parameter of the sampled space (defaults to ["p_1", "p_2", ..., "p_n"] where n is the same as nsamples).

ICSpace(
    lb,
    ub,
    samples::AbstractMatrix;
    ...
) -> ICSpace{S} where S<:(AbstractMatrix)
ICSpace(
    lb,
    ub,
    samples::AbstractMatrix,
    alg;
    labels
) -> ICSpace{S} where S<:(AbstractMatrix)

Generate some initial condition space using some collection of pre-existing samples.

Note that samples are in matrix form and alg = nothing since the samples already exist.

Optional Arguments

• labels::Vector{String}: names for each parameter of the sampled space (defaults to ["p_1", "p_2", ..., "p_n"] where n is the same as nsamples).

CtrlSpace(lb, ub; ...) -> CtrlSpace
CtrlSpace(lb, ub, prob_func; ...) -> CtrlSpace
CtrlSpace(
    lb,
    ub,
    prob_func,
    nsamples;
    ...
) -> CtrlSpace{S} where S<:(AbstractMatrix)
CtrlSpace(
    lb,
    ub,
    prob_func,
    nsamples,
    alg;
    param_labels,
    labels
) -> CtrlSpace{S} where S<:(AbstractMatrix)

Generate some control space within specified bounds using a provided control function.

Control space consists of samples for pre-defined time varying inputs that drive a system. For example, a pre-defined time varying input could be a*sin(t + b) where a and b are parameters - each with a lower and upper bound to sample from. In contrast to a system's state space, which represents possible values that state can take depending on model variables, control space depends on the bounded subset of values allowed for controls applied to a system.

Optional Arguments

• nsamples::Integer: number of samples to generate (defaults to $100$).
• alg<:SamplingAlgorithm: algorithm to generate samples (defaults to Latin Hypercube).
• labels::Vector{String}: names for each control parameter of the sampled space (defaults to ["p_1", "p_2", ..., "p_n"] where n is the same as nsamples).
• ctrl_labels::Vector{String} : names for each control value. Defaults to Vector{String}[]

CtrlSpace(
    lb,
    ub,
    prob_func,
    samples::AbstractMatrix;
    ...
) -> CtrlSpace{S} where S<:(AbstractMatrix)
CtrlSpace(
    lb,
    ub,
    prob_func,
    samples::AbstractMatrix,
    alg;
    param_labels,
    labels
) -> CtrlSpace{S} where S<:(AbstractMatrix)

Generate some control space using some collection of pre-existing samples.

Note that samples are in matrix form and alg = nothing since the samples already exist.

CtrlSpace(
    param_space::ParameterSpace,
    prob_func;
    labels
) -> CtrlSpace

Generate a control space by consuming a ParameterSpace. The control functions are parameterized using the prob_func by the parameters sampled from the param_space

AbstractController

Abstract type for controllers. Controllers are used to generate open loop or closed loop control inputs for the system.

Every controller needs to define dispatches for sample_params and construct_ctrl to sample parameters and construct the controller from the sampled parameters respectively.

How to define a new controller?

struct SineController <: AbstractController
    omega_min::Float64
    omega_max::Float64
end

function DataGeneration.sample_params(f::DataGeneration.AbstractSpaceConfig,
                                      alg::SineController)
    rand(Uniform(alg.omega_min, alg.omega_max), f.nsamples)
end

function DataGeneration.construct_ctrl(p,
                                       f::DataGeneration.AbstractSpaceConfig, 
                                       alg::SineController)
    t -> sin.(p .* t)
end

alg = SineController(1.0, 10.0)
lb = [0.0, 0.0]
ub = [10.0, 10.0]
nsamples = 1000
ctrl_space = CtrlSpace(lb, ub, simple_open_loop_f, nsamples, alg;
                           labels = ["F", "Q̇"])
sim_config = SimulatorConfig(ctrl_space)

A random neural controller datagen algorithm. This algorithm generates a random neural network controller with the given number of inputs, hidden layers, and hidden nodes per layer. The number of outputs is equal to the number of bounds in the function space configuration. The activation function is applied to the hidden layers, and the output of the neural network is scaled to the bounds of the function space.

RandomNeuralController(
    n_inputs::Int;
    n_hidden::Int = 5,
    n_hidden_layers::Int = 0,
    out_activation::Function = x -> sin(2pi * x),
    hidden_activation::Function = sin,
    scale::Function = nothing
)

Arguments

• n_inputs::Int: The number of inputs to the controller.
• n_hidden::Int = 5: The number of hidden nodes per layer.
• n_hidden_layers::Int = 0: The number of hidden layers.
• out_activation::Function = x -> sin(2pi * x): The activation function to apply to the last layer.
• hidden_activation::Function = sin: The activation function to apply to the hidden layers.
• scale::Function = nothing: The scale function to apply to the output of the network. When left as nothing, defaults to out -> lb .+ (ub .- lb) .* (out .+ 1) ./ 2 where lb and ub are the lower and upper bounds of the function space configuration. and ub are the bounds defined in the CtrlSpace and out is the output of the network.

Returns

• alg: A random neural controller datagen algorithm.

A SIREN controller datagen algorithm. This algorithm generates an initialized SIREN neural network with the given number of inputs, hidden layers, and hidden nodes per layer. The number of outputs is equal to the number of bounds in the function space configuration. The activation function is applied to the hidden layers, and the output of the neural network is scaled to the bounds of the function space.

SIRENController(
        n_inputs::Int;
        n_hidden::Int = 64,
        n_hidden_layers::Int = 3,
        out_activation::Function = x -> tanh.(x./150.0),
        hidden_activation::Function = Layer.σ_siren,
        omega = 1.0/10.0,
        scale::Function = nothing,
    )

Arguments

• n_inputs::Int: The number of inputs to the controller.
• n_hidden::Int = 64: The number of hidden nodes per layer.
• n_hidden_layers::Int = 0: The number of hidden layers.
• out_activation::Function = x -> tanh.(x./150.0): The activation function to apply to the last layer.
• hidden_activation::Function = Layer.σ_siren: The activation function to apply to the hidden layers.
• omega::Real = 1.0/10.0: The value of omega set for SIREN
• scale::Function = nothing: The scale function to apply to the output of the network. When left as nothing, defaults to out -> lb .+ (ub .- lb) .* (out .+ 1) ./ 2 where lb and ub are the lower and upper bounds of the function space configuration. and ub are the bounds defined in the CtrlSpace and out is the output of the network.

Returns

• alg: A SIREN controller datagen algorithm.

FixedICController(
    controller,
    x0
)

A fixed initial value controller datagen algorithm. This algorithm can be used for generating controllers whose value at t=0 is constant.

Arguments

• controller: Any controller defined in Controllers.jl.
• x0: The fixed value at t=0

Returns

• alg: A Fixed initial value controller datagen algorithm.

simple_open_loop_f(u, p, t)

A simple open loop param_func function which calls the parameter p with the time t vectorized.

Arguments

• u: The state of the system.
• p: The controller constructed using construct_ctrl.
• t: The time.

Returns

• Vector{Float64}: The input produced by the controller.

simple_closed_loop_f(u, p, t)

A simple closed loop param_func function which calls the parameter p with the input u.

Arguments

• u: The state of the system.
• p: The controller constructed using construct_ctrl.
• t: The time.

Returns

• Vector{Float64}: The input produced by the controller.

sample_params(f::AbstractSpaceConfig, alg::AbstractController)

Default implementation of sample_params which returns a list of nothing values of length f.nsamples.

All new controllers should choose to define their own implementation of sample_params if they need to sample parameters to construct the controller from.

See AbstractController for more details and example.

Arguments

• f::AbstractSpaceConfig: The function space configuration.
• alg::AbstractController: The controller type to be used for sampling.

Returns

• A list of parameters.

construct_ctrl(p, f::AbstractSpaceConfig, alg::AbstractController)

Default implementation of construct_ctrl which shouldn't be used. All new controllers should choose to define their own implementation of construct_ctrl.

See AbstractController for more details and example.

Arguments

• p: The parameter set sampled using sample_params.
• ctrl::AbstractController: The controller type to be used for sampling.

Returns

• Matrix{Function}: The a list of controllers with the method signature p -> control_inputs. The shape of the array is (1, f.nsamples).

construct_ctrl(::Nothing, f::FunctionSpaceConfig, alg::RandomNeuralController)

Construct the random neural controller given the f::FunctionSpaceConfig which contains The bounds of the function space and the alg::RandomNeuralController which contains the controller hyperArguments.

construct_ctrl(::Nothing, f::FunctionSpaceConfig, alg::SIRENController)

Construct the SIREN controller given the f::FunctionSpaceConfig which contains The bounds of the function space and the alg::SIRENController which contains the controller hyperparameters.

construct_ctrl(::Nothing, f::FunctionSpaceConfig, alg::FixedICController)

Construct the FixedICController given the f::FunctionSpaceConfig which contains The bounds of the function space and the alg::FixedICController which contains the controller and the initial value.

datagen_tuning_app(; kwargs...)

Starts the DataGeneration tuning app.

Arguments

• kwargs: Optional keyword arguments for PlutoVSCodeApp.create_app function call

add_samples(
    sample_space::AbstractSampleSpace,
    samples;
    alg
) -> Any

Function to add manual samples to an existing sample space.

add_samples(
    sample_space::AbstractCtrlSpace,
    samples::AbstractArray{<:Number};
    alg
) -> Any

Function to add manual samples to an existing control space. The samples passed here are the parameters which are used to generate the functions, otherwise it will lead to an ArgumentError

struct SimulatorConfig{A, S} <: AbstractSpaceConfig

Simulator configurations contain information for all three sampling spaces and the algorithm to combine them:

• ParameterSpace
• ICSpace
• CtrlSpace
• combine_alg

Simulators help users run simulation ensembles over these sampling spaces in parallel, and combine them using the algorithm specified as combine_alg. The combine_alg is defaulted to CrossProduct when not specified. Once a SimulatorConfig object is created, it can be called as a function with an FMU or ODEProblem as its argument (kwargs may also be passed for the running the simulations).

Note that Simulator configurations do not require all sampling spaces. For example, if a system does not have a control space, then a Simulator configuration can be created without one.

struct CrossProduct

Algorithm to combine samples defined in spaces using a cross product.

ExperimentData(dict::AbstractDict)

Constructs an ExperimentData object using a given dictionary of the following format.

Note that the labels in the dictionary must be exactly as shown.

* "states_labels": Vector{String},
* "states": Vector{Matrix{Float64}} with every element matrix being size (state_num, time_num)
* "observables_labels": Vector{String},
* "observables": Vector{Matrix{Float64}} with every element matrix being size (observable_num, time_num)
* "params_labels": Vector{String} every element corresponds to the name of a parameter
* "params": Vector{Vector{Float64}} with every element being a vector of real values
* "controls_labels": Vector{String} every element corresponds to the name of a control
* "controls": Vector{Matrix} where every element matrix of shape (state_num, time_num)
* "ts": Vector{Vector} where every element is a vector of real values corresponding to the time steps the simulation was evaluated at

Each of states, params, controls and ts must be of length of the number of trajectories in the experiment.

Each of states_labels, param_labels, control_labels must of the length corresponding to the number of states, parameters and controls in the experiment respectively.

Note: In the case that any field out of states, controls or params does not exist, it (along with the corresponding labels field) must be set to nothing.

Optional Arguments

• states_interp::AbstractInterpolation : interpolation used for states and observables. Defaulted to CubicSpline
• controls_interp::AbstractInterpolation : interpolation used for controls. Defaulted to ConstantInterpolation

combine(
    ed1::ExperimentData,
    ed2::ExperimentData
) -> ExperimentData{T, S} where {T<:(DSResults{S, O, C, TS} where {S<:(JSSBase.StatsAndVals{S} where S<:(NamedTuple{(:lb, :ub, :mean, :std), <:NTuple{4, Any}})), O<:(JSSBase.StatsAndVals{S} where S<:(NamedTuple{(:lb, :ub, :mean, :std), <:NTuple{4, Any}})), C<:(JSSBase.StatsAndVals{S} where S<:(NamedTuple{(:lb, :ub, :mean, :std), <:NTuple{4, Any}})), TS<:(JSSBase.StatsAndVals{Nothing, _A, Nothing} where _A)}), S<:(DSSpecification{V, S, O, TS} where {V<:(JSSBase.StatsAndVals{S} where S<:(NamedTuple{(:lb, :ub, :mean, :std), <:NTuple{4, Any}})), S<:(JSSBase.StatsAndVals{S} where S<:(NamedTuple{(:lb, :ub, :mean, :std), <:NTuple{4, Any}})), O<:(JSSBase.StatsAndVals{S} where S<:(NamedTuple{(:lb, :ub, :mean, :std), <:NTuple{4, Any}})), TS<:(JSSBase.StatsAndVals{Nothing, _A, Nothing} where _A)})}

Function to construct a new ExperimentData object by combining two existing ExperimentData objects.

@datagen script

Macro for capturing the code which would be submitted for running a data generation job on JuliaHub. For adding custom serialization, we need to set the JSS_DATAGEN_DATASET_PATH environment variable to the path of the serialized object.

Arguments

script: Code block enclosed in a begin..end which would be cached and submitted for running a data generation job on JuliaHub

Example

@datagen begin
    using DataGeneration
    ed = nothing
end

@datagen begin 
    using DataGeneration, JLD2
    ed = nothing
    custom_path = "/dataset/ed_dict.jld2"
    JLD2.save(custom_path, ed)
    ENV["JSS_DATAGEN_DATASET_PATH"] = custom_path
end

run_datagen(
    directory,
    batch_image::JuliaHub.BatchImage;
    dataset_name,
    auth,
    specs,
    kwargs...
)

Function to run a data generation job on JuliaHub.

This function should called after calling @datagen to cache in the code which needs to be run on the job. It modifies the cached code such that the result of the code block is serialised in JLSO format. The expectation is the script ends with the call to which generates an ExperimentData in the dictionary format

The function then orchestrates a batch job on JuliaHub with the given batch image which executes the code and stores the generated data as a JuliaHub.Dataset.

Arguments

directory: Path to the directory that would be used to upload as an appbundle. Any additional required files like FMU should be inside this directory batch_image: Job image to be used for the batch job. This is of type JuliaHub.BatchImagedataset_name: Name of the dataset in which the result of the script is serialised and uploaded auth: Authentication object of type JuliaHub.Authentication for verification while performing various operations on Juliahub specs: Named Tuple giving the specifications of compute for the batch job, i.e, ncpu, ngpu, memory etc.

Returns

Named tuple of job object of type JuliaHub.Job and result dataset object of type JuliaHub.Dataset