Including Data from a Dataset
In the previous sections we have demonstrated how we can generate data by simulating either an ODEProblem or a FMU. But, sometimes, the FMUs or underlying equations cannot be shared due to licensing purposes. In such a case, we need to include dataset, that contain information about simulations such as: trajectories, parameters, states etc.
As we have discussed earlier, in order to train a DigitalEcho we need a ExperimentData object that contains information of the experiment and simulations within that experiment. With JuliaSimSurrogates, we can convert an external dataset into a ExperimentData object.
We set up the enviornment by loading JuliaHub for downloading a public dataset and JLSO for deserializing the dataset.
using JuliaHub, JLSO, DataGenerationThe dataset should be a Julia Dict object, and should adhere to the format: ExperimentData.
We will demonstrate constructing an ExperimentData object for the Robertson Chemical Reactions model. We have already created the dataset, and will use JuliaHub.jl to download the dataset.
train_dataset_name = "robertson"
path_to_dataset = JuliaHub.download_dataset(("juliasimtutorials", train_dataset_name), "path to save");Now that we have the path to our dataset, we can use JLSO to load the dataset into a dictionary.
train_data = JLSO.load(path_to_dataset)[:result]Dict{String, Union{Nothing, Vector}} with 9 entries:
"states_labels" => Any["states_1", "states_2", "states_3"]
"params_labels" => Any["p_1", "p_2", "p_3"]
"controls" => nothing
"controls_labels" => nothing
"observables" => nothing
"params" => Any[[0.0361875, 2.93906e7, 10640.6], [0.0401875, 3.23…
"states" => Any[[1.0 1.0 … 0.132767 0.132729; 0.0 5.80684e-11 … 5…
"observables_labels" => nothing
"ts" => Any[[0.0, 1.60465e-9, 1.76512e-8, 1.78117e-7, 1.43022…As we can see, the above dataset is already in the compatible format mentioned earlier. Since the dataset does not have controls or observables, the corresponding fields and their labels are set to nothing.
@show train_data["controls"]
@show train_data["controls_labels"]
@show train_data["observables"]
@show train_data["observables_labels"]train_data["controls"] = nothing
train_data["controls_labels"] = nothing
train_data["observables"] = nothing
train_data["observables_labels"] = nothingNow we simply call the ExperimentData constructor on this dictionary to construct an ExperimentData object.
ed = ExperimentData(train_data) Number of Trajectories in ExperimentData: 100
Basic Statistics for Given Dynamical System's Specifications
Number of u0s in the ExperimentData: 3
Number of ps in the ExperimentData: 3
╭─────────┬──────────────────────────────────────────────────────────────────...
────╮...
│ Field │...
│...
├─────────┼──────────────────────────────────────────────────────────────────...
────┤...
│ │ ╭────────────┬──────────────┬──────────────┬────────┬─────────...
│ │ │ Labels │ LowerBound │ UpperBound │ Mean │ StdDev...
│ │ ├────────────┼──────────────┼──────────────┼────────┼─────────...
│ │ │ states_1 │ 1.0 │ 1.0 │ 1.0 │ 0.0...
│ u0s │ ├────────────┼──────────────┼──────────────┼────────┼─────────...
│ │ │ ⋮ │ ⋮ │ ⋮ │ ⋮ │...
│ │ │ ⋮ │ ⋮ │ ⋮ │ ⋮ │...
│ │ ├────────────┼──────────────┼──────────────┼────────┼─────────...
│ │ │ states_3 │ 0.0 │ 0.0 │ 0.0 │ 0.0...
│ │ ╰────────────┴──────────────┴──────────────┴────────┴─────────...
├─────────┼──────────────────────────────────────────────────────────────────...
────┤...
│ │ ╭──────────┬──────────────┬──────────────┬─────────────┬────────...
│ │ │ Labels │ LowerBound │ UpperBound │ Mean │ StdDev...
│ │ ├──────────┼──────────────┼──────────────┼─────────────┼────────...
│ │ │ p_1 │ 0.036 │ 0.044 │ 0.04 │...
│ ps │ ├──────────┼──────────────┼──────────────┼─────────────┼────────...
│ │ │ ⋮ │ ⋮ │ ⋮ │ ⋮ │...
│ │ │ ⋮ │ ⋮ │ ⋮ │ ⋮ │...
│ │ ├──────────┼──────────────┼──────────────┼─────────────┼────────...
│ │ │ p_3 │ 9015.625 │ 10992.188 │ 10007.188 │ 581.41...
│ │ ╰──────────┴──────────────┴──────────────┴─────────────┴────────...
╰─────────┴──────────────────────────────────────────────────────────────────...
────╯...
Basic Statistics for Given Dynamical System's Continuous Fields
Number of states in the ExperimentData: 3
╭──────────┬─────────────────────────────────────────────────────────────────...
──╮...
│ Field │...
│...
├──────────┼─────────────────────────────────────────────────────────────────...
──┤...
│ │ ╭────────────┬──────────────┬──────────────┬─────────┬─────────...
│ │ │ Labels │ LowerBound │ UpperBound │ Mean │ StdDev...
│ │ ├────────────┼──────────────┼──────────────┼─────────┼─────────...
│ │ │ states_1 │ 0.081 │ 1.0 │ 0.587 │ 0.299...
│ states │ ├────────────┼──────────────┼──────────────┼─────────┼─────────...
│ │ │ ⋮ │ ⋮ │ ⋮ │ ⋮ │...
│ │ │ ⋮ │ ⋮ │ ⋮ │ ⋮ │...
│ │ ├────────────┼──────────────┼──────────────┼─────────┼─────────...
│ │ │ states_3 │ 0.0 │ 0.919 │ 0.413 │ 0.299...
│ │ ╰────────────┴──────────────┴──────────────┴─────────┴─────────...
╰──────────┴─────────────────────────────────────────────────────────────────...
──╯...