Write a custom analysis
In Dyad, an analysis is a runnable query that can be performed on a model to produce a solution object that is used to build various visualizations to display to the user information about the model. The Dyad Analysis Interface is the Julia-level interface for calling analysis queries and interacting with the solution object.
High-Level Definition
All analyses are defined in terms of extending other analyses. At the lowest level we have the special Analysis
type in Dyad which is extended by all analyses defined by julia packages. We call these analyses "base analyses". When a user writes an analysis for a specific context, they will extend one of the base analyses. This new analysis is named "derived analysis" and it can reference one or more models and it has its own parameters besides the parameters inherited from the base analysis.
Creating analyses in Dyad
A base analysis is specified according to a JSON schema inside .dyad files. This schema should live in the metadata section of a corresponding .dyad file in a top level folder
dyad
with its name matching the abstract spec, i.e.dyad/TransientAnalysis.dyad
defines theTransientAnalysisSpec
with its associated "TransientAnalysisSolution".To define a derived analysis, the user would write Dyad code that extends one of the base analysis types.
The Dyad kernel will codegen julia code specific to that derived analysis.
To run an analysis, a user can either call the derived analysis constructor
<DerivedAnalysisName>Spec
and thenrun_analysis
.run_analysis(spec::AbstractAnalysisSpec)
returns anAbstractAnalysisSolution
.One can know what the available artifacts from an
AbstractAnalysisSolution
are by running the commandAnalysisSolutionMetadata(::AbstractAnalysisSolution)
. This
metadata is designed to be a serializable object which can be stored by Dyad Builder to allow for querying the available visualizations in absence of the AbstractAnalysisSolution
.
For artifacts, such as a standard plot or the generation of a standard data table,
artifacts(::AbstractAnalysisSolution, name::Symbol)
is called using the name of the artifact.Each
AbstractAnalysisSolution
can also be imbued with a "customizable visualization". For the customizable visualization, more customization options are given to the user / Dyad Builder provider, such as the front end allowing the user to choose colors, fonts, etc. For this visualization, the provider
gives a AbstractCustomizableVisualizationSpec
from which customizable_visualization(::AbstractAnalysisSolution, ::AbstractCustomizableVisualizationSpec)
should return a visualization of the "standard form" which satisfies the given spec. If the analysis does not have customizable visualizations, then this method does not need to be implemented and the default fallback of missing
will be used.
Creating analyses in julia only
If one wants to use the DyadInterface interface without using the Dyad codegen, then they would only interact with base analyses. In this case it is recommended to have control over the MTK model creation too. The dyad kernel generates functions that can build the ready-to-use ODESystem
for a particular model, but they also add a caching layer which is to be used by the <DerivedAnalysisName>Spec
constructors. If you are using the base analysis interface, then you should create the model from scratch and not use the cached version.
In this case the steps would be:
Create a base analysis spec by first defining an abstract type
AbstractCustomAnalysisSpec <: AbstractAnalysisSpec
and then aCustomAnalysisSpec <: AbstractCustomAnalysisSpec
.Create a
CustomAnalysisSolution <: AbstractAnalysisSolution
that is a fully serializable struct (viaserialize_solution
).Define
DyadInterface.run_analysis(spec::CustomAnalysisSpec)
that returns aCustomAnalysisSolution
.Define
DyadInterface.AnalysisSolutionMetadata(::CustomAnalysisSolution)
such that it returns the available artifacts that your analysis defines.Define
DyadInterface.artifacts(res::CustomAnalysisSolution, name::Symbol)
which returns the requested artifact from your result based on thename
that is passed.Define
DyadInterface.customizable_visualization(sol::CustomAnalysisSolution, ::AbstractCustomizableVisualizationSpec)
for vizualizations that take user input.
Creating analyses from Dyad Builder
With the current design Dyad Builder can only create derived analyses. As such the user will first select from one of the predefined base analyses types and then fill in the inherited base analysis parameters. The use can potentially add new analysis parameters if they want to override model parameters. For example if a model associated to the analysis has a parameter p
with a default value of 1, the user can override it at the analysis level (without chanigng the original model) by creating an analysis parameter p
that will be identically mapped to the model parameter p
by the Dyad codegen. In this way we can generate JSON files that specify values for p
and change its value without re-running the codegen. Only if the structure of the analysis changes (like adding new analysis parameters) we will need to re-run the codegen.
Abstract Interface Definitions
DyadInterface.AbstractAnalysisSpec Type
AbstractAnalysisSpec
The abstract type for all analysis specifications that maps to Dyad analyses. The subtypes of this type will use run_analysis
to perfom analyses. The result of an analysis is always a subtype of AbstractAnalysisSolution
.
The argument to run_analysis
is a base analysis specification defined on the Julia side as a struct
that subtypes AbstractAnalysisSpec
and referenced on the Dyad side via a partial analysis
that extends Analysis
. Base analysis specifications can be built either manually via their constructors or by derived analysis specifications that are created by the codegen. To manually build a base analysis specification like the TransientAnalysisSpec
, one can use
model = MyModel() # build an MTK model
spec = TransientAnalysisSpec(;
model, name = :MyModelTransient, abstol = 1e-6, reltol = 1e-3, stop = 10.0)
For more details, check out the julia workflow for the Transient Analysis.
The difference between a base analysis and a derived one is that a base analysis does not impose defaults for all of its fields, but the values need to be provided upon construction. A derived analysis will have default values for all its parameters and it will build the appropriate base analysis. Since base analyses and derived ones share behaviours, it is recommended to define an abstract type togheter with the base type. As an example, one can write a show
method for the abstract type that corresponds to the analysis and that would then also provide a show
method that will work for the derived analysis specifications that will be defined by the codegen. Suppose we want to build an analysis named CustomAnalysis
. In this case we would have the following:
abstract type AbstractCustomAnalysisSpec <: AbstractAnalysisSpec end
@kwdef struct CustomAnalysisSpec{M, T} <: AbstractCustomAnalysisSpec
name::Symbol
model::M
analysis_parameter::T
end
which define the base analysis and if we have a particular model, ModelA
, a custom analysis created by the codegen would be something like
@kwdef mutable struct ModelACustomAnalysisSpec <: AbstractCustomAnalysisSpec
name::Symbol = :ModelACustomAnalysis
analysis_parameter::Float64 = 1.0
model::Union{Nothing, ODESystem} = ModelA(; name)
end
and one would use this in the following way:
spec = ModelACustomAnalysisSpec()
run_analysis(spec)
The derived analysis specification is used to fully represent a concrete analysis declaratively and it can be mapped to a JSON file which can be used by Dyad Builder to interact in an efficient manner with the analysis by avoiding codegen on non-structural analysis modifications. For more details see the the JSON workflow for the Transient Analysis.
The corresponding Dyad code would be
partial analysis CustomAnalysis
extends Analysis
parameter analysis_parameter::Real
model::Empty = Empty()
end
for the base analysis, which should be placed in a .dyad
file inside a YourPackage/dyad
folder, and
analysis ModelACustomAnalysis
extends CustomAnalysis(analysis_parameter=1.0)
model = ModelA()
end
for the derived analysis that an user would write. The model
parameter is separate from other parameters of the analysis because one can also override model parameters inside an analysis:
analysis ModelACustomAnalysis
extends CustomAnalysis(analysis_parameter=1.0)
parameter model_parameter1::Real = 1
model = ModelA(model_parameter=model_parameter1)
end
Note that the analysis parameter model_parameter1
is mapped to the model parameter model_parameter
, so the parameters in the analysis are not restricted to the names inside the model. For more details see the the Dyad workflow for the Transient Analysis.
The base analysis also needs a JSON schema
{
"title": "CustomAnalysis",
"type": "object",
"properties": {
"name": {
"type": "string",
"description": "Analysis Type",
"default": "CustomAnalysis"
},
"model": {
"type": "object",
"description": "Model to simulate",
"dyad:type": "component"
},
"analysis_parameter": {
"type": "number",
"description": "Analysis parameter"
},
"required": [
"name",
"analysis_parameter"
]
}
}
which should be placed in a root folder named assets
. In the future this step should be simplified so that we only require just one source for the analysis definition. For a complete implementation, see the TransientAnalysis
.
Note that currently for an analysis to be recognized, the .dyad
file that defines it must be inside of a component library.
DyadInterface.AbstractAnalysisSolution Type
AbstractAnalysisSolution
The abstract type and interface on the Julia result from running an analysis.
interface
All AbstractAnalysisSolution
s must implement this interface:
AnalysisSolutionMetadata(::AbstractAnalysisSolution)
returns a serializable description of the visualizations that can be constructed from the solution object.artifacts(::AbstractAnalysisSolution, name::Symbol)
returns the artifacts of namename
. The allowedartifact
s are defined by theAnalysisArtifactMetadata
provided byget_metadata
.customizable_visualization(::AbstractAnalysisSolution, ::AbstractVisualizationSpec)
returns a visualization object. For example, for aPlotlyVisualizationSpec
, this would return a Plots.jl plot built by the Plotly backend.serialize_solution(serializable_solution::AbstractAnalysisSolution)
anddeserialize_solution(serialized_solution)::AbstractAnalysisSolution
, where the visualizations from the deserialized version should be the same as the version that is never serialized.
DyadInterface.AbstractCustomizableVisualizationSpec Type
AbstractStandardVisualizationSpec
An AbstractCustomizableVisualizationSpec is a visualization spec for a "customizable visualization". Each AbstractAnalysisSolution
is imbued with a customizable visualization which Dyad Builder has more plotting controls on, and this type is the specification of those plot controls on the standrad plot.
Reusable interface utilities
When creating new analyses, it can be useful to be able to reuse the translation of certain parts of the spec.
DyadInterface.ODEProblemConfig Type
ODEProblemConfig(spec::AbstractAnalysisSpec)
Translate ODEProblem specific analysis specification attributes from strings to their julia native counterparts.
Compatible specs need the following fields:
- `alg`
- `start`
- `stop`
- `abstol`
- `reltol`
The following optional fileds are also supported:
- `saveat`
- `dtmax`
The fields that one can use from this struct are:
- `alg`: the ODE integrator to use (supported values in the spec are: "auto", "Rodas5P", "FBDF", "Tsit5").
- `tspan`: the timespan of the problem (obtained from `start` & `stop` in the spec).
- `saveat`: the `saveat` keyword to be passed when solving. Optional in the spec, defaults to `Float64[]`.
- `abstol`: the `abstol` keyword to be passed when solving.
- `reltol`: the `reltol` keyword to be passed when solving.
- `dtmax`: the `dtmax` keyword to be passed when solving. Optional in the spec, defaults to `spec.stop - spec.start`.
Another reusable part is getting a structurally simplified model out of an analysis spec. This can be useful over just calling structural_simplify
in your own analysis becasue it also handles adding additional passes.
DyadInterface.get_simplified_model Function
get_simplified_model(spec::AbstractAnalysisSpec)
This function takes in a AbstractAnalysisSpec and returns a structurally simplified model. If the model is already simmplified in the spec it just returns that. If the spec has additional passes (only IfLifting
for now) for structural_simplify
, they are applied.
The spec needs to contain the model in .model
. For IfLifting
, a boolean field with the same name must be present.
Interface Metadata Queries
DyadInterface.ArtifactMetadata Type
ArtifactMetadata
Metadata describing an available artifacts for a given AbstractAnalysisSolution.
Fields
name::Symbol
: The name of the artifact. This is meant to be a unique symbol identifer which is then used in theartifacts
function in order to choose this plot.type::ArtifactType
: The type of the artifact.title::String
: The title of the artifact. This is meant to be the display name in Dyad Builder for the user to select the artifact.description::String
: The description of the artifact. This is meant to be a more detailed description that will be shown to Dyad Builder user if they ask for more information about the artifact.
DyadInterface.AnalysisSolutionMetadata Type
AnalysisSolutionMetadata(sol::AbstractAnalysisSolution)
A serializable description of the visualizations that are allowed from a given AbstractAnalysisSolution
.
Fields
artifacts
::Vector{ArtifactMetadata}`: a description of the artifacts allowed for the given analysis solutionallowed_symbols::Vector{Symbol}
: the symbols which are allowed to be chosen in the customizable visualization.
DyadInterface.rebuild_sol Function
rebuild_sol(sol::AbstractAnalysisSolution)
[EXPERIMENTAL]: Rebuild the fields of a serialized AbstractAnalysisSolution
such that it can be used for plotting. Currently it is assumend that run_analysis
is stripping the solution such that the resulting AbstractAnalysisSolution
is easily serializable.
Customizable Visualizations
DyadInterface.PlotlyVisualizationSpec Type
PlotlyVisualizationSpec <: AbstractCustomizableVisualizationSpec
The plot controls for a standard Plotly plot.
Fields
symbols::Vector{Symbol}
: The symbols specifying the parts of the solution to plot. This should be a subset of theallowed_symbols
from theAnalysisSolutionMetadata
.plots_attributes
: The plotting attributes changing items like color, font size, etc. for the standard visualization. These attributes are required to match the Plots.jl standard attributes and it is designed to be a valid structure for splatting into a plot call, i.e.plot(something; plots_attributes...)
should generate the correct plot with respect to the given attributes.
DyadInterface.customizable_visualization Function
customizable_visualization(::AbstractAnalysisSolution, ::PlotlyVisualizationSpec)
Generates the standard visualization of a given AbstractAnalysisSolution
. This standard visualization should respect the formatting decisions as specified by PlotlyVisualizationSpec
and the return should be a Plots.jl-generated plot generated by the Plotly backend.
If the analysis does not have customizable visualization, it should return missing
, which is also the default fallback method.
Artifacts
DyadInterface.artifacts Function
artifacts(sol::AbstractAnalysisSolution)
Return the names of the artifacts supported by the given analysis solution based on the AnalysisSolutionMetadata
.
artifacts(sol::AbstractAnalysisSolution, name::Symbol)
Generates the artifact with the given name
for a given AbstractAnalysisSolution
. The name
must be one of the allowed names in the artifacts(sol)
. The result of this function should match the corresponding ArtifactType
declared in the AnalysisSolutionMetadata
.
DyadInterface.ArtifactType Module
ArtifactType
The set of allowed artifact types for an AbstractAnalysisSolution
.
DyadInterface.ArtifactType.PlotlyPlot Constant
ArtifactType.PlotlyPlot
A plot generated by Plots.jl with the Plotly backend to be displayed in Dyad Builder with a few modifications.
sourceDyadInterface.ArtifactType.DataFrame Constant
ArtifactType.DataFrame
A DataFrames.jl DataFrame table of results to be displayed in Dyad Builder in an interactive table.
sourceDyadInterface.ArtifactType.Download Constant
ArtifactType.Download
A blob that is meant to be downloaded on demand by the user. This blob should be a standard serializable object that the user knows how to deal with, such as an FMU or JLD2 file.
sourceRunning analyses
DyadInterface.run_analysis Function
run_analysis(spec::AbstractAnalysisSpec)::AbstractAnalysisSolution
Runs the analysis corresponding to the specification. The return type should always be a subtype of AbstractAnalysisSolution
and be something serializable.
Positional Arguments
spec
:AbstractAnalysisSpec
object.
Serialization
The results of run_analysis
are serialized and can be deserialized to inspect earlier runs. The analysis author can customize this step if needed.
DyadInterface.serialize_solution Function
serialize_solution(filename::AbstractString, serializable_solution::AbstractAnalysisSolution)
Generates a serialized artifact for long term storage and saves it at the filename location. The default implementation accepts any julia type that is serializable via the Serialization package. This function can be overridden if the analysis implementer wants to serialize using a different method.
sourceDyadInterface.deserialize_solution Function
deserialize_solution(filename::AbstractString)::AbstractAnalysisSolution
Regenerates the original solution that was serialized using serialize_solution from an artifact located at the filename path. The default implementation will regenerate any solution the was serialized via the Serialization package. This function can be overridden if the analysis implementer wants to deserialize using a different method.
source