DigitalEcho

The DigitalEcho model that encompass the EmbeddingSystem and the Decoder

@train script

Macro for capturing the code which would be submitted for running a training job on JuliaHub For adding custom serialization, we need to set the JSS_SURROGATE_PATH environment variable to the path of the serialized object.

Arguments

script: Code block enclosed in a begin..end which would be cached and submitted for running a training job on JuliaHub

Example

@train begin
    using Surrogatize
    surrogate = nothing
end

@train begin
    using Surrogatize, JLD2
    surrogate = nothing
    custom_path = "models/surrogate.jld2"
    jldsave(custom_path, result = surrogate)
    ENV["JSS_SURROGATE_PATH"] = custom_path
end

run_training(
    directory,
    batch_image::JuliaHub.BatchImage,
    dataset_name;
    dataset_version,
    surrogate_name,
    auth,
    specs,
    kwargs...
)

Runs a training job on JuliaHub. This function should called after calling @training to cache in the code which needs to be run on the job. It modifies the cached code such that the dataset_name passed is downloaded and the path for it is stored in JSS_DATASET_PATH and the result of the code block is serialised in JLSO format. The expectation is the script ends with the call to which trains a surrogate.

The function then orchestrates a batch job on JuliaHub with the given batch image which executes the code and stores the generated surrogate as a JuliaHub.Dataset.

Arguments

directory: Path to the directory that would be used to upload as an appbundle. Any additional required files like FMU should be inside this directory batch_image: Job image to be used for the batch job. This is of type JuliaHub.BatchImagedataset_name: Name of the dataset which gets downloaded into the job which can be used for training dataset_version: Version of dataset_name. Defaults to the latest version surrogate_name: Name of the dataset in which the result of the script, i.e., the trained surrogate is serialised and uploaded auth: Authentication object of type JuliaHub.Authentication for verification while performing various operations on Juliahub specs: Named Tuple giving the specifications of compute for the batch job, i.e, ncpu, ngpu, memory etc.

Returns

Named tuple of job object of type JuliaHub.Job and result surrogate dataset object of type JuliaHub.Dataset