Jobs

const JobReference :: Type

A type constraint on the arguments of many jobs-related functions that is used to specify the job. A job reference must be either a Job object, or an AbstractString containing the unique job ID.

JuliaHub.jobs(; [limit::Integer], [auth::Authentication]) -> Vector{Job}

Retrieve the list of jobs, latest first, visible to the currently authenticated user.

By default, JuliaHub only returns up to 20 jobs. However, this default limit can be overridden by passing the limit keyword (which must be a positive integer).

JuliaHub.job(job::JobReference; throw::Bool=true, [auth::Authentication]) -> Job

Fetch the details of a job based on the job reference ref. Will throw an InvalidRequestError if the job does not exist, or returns nothing if throw=false is passed.

JuliaHub.isdone(::Job)

A helper function to check if a Job is "done", i.e. its status is one of Completed, Stopped, or Failed.

wait_job(
    job::AbstractString;
    interval::Integer = 30, [auth::Authentication]
) -> Job

Blocks until remote job referred to by the job reference job has completed, by polling it with every interval seconds. Returns an updated Job object.

JuliaHub.kill_job(job::JobRefererence; [auth::Authentication]) -> Job

Stop the job referred to by the job reference ref. Returns the updated Job object.

See also: Job.

JuliaHub.extend_job(job::JobReference, extension::Limit; [auth::Authentication]) -> Job

Extends the time limit of the job referred to by the job reference ref by extension (Dates.Period, or Integer number of hours). Returns an updated Job object.

See Limit for more information on how the extension argument is interpreted. Note that Unlimited is not allowed as extension.

See also: Job.

struct JobLogMessage

Contains a single JuliaHub job log message, and has the following fields:

• timestamp :: Union{ZonedDateTime, Nothing}: log message timestamp (in UTC)
• message :: Union{String, Nothing}: log message string. This generally corresponds to one line of printed output

Fields that can also be nothing may be missing for some log messages.

See also: job_logs, job_logs_buffered.

JuliaHub.job_logs(job; offset::Integer = 0, [limit::Integer], [auth::Authentication]) -> Vector{JobLogMessage}

Fetches the log messages for the specified JuliaHub job. The job is specifed by passing the job name as a string, or by passing a Job object (i.e. job::Union{AbstractString,Job}). Returns the log messages as an array of JobLogMessage objects.

Optionally, the function takes the following keyword arguments:

• offset::Integer: the offset of the first log message fetched (0 corresponds to the first message); for the first method, this defaults to 0; however, in the second (callback) case, if offset is not specified, any existing logs will be ignored.

• limit::Integer: the maximum number of messages fetched (all by default)

JuliaHub.job_logs_buffered(
    [f::Base.Callable], job::Union{Job,AbstractString};
    streaming::Bool=true, [offset::Integer],
    [auth::Authentication]
) -> AbstractJobLogsBuffer

A lower-level function to work with log streams, and is particularly useful when working with jobs that have not finished yet and are actively producing new log messages.

The function accepts the following arguments:

• f :: Base.Callable: an optional callback function that gets called every time the buffer is updated. The callback must take two arguments: f(::AbstractJobLogsBuffer, ::AbstractVector). The first argument is the buffer object itself, and the second argument will be passed a read-only view of all the logs that have been loaded into the buffer, including the new ones.
• job :: Union{Job,AbstractString}: either the job name or a Job object.
• streaming :: Bool: if set to true, the buffer object The streaming can be stopped with interrupt!.
• offset :: Integer: optional non-negative value to specify the starting point of the buffer

Interface of the returned object

Returns an instance of the abstract AbstractJobLogsBuffer type. These objects contain log messages (of type JobLogMessage), but not all the log messages are immediately available. Instead, at any given time the buffer represents a continous section of logs that can be extended in either direction.

The following functions can be used to interact with log buffers: job_logs_newer!, job_logs_older!, JuliaHub.hasfirst, JuliaHub.haslast. Additionally, the objects will have a .logs :: Vector{JobLogMessage} property that can be used to access the log messages that have been loaded into the buffer.

See also: job_logs, Job.

JuliaHub.job_logs_older!(
    buffer::AbstractJobLogsBuffer; [count::Integer], [auth::Authentication]
) -> AbstractJobLogsBuffer

Updates the AbstractJobLogsBuffer object by adding up to count log messages to the beginning of the buffer. If count is omitted, it will seek all the way to the beginning of the logs.

If all the logs have already been loaded into the buffer (i.e. JuliaHub.hasfirst(buffer) is true), the function is a no-op.

See also: job_logs_buffered, job_logs_newer!.

JuliaHub.job_logs_newer!(
    buffer::AbstractJobLogsBuffer; [count::Integer], [auth::Authentication]
) -> AbstractJobLogsBuffer

Updates the AbstractJobLogsBuffer object by adding up to count log messages to the end of the buffer. If count is omitted, it will seek all the way to the end of the current logs.

For a finished job, if all the logs have already been loaded into the buffer (i.e. JuliaHub.haslast(buffer) is true), the function is a no-op. If the buffer is actively streaming new logs for a running job, then the function is also a no-op.

See also: job_logs_buffered, job_logs_older!.

abstract type AbstractJobLogsBuffer

Supertype of possible objects returned by job_logs_buffered. See the job_logs_buffered function for a description of the interface.

JuliaHub.hasfirst(::AbstractJobLogsBuffer) -> Bool

Determines whether the job log buffer has the first message of the job logs.

See also: haslast, job_logs_buffered.

JuliaHub.haslast(::AbstractJobLogsBuffer) -> Bool

Determines whether the job log buffer has the last message of the job logs. Note that if the job has not finished, this will always be false, since the job may produce additional logs.

See also: hasfirst, job_logs_buffered.

JuliaHub.interrupt!(::AbstractJobLogsBuffer; wait::Bool=true)

Can be use to interrupt the asynchronous log streaming task. If the log buffer is not streaming, this function is a no-op.

Note that the effect of JuliaHub.interrupt! may not be immediate and the function will block until the task has stopped. wait = false can be passed to make interrupt! return immediately, but in that case the buffer may stream for a little while longer.

JuliaHub.job_files(job::Job, [filetype::Symbol]) -> Vector{JobFile}

Return the list of inputs and/or output files associated job.

The optional filetype argument should be one of :input, :source, :result or :project, and can be used to filter the file list down to either just job input files (such as the appbundle or Julia environment files), or output files (such as the one uploaded via RESULTS_FILE).

Note: job_file(job) is equivalent to job.files, and the latter is preferred. This function is primarily meant to be used when filtering by file type.

See also: Job.

JuliaHub.job_file(job::Job, type::Symbol, filename::AbstractString) -> JobFile | Nothing

Searches for a job output file of a specified type and with the specific filename for job job, or nothing if the file was not found.

type should be one of the standard job file types. See JobFile and job_files for more information.

JuliaHub.download_job_file(file::JobFile, path::AbstractString; [auth]) -> String
JuliaHub.download_job_file(file::JobFile, io::IO; [auth])

Downloads a JobFile to a local path. Alternative, writeable stream object can be passed as the second argument to write the contents directly into the stream.

When a local path is passed, it returns the path (which can be useful when calling the function as e.g. JuliaHub.download_job_file(file, tempname()))). When an IO object is passed, it returns nothing.

For example, to download a file into a temporary file:

julia> file = JuliaHub.job_file(JuliaHub.job("jr-eezd3arpcj"), :result, "outdir.tar.gz")
JuliaHub.JobFile outdir.tar.gz (jr-eezd3arpcj, :result, 632143 bytes)
sha2_256:e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
Uploaded: 2023-03-15T07:59:29.473898+00:00

julia> tmp = tempname()
"/tmp/jl_nE3uvkZwvC"

julia> JuliaHub.download_job_file(file, tmp)
"/tmp/jl_BmHgj8rQXe"

julia> bytes2hex(open(sha2_256, tmp))
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"

Alternatively, you can also download the file into a writable IO stream, such as IOBuffer:

julia> buffer = IOBuffer();

julia> JuliaHub.download_job_file(file, buffer)

julia> bytes2hex(sha2_256(take!(buffer)))
"e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855"

See also: Job, JobFile.

struct Job

Represents a single job submitted to JuliaHub. Objects have the following properties:

• id :: String: the unique, automatically generated ID of the job
• alias :: String: a non-unique, but descriptive alias for the job (often set by e.g. applications)
• status :: JobStatus: a string-like JobStatus object storing the state of the job
• env :: Dict: a dictionary of environment variables that were set when the job was submitted
• results :: String: the output value set via ENV["RESULTS"] (an empty string if it was not explicitly set)
• files :: Vector{JobFiles}: a list of JobFile objects, representing the input and output files of the job (see: job_files, job_file, download_job_file).
• hostname :: Union{String, Nothing}: for jobs that expose a port over HTTP, this will be set to the hostname of the job (nothing otherwise; see: the relevant section in the manual)

See also: job, jobs.

struct JobStatus

Type of the .status field of a Job object, representing the current state of the job. Should be one of: Submitted, Running, Failed, Stopped, Completed.

In practice, the .status field should be treated as string and only used in string comparisons.

See also: isdone.

julia> job = JuliaHub.job("jr-novcmdtiz6")
JuliaHub.Job: jr-novcmdtiz6 (Completed)
 submitted: 2023-03-15T07:56:50.974+00:00
 started:   2023-03-15T07:56:51.251+00:00
 finished:  2023-03-15T07:56:59.000+00:00
 files:
  - code.jl (input; 3 bytes)
  - code.jl (source; 3 bytes)
  - Project.toml (project; 244 bytes)
  - Manifest.toml (project; 9056 bytes)
 outputs: "{}"

julia> job.status == "Submitted"
false

julia> job.status == "Completed"
true

struct JobFile

A reference to a job input or output file, with the following properties:

• .name :: String: the name of the Job this file is attached to
• .type :: Symbol: indicated the file type (see below)
• .filename :: String: file name
• .size :: Int: size of the file in bytes (reported to be zero in cases where the file contents is missing)
• .hash :: Union{FileHash, Nothing}: a FileHash object containing the file hash, but may also be missing (nothing) in some cases, like when the file upload has not completed yet.

The file is uniquely identified by the (job, type, filename) triplet.

The type of the file should be one of:

• :input: various job input files, which includes code, environment, and appbundle files.
• :source: source
• :project: Julia project environment files (e.g. Project.toml, Manifest.toml, Artifacts.toml)
• :result: output results file, defined via the RESULTS_FILE environment variable in the job

Note that some files are duplicated under multiple categories.

See also: Job, job_files, job_file.

struct FileHash

Stores a hash and the algorithm used to calcute it. The object has the following properties:

• .algorithm :: Symbol: hash algorithm
• .hash :: Vector{UInt8}: the hash as a sequence of bytes

function request(
    job::Job,
    method::AbstractString,
    uripath::AbstractString,
    [body];
    [auth::Authentication],
    [extra_headers],
    kwargs...
) -> HTTP.Response

Performs an authenticated HTTP request against the HTTP server exposed by the job (with the authentication token of the currently authenticated user). The function is a thin wrapper around the HTTP.request function, constructing the correct URL and setting the authentication headers.

Arguments:

• job::Job: JuliaHub job (from JuliaHub.job)

• method::AbstractString: HTTP method (gets directly passed to HTTP.jl)

• uripath::AbstractString: the path and query portion of the URL, which gets appended to the scheme and hostname port of the URL. Must start with a /.

• body: gets passed as the body argument to HTTP.jl

Keyword arguments:

• auth :: Authentication: optional authentication object (see the authentication section for more information)

• extra_headers: an iterable of extra HTTP headers, that gets concatenated with the list of necessary authentication headers and passed on to HTTP.request.

• Additional keyword arguments must be valid HTTP.jl keyword arguments and will get directly passed to the HTTP.request function.