Project Batch Jobs

Projects can be submitted as batch jobs on JuliaHub. This is similar to regular batch jobs on JuliaHub but with the project environment activated in the job processes. The Project must contain Project.toml and Manifest.toml files in its root directory. Like a Julia package source directory, if there is a src directory in the project with a file named the same as the name paramter in the Project.toml, and this file contains a top-level module definition of the same name, then that module is available in the job environment. For example:

Project.toml

name = "MyProject"
uuid = "b827442a-b7b4-4b99-9a82-02445832f675"
authors = ["JuliaHub Inc"]
version = "2.0.0"

[deps]
Dates = "ade2ca70-3891-5945-98fb-dc099432e06a"
HTTP = "cd3eb016-35fb-5094-929b-558a96fad6f3"
JSON = "682c06a0-de6a-54ab-a142-c8b1cf79cde6"

src/MyProject.jl

module MyProject
    # Your code here
end

Then, using MyProject will work in the job environment. See the below steps on how to submit such a code snippet:

  1. Select your project from the Projects list to open the project detailed view.

Project Details

  1. Click on the "Manage" dropdown and select "Manage batch job specifications".

Manage project batch job specifications drop down item

  1. Click on "Add new" to create a new specification.

Project batch job specifications page

  1. Enter the parameters for the job as necessary:

Enter a name and optional description for the specification. Select a Machine type from the dropdown. Choose whether this should be a "single-node" or "distributed" batch job. If you selected "distributed" also select the number of nodes and whether Julia proecesses must be started per vCPU or per node. Select the time limit for the job or disable it. Select whether the code you want to execute is julia or shell. Enter the code snippet that needs to be executed. As mentioned above, your project will be available as a package for the job code, provided you have the correct Project.toml, Manifest.toml and files in src directory. Expand the advanced section to specify environment variables. Click "Create" to save the specification. Add new project batch job specification dialog

  1. Go back to the project detailed view and click on "Create Job". Select the specification from the dropdown list. Review the summary then click "Submit" to start the job.

Create Job in Project Details

  1. Click on the "Job tracker" tab to see the status of the deployed job.

Project Batch Job Tracker

Editing, Deleting project batch job specifications and viewing job results

Project batch job specifications can be created, used for batch job deployments, edited and deleted by all users that have "Editor" access on the Project. Such users also have access to the "Job Tracker" and the details of all jobs in the Job Tracker list for that project.

To edit an existing batch job specification, go to the project batch job specifications page and click the "Edit" button on the row corresponding to the specification that you want to edit. To permanently delete a specification, click "Delete".