Sysimage Service

The sysimage service is a feature available in JuliaHub, which makes working with sysimages built by PackageCompiler.jl simple and accessible.

When you request a sysimage for your JuliaHub job the service will spawn a build based on your Manifest.toml and automatically attach it to your job. The next time you use the same environment the sysimage will come from the cache saving you the build time. However, you do not have to worry about the sysimage getting out of sync with your environment – when your environment changes, JuliaHub will detect that and automatically build and cache a new sysimage for you.

When to use

You should consider using the sysimage service when:

1. repeatedly running jobs with an environment that takes a noticeable amount of time to instantiate (to save time on each run)
2. running distributed jobs with many processes (to save instantiate time on each process and mitigate any instantiate related worker startup issues)

Availability

Currently the Sysimage Service is only available for standard Julia batch jobs and package based jobs such as My Applications or Registered Applications. Coverage may be extended in the future to further compute types.

You can use the feature when submitting through the website, JuliaHubClient.jl and the Julia IDE interface (JuliaHub.jl support is pending).

How to use the Sysimage Service

To use Sysimage Service navigate to the Jobs from the left menu bar in the homepage.

Once you are redirected to the jobs page click on Start Job at the top right and select Standard Batch.

A pop up will then appear to provide the details of the job that you want to run including selecting Machine type, Mode, Time limit and code file. Click on advance options at the bottom above start option.

To use the Sysimage Service click on the checkbox Use a sysimage and you will also require to add Manifest.toml file to use the service.

1. Make sure you have a Julia package environment (i.e. a Manifest.toml file) ready. When using appbundles, make sure the file is in the directory you are bundling up.
2. Submit a job with the Sysimage build checkbox checked (or equivalent API parameter set to true).
3. On your first run you will see two new jobs in the job list. One is the sysimage build and the second one is your main job.
4. You will see that the sysimage build job will run first and after it's done your main job will run.
5. All done!

Useful tips

Making sure your job used the sysimage

If you're unsure whether the job you just submitted ran with a sysimage properly you can check that in the job logs. Jobs that ran with a sysimage attached will contain this log message or a similar one.

Invalidating a cached sysimage

If you think that the sysimage that was built for your environment wasn't built properly and you want to rebuild it you need to follow these steps:

1. Go to the job details page of a job that uses this environment.
2. In the top part you will see an Clear sysimage cache option.
3. Click on it and you should see a message telling you that one sysimage was successfully invalidated. Clicking on the button again will result in a Number of sysimages invalidated: 0 message, which is expected.

Now you can run a job again and the sysimage will be built from scratch.