Time Capsule for Reproducibility

JuliaHub captures the state of a completed job, including the code, environment, inputs, and outputs, to ensure that the job can be reproduced exactly as it was run. This reproducibility data set ensures that the job can be reproduced exactly as it was originally executed. This feature is vital for maintaining reproducibility, particularly in scientific research, regulated industries, and collaborative projects, by providing a verifiable "single source of truth" for your digital workflows. It enhances scientific transparency, validation, and auditability.

Re-run vs. Reproduce: Understanding the Difference

Here's a quick distinction between "Re-run Job" and "Reproduce Job":

  • Re-run Job: Re-executes the job's code using the current default JuliaHub environment. This is useful for quickly re-running a job with potentially updated dependencies.
  • Reproduce Job: Recreates the exact environment (Julia version, packages, etc.) from the reproducibility data to ensure identical execution conditions. This guarantees that the job runs under the identical conditions of the original execution, ensuring true reproducibility.

You can watch a quick video to see how "Reproduce Job" works:

Guide to Reproducing a Job

Accessing Your Job List

Begin by navigating to the "Jobs" section in your JuliaHub interface. From there, select the "Completed Jobs" tab to view a list of your finished jobs.

Reviewing Job Details for Reproduction

In the list of completed jobs, click on the specific job you wish to reproduce. This action will open a detailed view providing comprehensive information about that job's execution.

JuliaHub automatically saves critical information for each completed batch job, enabling precise reproduction. This information includes:

  • The exact Julia Language Version used (Julia v1.8).
  • Details of each Package and its specific Version (DataFrames v1.2.2).
  • Input Data provided to the job (parameters, resources, datasets).
  • Output Data generated by the job (visualizations, statistics, results).
  • Project Container (project.toml, binary info).
  • Environments and Metadata.

Initiating the Reproduction Process

Within the detailed job view, locate and click the "Reproduce Job" button. This button is found in the top right corner of the job details page.

Upon clicking "Reproduce Job," a confirmation pop-up will appear, indicating that your request to reproduce the job has been successfully submitted. The pop-up will also provide the job name of the newly queued reproduced job.

Running the reproduced job

JuliaHub will automatically queue and start the reproduced job based on the reproducibility data. You will receive a notification when the job begins. These notifications can be viewed by clicking the bell icon in the JuliaHub interface.

Once the reproduced job is complete, it will appear in your "Completed Jobs" list. For easy reference, the reproduced job will display the name (Job ID) of the original job it was reproduced from. You can click on the reproduced job entry to view its details, just like any other completed job.

Comparing Jobs

JuliaHub provides a feature to compare jobs, which is particularly useful for verifying the reproducibility of a job run from reproducibility data. You can compare a reproduced job with its original, with other reproduced jobs, or with any other job.

To compare jobs:

  1. Navigate to the "Completed Jobs" list.
  2. Select the checkboxes next to the jobs you wish to compare (e.g., the original job and its reproduced version).
  3. Click the "Actions" button and select the "Compare" option.

JuliaHub will then display a comparison view, in a pop-up window. This view allows you to see how inputs and outputs might differ between the selected jobs. This helps confirm that reproducing the job with the same inputs yields the same outputs.

Time Capsule to Archive and Group Jobs

JuliaHub allows you to group your jobs under a Time Capsule, capturing the complete state of a computational job at a specific point in time. This ensures precise reproducibility of the job, including its code, environment, and data, ensuring its future reproducibility. Follow these steps to create and manage your time capsules:

Creating a New Time Capsule

Navigate to the "Jobs" section in your JuliaHub interface. Select the "Time Capsules" tab to view all existing time capsules. To create a new one, click on "Create time capsule" located under the "Actions" tab.

Naming and Describing Your Time Capsule

A pop-up window will appear. Enter a descriptive name for your time capsule and an optional description. Click the "Save" button.

Your new time capsule will now be listed under the "Time Capsules" tab.

Managing and Utilizing Your Time Capsule

Click on the time capsule to open its detailed view. Here you will see details like its ID, creation date, and creator. Within the time capsule's detailed view, you can modify its name and description by clicking on the "Edit" button. To permanently remove a time capsule, click on the "Delete" button.

To add jobs, click on the "Add Job" tab.

A pop-up will display a dropdown list of your completed jobs. Select all the jobs you wish to save within this time capsule from the dropdown. Click the "Add" button.

All selected jobs will then be listed under the time capsule, allowing you to "come back in time" and view these jobs whenever required under their original execution conditions.

Removing Jobs from a Time Capsule

Within the time capsule's detailed view, you can select specific jobs that you wish to remove. Once selected, click on the "Remove Selected" button.

A confirmation pop-up will appear; click "Remove" to confirm, and the job will be detached from the time capsule.

Alternative Method: Creating a Time Capsule from Completed Jobs

You can also create a time capsule directly from your "Completed Jobs" view:

Go to the "Completed Jobs" tab. Select the specific jobs you want to include in a new time capsule by checking their respective boxes. Click on the "Actions" tab and choose "Create time capsule" from the dropdown menu. You will be prompted to enter a name and description for the new time capsule.

Once completed, the time capsule will be created with the selected jobs already added and can be viewed from the "Time Capsules" tab.

Time Capsules capture all the materials associated with the batch jobs; they can be used to build a Software Bill of Materials.

Summary

Time Capsule was built to allow organizations that submit research to compliance bodies a way to re-run and reproduce jobs to ensure the outcomes are the same. The added ability to archive a group of jobs helps organizations organize runs into whatever system of hierarchy will allow them to come back and look at that "Time Capsule" even years in the future.

This not only provides absolute confidence in results but also significantly facilitates robust scientific validation, auditing, and streamlines collaborative efforts, making digital workflows transparent and universally verifiable. Embracing JuliaHub's commitment to reproducibility, particularly through the use of Time Capsules, is key to building trust and accelerating innovation in all your projects.