Dyad

Appearance

ADVANCED

Testing with Dyad ​

Within the Dyad ecosystem, components and analyses are often developed as standard Julia packages. A crucial part of developing robust and reliable models is testing. Dyad encourages and supports test-driven development, where tests are not an afterthought but an integral part of the component and analysis design.

The simplest possible test is a reference test - whether your component, when simulated, matches a reference trajectory or known result.

Here's how you would test a component in Dyad:

dyad
component Hello
  parameter k::Real = 1.0
  variable x::Real = 0.0
relations
  initial x = 1.0
  der(x) = k * x
metadata {
    "Dyad": {
        "tests": {
            "case1": {
                "stop": 10, 
                "expect": {"initial": {"x": 1.0}}
            }
        }
    }
}
end

You can run these tests locally at any time from the Julia REPL:

julia
using Pkg
Pkg.test("YourComponentPackage")

Running tests locally is a great way to catch issues early and ensure that your recent changes haven't introduced any regressions. However, to truly leverage the power of automated testing, we turn to Continuous Integration.

What is CI/CD? ​

Continuous Integration (CI) is the practice of frequently merging all developers' working copies of code to a shared mainline. In practice, this is coupled with an automated system that runs a series of checks and tests every time new code is pushed to the repository. If any test fails, the system immediately notifies the developers, so they can fix the issue before it gets integrated into the main product.

Continuous Delivery or Deployment (CD) is the next logical step after CI. It's the practice of automatically deploying every change that passes the CI stage to a testing or production environment.

For modeling and simulation, this paradigm is incredibly powerful. Imagine developing a complex vehicle dynamics model. A seemingly small change to a suspension component could have unintended consequences on the vehicle's overall stability at high speeds.

  • Without CI: This issue might only be discovered days or weeks later, during manual integration testing, making it difficult to trace back to the original change.

  • With CI: The moment the change is committed, an automated suite of tests runs. These could include simulations that specifically target high-speed stability. If the test fails, the developer is notified immediately.

This tight feedback loop prevents regressions, ensures that the models in your main branch are always in a working state, and gives you the confidence to iterate and innovate quickly. For example, a vehicle manufacturer could use CI/CD to safely test and deploy over-the-air updates that improve suspension performance based on real-world data, knowing that a rigorous, automated testing process has vetted every change.

How to use CI/CD with Dyad ​

For projects hosted on GitHub, GitHub Actions is a popular and powerful CI/CD platform. It's free for open-source projects and well-integrated into the GitHub ecosystem.

You can set up a GitHub Actions workflow to automatically run your Dyad component tests on every commit or pull request. Here is a basic template to get you started. Create a file at .github/workflows/CI.yml in your repository with the following content:

yaml
name: CI

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  test:
    name: Julia ${{ matrix.version }} - ${{ matrix.os }} - ${{ matrix.arch }}
    runs-on: ${{ matrix.os }}
    strategy:
      matrix:
        version:
          - '1.10' # Or your desired Julia version
        os:
          - ubuntu-latest
        arch:
          - x64
    steps:
      - uses: actions/checkout@v4
      - uses: julia-actions/setup-julia@v2
        with:
          version: ${{ matrix.version }}
          arch: ${{ matrix.arch }}
      - uses: julia-actions/cache@v2
      - name: Install dependencies
        run: julia --project -e 'using Pkg; Pkg.instantiate()'
      - name: Run tests
        run: julia --project -e 'using Pkg; Pkg.test()'

Understanding the Workflow ​

  • on: [push, pull_request]: This trigger tells GitHub Actions to run the workflow on every push and pull request to the main branch.

  • jobs: test:: This defines a job named test.

  • runs-on: ${{ matrix.os }}: This specifies that the job will run on the operating system defined in the strategy matrix (in this case, ubuntu-latest).

  • strategy: matrix:: This allows you to run your tests against multiple versions of Julia, operating systems, or architectures. The example runs on Julia 1.10 on Ubuntu.

  • steps:: These are the individual commands that make up the job.

    1. actions/checkout@v4: This step checks out your repository's code into the runner.

    2. julia-actions/setup-julia@v2: This step sets up the specified version of Julia.

    3. julia-actions/cache@v2: This step caches your project's dependencies, so they don't have to be re-downloaded every time, speeding up your workflow.

    4. Install dependencies: This runs Pkg.instantiate() to install all the Julia packages defined in your Project.toml and Manifest.toml.

    5. Run tests: This runs your test suite using Pkg.test(). If any test fails, this step will fail, and the entire CI run will be marked as failed.

By adapting this template, you can create a robust CI pipeline for your Dyad projects, ensuring the quality and reliability of your models as they evolve.