Skip to main content

Run test batches

Running tests one at a time is perfect while you're exploring or troubleshooting. But when you want faster feedback — or need to verify several scenarios at once — reach for a test batch.

A test batch runs multiple tests in a single action, either one after another or in parallel, with a shared configuration defined once for the whole group.

🤔 What is a test batch?​

A test batch is a group of tests executed together with shared settings.

  • Start multiple tests with a single action
  • Define common configuration (variables and gateway) once for the whole batch
  • See results in a condensed batch summary, with per-test details

📍 Where to find batches​

Batches are managed from the Scheduler in AIVA:

🎯 When should you use test batches?​

Test batches are especially helpful when you:

  • Want quick feedback before a release
  • Need to rerun a whole group of tests after a change
  • Test the same scenarios across different environments
  • Save time by running independent tests in parallel

How to select tests for a batch​

You have two ways to decide which tests belong to a batch.

1. Select tests by label​

If you organize your tests using labels, you can start a batch by selecting one or more of them, thus creating a dynamic filter. Every test that has all of the selected labels is included automatically.

This works well when:

  • You group tests by feature, environment, or release
  • You want to rerun the same logical set of tests repeatedly

This is also how the GitHub Action can trigger a batch directly from your CI pipeline, without creating anything in the AIVA UI first.

2. Select tests from the tests table​

You can also pick individual tests directly from the tests table and run them as a batch.

This works well when:

  • You need a one-off combination of tests
  • You want full control over exactly which tests are included

⚙️ Choose the execution mode​

When starting a batch, choose how the tests are executed.

Sequential execution​

Tests run one after another. The next test starts only after the previous one finishes. The order of execution is determined by sorting the test names alphabetically.

Use this mode when:

  • One test sets up or changes data that another test relies on (shared state)
  • You want predictable, ordered execution
  • You are troubleshooting and want clear timing

Parallel execution​

Multiple tests run at the same time, up to a limit you choose.

Use this mode when:

  • Tests are independent
  • You want results as fast as possible
note

The maximum number of parallel tests is limited by your available agents — the workers that actually run your tests. If you select more parallel tests than you have agents, the extra tests wait in a queue until an agent becomes free. If you need more agents, please check our subscription plans or contact us.

🎛️ Override variables and gateways​

When defining a batch, you can override variables and the gateway (the connection used to reach your application). Overrides apply to the whole batch, not to individual tests:

  • All tests in the batch use the same variable values
  • All tests use the same selected gateway

This keeps the batch consistent — you can't set different values for different tests within the same run. It's especially useful for:

  • Running the same tests against a different environment
  • Switching between local, staging, or production gateways
  • Avoiding repetitive configuration for each test

🗓️ Schedule a batch​

A batch can run on demand or on a schedule. When you create or edit a scheduled batch, pick one of three schedule types:

  • Simple interval — every N minutes, hours or days.
  • Day & time — on the days of the week you pick, at the times you pick, in quarter-hour steps. You enter the times in your own time zone; AIVA stores them in UTC.
  • Cron expression — a standard five-field cron expression: minute hour day-of-month month day-of-week. It runs in UTC, not in your time zone.

For example, to run a batch every night at 2:00 UTC, use 0 2 * * *. To run it at 2:00 in your own time zone, shift the hour by your UTC offset (in Prague in winter, UTC+1, that is 0 1 * * *), or enter 2:00 in Day & time.

Both are kept in UTC, so when your clocks change between summer and winter time, a scheduled run comes an hour earlier or later in your time. To move it back, edit the schedule after the change.

🔍 Understanding batch results​

After the batch finishes, a condensed batch result summarizes the outcome of all included tests. Each test ends up in one of the following states:

  • Pass — The test ran successfully and all steps completed as expected.
  • Fail — The test ran but failed on an error or assertion.
  • Skipped — The test could not start at all. One common reason is that the selected gateway was not running.
  • Stopped — The test was manually stopped by a user during execution.

💬 Need help?​

Running batches from the REST API or CI? See Run tests via API and CI/CD integrations. For anything else, join the AIVA community on Discord.