Testmo Automation Linking CLI Extension – Testmo

Automatically link your test automation results to test cases in your Testmo repository with the Testmo automation linking tool.

About Automation Linking

When you submit test automation results to Testmo, your test results appear in automation runs but are not automatically connected to your test cases in the repository. The automation linking tool solves this by automatically creating links between your automation cases and your repository test cases based on configurable rules.

Two-Tier Linking Architecture

Testmo uses a two-tier linking system:

Tier 1 (Automatic): When you submit test results via Testmo CLI, automation cases are automatically created/updated based on the test key field from your JUnit XML. Each test execution is linked to its automation case. This happens automatically—you don't need to do anything.
Tier 2 (This Tool): The automation linking tool creates links between automation cases (from Tier 1) and your repository test cases (manual test cases). This is what enables tracking automation coverage on your test cases.

In short: Test results → Automation cases (automatic) → Repository test cases (this tool)

This makes it easy to:

Track automation coverage - See which test cases have automated tests linked to them
Monitor test execution history - View the latest automation results directly in your test case repository
Identify gaps - Quickly find test cases without automation coverage
Maintain traceability - Keep your test repository synchronized with your automation suite

The automation linking tool works seamlessly with your existing Testmo CLI workflow. You continue to submit your test results with the standard testmo command, and then run the linking tool as a separate step to create the connections.

Supported languages and frameworks

Language Support: The automation linking tool is language-agnostic and works with any programming language and test framework combination, as long as your tests can export results in JUnit XML format.

Supported Languages (for annotation scanning):

Python (pytest, unittest, nose)
Java (JUnit, TestNG)
JavaScript/TypeScript (Jest, Mocha, Jasmine, Vitest)
C# (.NET, NUnit, xUnit, MSTest)
Ruby (RSpec, Minitest)
Go (go test)
Rust (cargo test)
PHP (PHPUnit)
Kotlin (JUnit)
Scala (ScalaTest)
Any other language with text-based comments

Framework Requirement: Your test framework must be able to export test results in JUnit XML format. Most popular testing frameworks support this either natively or via plugins.

Design goals

The automation linking tool is designed to support various use cases and scenarios:

Works as a post-submission step after your existing Testmo CLI workflow
Supports multiple linking methods: source code annotations, explicit configuration mappings, and pattern matching
Integrates with any CI/CD and build system such as GitHub Actions, GitLab CI, Jenkins, and more
Language-agnostic: Works with any test automation framework that generates JUnit XML reports
Configurable linking rules via YAML configuration file
Supports dry-run mode to preview links before creating them
Provides clear reporting on matched and unmatched tests
Non-interactive and suitable for automated CI/CD pipelines

Installation & getting started

The automation linking tool is distributed as an NPM package, separate from the Testmo CLI. You can install it globally or locally in your project.

Installation

$ npm install -g @testmo/testmo-link
$ testmo-link --help
Usage: testmo-link [options]

Link automation test results to repository test cases in Testmo.

Set TESTMO_TOKEN environment variable to your API token.

Options:
  -V, --version                 Output the version number
  --ansi                        Force ANSI console output (colors)
  --no-ansi                     Do not use ANSI console output (colors)
  --verbose                     Enable detailed output
  --instance <url>              Required: The full address of your Testmo
                                instance (https://***.testmo.net)
  --project-id <id>             Required: The ID of the project
  --run-id <id>                 Required: The ID of the automation run to link
  --config <file>               Path to a YAML or JSON configuration file with
                                linking rules
  --dry-run                     Preview what links would be created without
                                making any changes
  --force                       Re-link tests that are already linked to a
                                repository case
  --fail-on-unmatched           Exit with code 1 if any tests could not be
                                matched to a repository case
  --conflict-resolution <mode>  How to handle conflicts between linking
                                methods: error, warn, silent (default: error)
  --proxy <address>             An optional HTTP(S) proxy server to use for all
                                connections
  --output <file>               Write all log output to a file (always includes
                                debug detail regardless of --verbose flag)
  -h, --help                    Display help

The tool requires Node.js 20 or later. Most systems and environments such as CI/CD containers usually already come with NPM installed. If your system is missing NPM/Node.js, or is running an older version, just follow their guide on installing or upgrading these tools.

Please note that you do not need to use Node.js/JavaScript yourself for your projects or as your automation programming language. We just use NPM to conveniently deploy the tool on any platform.

Relationship to @testmo/testmo-cli

@testmo/testmo-link is a separate package from @testmo/testmo-cli. Not all customers need automation coverage tracking, so the two tools are published independently. You can use @testmo/testmo-cli alone to submit test results, and add @testmo/testmo-link only when you want to link those results to manual test cases in your repository.

How automation linking works

Automation linking is a two-step process that works after you submit your test results:

Step 1: Submit test results (existing workflow)

First, you run your tests and submit the results to Testmo using the standard Testmo CLI, exactly as you do today:

$ export TESTMO_TOKEN=********
$ testmo automation:run:submit \
    --instance https://<your-name>.testmo.net \
    --project-id 1 \
    --name "Backend Tests" \
    --source "backend-unit" \
    --results results/*.xml

The Testmo CLI outputs the automation run ID as a plain integer on stdout upon successful completion:

Progress messages (file scanning, upload progress) are written to stderr and do not interfere with capturing the run ID.

You'll need to capture this run ID for the next step.

Step 2: Link tests to cases (new step)

After submitting your results, you run the linking tool with the run ID:

$ testmo-link \
    --instance https://<your-name>.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml

The tool then:

Retrieves the test results from your automation run
Applies the linking rules from your configuration file
Creates links between automation tests and repository test cases
Reports a summary of linked and unmatched tests

That's it! Your automation results are now linked to your test cases and will appear in your test repository.

Complete workflow example

Here's a full example showing how to integrate automation linking into your existing workflow:

Before (existing workflow)

#!/bin/bash

# Run tests
npm test

# Submit results to Testmo
export TESTMO_TOKEN=********
testmo automation:run:submit \
    --instance https://example.testmo.net \
    --project-id 1 \
    --name "Build #$CI_BUILD_NUMBER" \
    --source "backend-unit" \
    --results results/*.xml

After (with automation linking)

#!/bin/bash

# Run tests
npm test

# Submit results to Testmo
export TESTMO_TOKEN=********
testmo automation:run:submit \
    --instance https://example.testmo.net \
    --project-id 1 \
    --name "Build #$CI_BUILD_NUMBER" \
    --source "backend-unit" \
    --results results/*.xml

Configuration file

The automation linking tool uses a YAML or JSON configuration file to define how test results should be linked to test cases. The configuration supports three different linking methods.

Note: The configuration file can be named anything you like and placed anywhere in your project. Common names include testmo-linking.yml, testmo-link.yml, .testmo/linking.yml, etc. Use the --config <file> flag to specify the path. The examples below use testmo-linking.yml as a convention.

Basic configuration

Create a file called testmo-linking.yml in your project:

# Linking method precedence (which methods to try and in what order)
precedence:
  - annotation    # Try source code annotations first
  - config        # Then explicit mappings
  - pattern       # Finally pattern matching

# Method 1: Scan source code for annotations
annotations:
  enabled: true
  paths:
    - "test/**/*.py"
    - "test/**/*.java"
  patterns:
    - "@TestmoId:(\d+)"

# Method 2: Explicit test-to-case mappings
config_mappings:
  - test_name: "testLoginWithUser"
    case_id: 42
  - test_name: "testCheckoutFlow"
    case_id: 100

# Method 3: Pattern matching (case-insensitive name comparison)
pattern_matching:
  enabled: true

Linking methods explained

The automation linking tool supports three methods for linking tests to cases. You can use one, two, or all three methods together. The tool tries each method in the order specified by the precedence setting.

Requirements by Linking Method

Before choosing a linking method, verify your environment meets the requirements:

Requirement	Annotations	Explicit Mappings	Pattern Matching
Source code checkout	✅ Required	❌ Not needed	❌ Not needed
JUnit XML with `file` attribute	⚠️ Optional (disambiguation only)	⚠️ Optional (for filtering)	⚠️ Optional (for filtering)
JUnit XML with `line` attribute	❌ Not used	❌ Not needed	❌ Not needed
Test case IDs known	✅ Required (add to code)	✅ Required (add to config)	❌ Not needed (auto-discovered)
Exact test names	❌ Not needed	✅ Required	⚠️ Optional (fuzzy matching)
Safe in CI without repo	❌ No (needs source)	✅ Yes	✅ Yes
Risk of wrong links	⬇️ Low (explicit IDs)	⬇️ Low (explicit mapping)	⬆️ Higher (auto-matching)

Legend:

✅ Required: Method won't work without this
⚠️ Conditionally needed: Required for certain scenarios
❌ Not needed: Method works without this
⬇️ Low risk: Explicit configuration prevents mis-linking
⬆️ Higher risk: Auto-matching can create wrong links if not validated

Note on file attribute: Annotation scanning matches by test name — the file and line attributes in JUnit XML are not required. The file attribute is used only as a tiebreaker when two source files contain a test function with the same name. Most frameworks work without it.

Method 1: Annotations

Link tests to cases by adding special comments in your test source code.

Language Support: Annotation scanning is language-agnostic and works with any programming language that uses text-based comments or decorators. The tool performs regex pattern matching on source files, so it supports Python, Java, JavaScript, TypeScript, C#, Ruby, Go, Rust, PHP, and any other language where you can add text annotations.

Example in Python:

# @TestmoId:42
def test_login_with_user():
    # Test implementation
    pass

Decorators between the annotation and the function definition are skipped automatically:

# @TestmoId:42
@pytest.mark.parametrize("username,password", [("user1", "pass1"), ("admin", "admin123")])
def test_login_with_user(username, password):
    pass

Example in Java:

// @TestmoId:42
@Test
public void testLoginWithUser() {
    // Test implementation
}

Example in JavaScript/TypeScript:

// @TestmoId:42
test('login with user', () => {
    // Test implementation
});

Example in C#:

// @TestmoId:42
[Test]
public void TestLoginWithUser() {
    // Test implementation
}

Example in Go:

// @TestmoId:42
func TestLoginWithUser(t *testing.T) {
    // Test implementation
}

Linking one test to multiple cases:

A single test can be linked to more than one repository case by giving its annotation multiple IDs. Three syntaxes are accepted:

// Consecutive annotation lines
// @TestmoId:34806
// @TestmoId:34843
@Test
public void testNavigateToHomepage() { ... }

// Multiple tags on one comment line
// @TestmoId:34806, @TestmoId:34843, @TestmoId:34844

# Comma-separated IDs after a single tag (spaces optional)
# @TestmoId:100, 200, 300
def test_dashboard_widgets():
    pass

When an annotation carries multiple IDs the test is linked to every one of those repository cases. A single @TestmoId:42 still links to one case, exactly as before.

Annotation placement rule: The annotation must appear on a comment line above the test function (or above any decorators on that function). Placing it inside a docstring or below the function definition will not work:

# ❌ Wrong — annotation inside docstring
def test_login():
    """
    @TestmoId:42
    """
    pass

# ❌ Wrong — annotation below function
def test_login():
    pass
# @TestmoId:42

Configuration:

annotations:
  enabled: true
  paths:
    - "test/**/*.py"       # Python tests
    - "test/**/*.java"     # Java tests
    - "src/**/*.test.ts"   # TypeScript tests
    - "src/**/*.test.js"   # JavaScript tests
    - "tests/**/*.cs"      # C# tests
    - "*_test.go"          # Go tests
    - "spec/**/*.rb"       # Ruby tests
    # Add any file patterns for your language
  patterns:
    - "@TestmoId:(\d+)"       # Standard format (always supported)
    - "@TestCaseId:(\d+)"     # Alternative format (optional, configure if needed)

By default, paths are resolved relative to the working directory where testmo-link is run. To resolve paths relative to a different directory, use the base_path option:

annotations:
  enabled: true
  base_path: "/workspace/my-project"   # Resolve glob paths from here
  paths:
    - "test/**/*.py"
  patterns:
    - "@TestmoId:(\d+)"

Note: Additional annotation aliases like @TestCaseId or custom formats are supported when configured in the patterns list. The standard @TestmoId format is recommended for consistency.

Note: The paths key is required for annotation scanning to work. If annotations.enabled: true is set but paths is omitted or empty, the tool emits a warning (annotations.enabled is true but no paths are configured), skips annotation scanning, and affected tests appear as unmatched.

Recently expanded framework support:

Annotation scanning now handles a range of parameterized and framework-specific patterns that earlier versions did not, including:

Python: Multi-line @pytest.mark.parametrize decorators
JavaScript/Jest: test.each() / it.each() parameterized tests and the test.skip, test.only, and test.concurrent modifiers
Java/JUnit 5: @ParameterizedTest with @CsvSource (and other parameter sources) and protected test methods
Cypress: context() / specify() aliases, .skip variants, and ${…} template placeholders
Playwright: fixture-based tests created with test.extend()
Cucumber / Gherkin: .feature files (Scenario, Scenario Outline, Example), with multiple annotated scenarios per file

See the framework compatibility table below for the full list. If a specific structure still does not match, use explicit mappings or pattern matching as an alternative.

Benefits:

Test case ID lives right next to the test code
Easy for developers to see which case a test implements
Survives test name changes
Self-documenting

Requirements:

Test source code files must be present in the environment where testmo-link runs
In CI/CD: Requires repository checkout (e.g., actions/checkout@v3 in GitHub Actions)
Cannot be used if only test results (XML files) are available

Best Practices for Annotation Placement:

To ensure reliable annotation matching, follow these guidelines:

Place annotations immediately ABOVE test function definitions:
```
# @TestmoId:42
def test_login_with_user():
    pass
```
✅ Required - Annotation must be above (not inside or below) the test definition

Multiple tests per file fully supported:

# File: test_auth.py

# @TestmoId:42
def test_login():
    pass

# @TestmoId:43
def test_logout():
    pass

# @TestmoId:44
def test_password_reset():
    pass

✅ Works reliably — each test is matched by name lookup, not line proximity

Avoid incorrect placement:

# ❌ WRONG - Annotation inside function
def test_login():
    # @TestmoId:42  ← Won't match (below test definition)
    pass

# ❌ WRONG - Annotation after function
def test_login():
    pass
# @TestmoId:42  ← Won't match (below test definition)

How annotation matching works:

Source scan: The scanner reads each source file and extracts the test function name from the line immediately below the @TestmoId annotation (skipping blank lines, decorators, and attribute lines automatically). It builds an in-memory index mapping each normalized_name to its source file and case ID(s) — an annotation may specify more than one case ID.
Name lookup: For each test result from the automation run, the linker normalizes the test name (strips parameterized brackets, trailing parentheses, data-provider labels, Go subtest paths; lowercases) and looks it up in the index.
Disambiguation: If the same function name exists in multiple source files, the file attribute from the test result is used to select the correct one. If file is absent and the name is ambiguous, the test is skipped with a warning — it is never wrong-linked.

Name normalization examples:

Reported test name	Normalized	Rule applied
`test_login[admin]`	`test_login`	pytest/JUnit parameterized bracket
`testLoginSuccess()`	`testloginsuccess`	JUnit 5/Kotlin trailing parens
`TestLoginWithRoles/admin`	`testloginwithroles`	Go subtest path
`testLogin with data set "admin"`	`testlogin`	PHPUnit data provider label
`Auth > should login`	tries `should login`	Jest/Playwright describe prefix

Requirements for annotations:

✅ Source code checkout: Test source files must be present where testmo-link runs
⚠️ JUnit XML file attribute: Optional — only needed to disambiguate when the same test function name exists in multiple source files
❌ JUnit XML line attribute: Not used — matching is by normalized test name, not line proximity

Framework compatibility:

Most frameworks work without any file or line attributes in JUnit XML:

Framework	Reported test name	Notes
pytest	`test_login_success`	Works — name matches function name
pytest parameterized	`test_login[admin]`	Bracket suffix stripped automatically
JUnit 4 / Surefire	`testLoginSuccess`	Works — method name
JUnit 5 / Surefire	`testLoginSuccess()`	Trailing `()` stripped automatically
JUnit 5 parameterized	`testWithInput(String)[1]`	Type + index suffix stripped
Jest	`should login successfully`	String literal from `test()`
Jest with describe prefix	`Auth > should login`	Last segment matched
Mocha	`logs in with valid token`	String literal from `it()`
Go top-level	`TestLoginSuccess`	Function name
Go subtests	`TestLoginWithRoles/admin`	Subtest path stripped
RSpec	`logs in with valid credentials`	String literal from `it` block
NUnit	`TestLoginSuccess`	Method name
NUnit parameterized	`TestLoginWithRole (row 1)`	Row suffix stripped
PHPUnit	`testLoginSuccess`	Method name
PHPUnit data provider	`testLogin with data set "admin"`	Data set suffix stripped
Playwright	`should login successfully`	String literal
Playwright file-prefixed	`auth.spec.ts > should login`	Last segment matched
Kotlin / Surefire	`testLoginSuccess()`	Trailing `()` stripped
Kotlin backtick	`login fails with bad password()`	Backtick name extracted
Jest / Vitest `test.each`	`adds 1 + 2 = 3`	Parameter placeholders (`%s`, `$var`, `${expr}`) normalized
Cypress	`logs in successfully`	`context()`/`specify()` aliases; `.skip`; `${…}` placeholders
Playwright fixtures	`should render dashboard`	`test.extend()` fixture-based tests supported
Cucumber / Gherkin	`User logs in with valid credentials`	`.feature` scenarios; multiple annotated scenarios per file

Known limitations:

Scenario	Behaviour	Workaround
JUnit 5 `@DisplayName`	Surefire reports the display name; scanner extracts the method name — they differ	Use `config_mappings` for affected tests, or remove `@DisplayName`
Older NUnit / xUnit	Some versions report fully-qualified names (`Namespace.Class.Method`)	Use `config_mappings` for affected tests

Use this method when:

Your team has access to test source code
CI/CD workspace includes the full repository checkout
You want to maintain links directly in the code
You can structure tests with clear annotation placement

Method 2: Explicit Mappings

Link tests to cases using a configuration file with explicit test name to case ID mappings.

Configuration:

config_mappings:
  # Authentication tests
  - test_name: "testLoginWithUser"
    case_id: 42
  - test_name: "testLoginWithout2FA"
    case_id: 43

  # Checkout tests
  - test_name: "testAddToCart"
    case_id: 100
  - test_name: "testRemoveFromCart"
    case_id: 101

Test names are matched exactly and case-sensitively against the test name as it appears in the JUnit XML / Testmo automation run. testLogin and TestLogin are treated as different names. For parameterized tests, this means each parameter variant needs its own mapping entry (e.g., "test_login[user1-pass1]"), or use pattern matching with a custom regex rule to strip parameter suffixes automatically.

Disambiguating duplicate test names: If the same test name appears in more than one file or suite, add an optional file and/or folder to the mapping so the tool links the correct one:

config_mappings:
  - test_name: "test_login"
    case_id: 42
    file: "tests/auth/test_login.py"   # optional — disambiguates duplicate names
    folder: "tests/auth"               # optional

Benefits:

No source code access required
Centralized configuration
Easy to bulk-edit mappings
Works with any test framework or language
Can be maintained by QA team

Requirements:

Only requires the configuration file (no source code access needed)
Works in any CI/CD environment, even with only test results available

Use this method when:

You don't have access to test source code in your CI/CD pipeline
Test results are generated in isolated containers without source
You prefer centralized configuration management
Multiple teams need to maintain mappings without touching test code

Method 3: Pattern Matching

Automatically link tests to cases based on name comparison. The tool fetches all repository cases for the project and compares each test name against case names using normalized, case-insensitive matching.

Default behavior (no rules):

Test names and case names are both normalized before comparison: lowercased, with spaces, underscores, and hyphens collapsed to a single space.

test name: "testLoginWithUser"   → normalized: "testloginwithuser"
test name: "test_login_with_user" → normalized: "test login with user"
case title: "Test Login With User" → normalized: "test login with user"

Configuration:

pattern_matching:
  enabled: true

Advanced: Regex transform rules

If you need to transform test names before matching (e.g., to strip parameter suffixes), add one or more name_match rules. Each rule applies a regex to the test name; the first capture group (if present) is used as the lookup term. The first rule that produces a match wins.

pattern_matching:
  enabled: true
  rules:
    - type: "name_match"
      pattern: "^([^\[]+)"   # Strip parameter suffix: "test_login[p1-p2]" → "test_login"

Benefits:

Zero manual configuration if naming is consistent
Automatically works for all tests
Scales to large test suites
No maintenance overhead

Requirements:

Only requires test results and Testmo case repository access (no source code needed)
Naming conventions must be consistent between tests and cases

Use this method when:

Your team follows consistent naming conventions between test names and test case titles
You want zero-configuration automated linking
Test naming standards are well-established and enforced

⚠️ Safety considerations:

Pattern matching can create incorrect links if naming is ambiguous. To use this method safely:

Always run with --dry-run first to review matches before creating actual links:

testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-link.yml \
    --dry-run

Ambiguous matches are automatically skipped — if multiple cases share the same normalized name, the test is logged as unmatched with a warning. No configuration is needed to enable this safe behavior.
Common scenarios that cause wrong links:
- Duplicate case titles: "Test Login" appears in multiple test suites
- Similar names: "Login Test", "Test Login", "User Login Test" all normalize similarly
- Legacy tests: Old tests may match unintended cases

Best practice workflow:

# Step 1: Preview matches
testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-link.yml \
    --dry-run

# Step 2: Review output for wrong/ambiguous matches

# Step 3: Fix ambiguous cases with explicit mappings
# Add to config:
# config_mappings:
#   - test_name: "test_login"
#     case_id: 42

# Step 4: Run for real
testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-link.yml

Risk level: ⬆️ Higher risk of incorrect links compared to annotations or explicit mappings. Use pattern matching as a supplement to explicit methods, not as the primary linking strategy for critical tests.

Using multiple methods together

You can combine all three methods for maximum flexibility:

precedence:
  - annotation    # Try annotations first (most explicit)
  - config        # Then explicit mappings (for special cases)
  - pattern       # Finally automatic matching (for the rest)

annotations:
  enabled: true
  paths:
    - "test/**/*.py"
  patterns:
    - "@TestmoId:(\d+)"

config_mappings:
  # Only define exceptions or special cases here
  - test_name: "testLegacyAuthFlow"
    case_id: 999

pattern_matching:
  enabled: true

How it works: For each test, the tool tries methods in order until a match is found. This means:

Tests with @TestmoId annotations are linked first (highest priority)
Tests in the config mappings are linked next
Remaining tests are automatically matched by pattern
Unmatched tests are reported for review

Handling conflicts

When a test matches multiple linking methods with different case IDs, the tool detects this as a conflict. You can configure how conflicts are handled:

settings:
  conflict_resolution: error  # Options: error (default), warn, silent

precedence:
  - annotation
  - config
  - pattern

Conflict resolution modes:

error (default): Counts conflicting tests as errors and reports them in the summary. Execution continues but those tests are not linked.
- Best for catching configuration mistakes early
- Ensures explicit intent
- Example output:
```
Error: Conflict for "test_login": multiple methods returned different case IDs
  (annotation→case 42, config→case 99)
```
warn: Logs warnings but continues execution using the first (highest precedence) match
- Good for gradual migration scenarios
- Conflicts are visible but don't block CI/CD
- Example warning:
```
Warning: Conflict for "test_login": using first match (annotation→case 42, config→case 99)
```
silent: Uses precedence order without warnings
- Not recommended for production use
- Only use if you intentionally have overlapping methods

Example conflict scenario:

# test_login.py
# @TestmoId:42  # Points to case 42
def test_login():
    pass

# testmo-linking.yml
config_mappings:
  - test_name: "test_login"
    case_id: 99  # Points to case 99 - CONFLICT!

With conflict_resolution: error (the default), the conflict is reported and the test is skipped. With conflict_resolution: warn, it will use case 42 (annotation has higher precedence) and warn you about case 99 being ignored.

Overriding settings with command-line flags

Some settings can be specified as either CLI flags or in the YAML configuration file. CLI flags always take priority over YAML settings.

Setting	YAML (`settings:`)	CLI Flag
Verbose logging	`verbose: true`	`--verbose`
Output file	`output_file: path`	`--output <file>`
Conflict resolution	`conflict_resolution: error`	`--conflict-resolution <mode>`
Dry run	CLI only	`--dry-run`
Force re-link	CLI only	`--force`
Fail on unmatched	CLI only	`--fail-on-unmatched`

verbose, output_file, and conflict_resolution can be set in either the YAML config or as CLI flags — CLI flags always take priority. --dry-run, --force, and --fail-on-unmatched are CLI-only flags and are not read from the config file.

Example: Testing different conflict modes

Your config file has conflict_resolution: error, but you want to test warn mode first:

# Use warn mode temporarily without changing the config file
$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --conflict-resolution warn

Example: Strict CI/CD validation

Your config allows unmatched tests, but you want production CI to fail if any tests are unmatched:

# Production pipeline enforces complete coverage
$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --fail-on-unmatched

Example: Development vs. production settings

# Development: Preview with warnings
$ testmo-link --config testmo-linking.yml --dry-run --verbose --conflict-resolution warn

# Production: Strict mode with failures on issues
$ testmo-link --config testmo-linking.yml --conflict-resolution error --fail-on-unmatched

Writing output to a file

Use the --output flag to write a full debug log to a file. When --output is set, all log output continues to appear on the terminal as normal, and additionally every message (including debug-level detail) is written to the specified file — regardless of whether --verbose is active.

This is useful when:

Debugging complex linking scenarios without cluttering CI logs
Generating detailed reports for team review or auditing
Retaining a full trace of what the tool did for a given run

Example:

$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --output testmo-linking-report.log

What gets written to the file:

All messages shown on the terminal
Debug-level detail for every test processed (regardless of --verbose)
API request URLs
Performance timing per phase

Example: Use in CI/CD pipeline

# GitHub Actions example
- name: Link automation tests to cases
  run: |
    testmo-link \
      --instance ${{ secrets.TESTMO_URL }} \
      --project-id 1 \
      --run-id ${{ steps.testmo-submit.outputs.run_id }} \
      --config .testmo/linking.yml \
      --output testmo-linking-report.log

- name: Upload linking report
  if: always()
  uses: actions/upload-artifact@v3
  with:
    name: testmo-linking-report
    path: testmo-linking-report.log

Specifying output file in YAML configuration:

settings:
  verbose: true
  output_file: logs/testmo-linking.log

The CLI flag --output overrides the YAML setting.

Re-linking with --force

By default, the linking tool skips automation cases that are already linked to repository test cases. This prevents accidentally overwriting existing links and makes the tool safe to run multiple times.

Default behavior (without --force):

Automation cases already linked to repository cases are skipped
Logged as "already linked" in the output
Counted separately from unmatched cases in the summary report
Safe to re-run the tool without changing existing links

Force mode (with --force):

Overwrites existing links with new repository case IDs determined by your linking rules
Useful when you've updated your annotations or configuration
Shows count of updated links in the summary report

When to use --force:

Updated annotations - You've changed @TestmoId annotations in your code and need to re-link

# After updating annotations from case 42 to case 100
$ testmo-link --config testmo-linking.yml --run-id 456 --force

Configuration changes - You've modified explicit mappings in your config file
```
$ testmo-link --config testmo-linking.yml --run-id 456 --force
```
Migration corrections - You need to fix incorrect links from a previous run
```
$ testmo-link --config testmo-linking-corrected.yml --run-id 456 --force
```

Example: Safe re-linking workflow

# First run: Create initial links
$ testmo-link --config testmo-linking.yml --run-id 456
  Matched/linked:  3
  Already linked:  0 (skipped)
  Unmatched:       0

# Update annotation in code (change @TestmoId:42 to @TestmoId:100)

# Second run without --force: Already-linked tests are skipped
$ testmo-link --config testmo-linking.yml --run-id 456
  Matched/linked:  0
  Already linked:  3 (skipped)
  Unmatched:       0

# Third run with --force: Re-evaluates and updates
$ testmo-link --config testmo-linking.yml --run-id 456 --force
  Matched/linked:  3
  Already linked:  0 (skipped)
  Unmatched:       0

⚠️ Important notes:

--force is a CLI-only flag — it cannot be set in the YAML config file
Always use --dry-run with --force first to preview changes before applying them
Force mode is destructive — it replaces existing links

Recommended workflow:

# 1. Preview changes with dry-run
$ testmo-link --config testmo-linking.yml --run-id 456 --force --dry-run

# 2. Review the preview output carefully

# 3. Apply changes if correct
$ testmo-link --config testmo-linking.yml --run-id 456 --force

Parameterized / data-driven tests

Parameterized tests run multiple times with different input values, producing multiple test results in JUnit XML — each with a parameter suffix appended to the test name.

Example JUnit XML for a parameterized pytest test:

<testcase name="test_login[user1-pass1]" file="tests/auth.py" line="15" .../>
<testcase name="test_login[user2-pass2]" file="tests/auth.py" line="15" .../>
<testcase name="test_login[admin-admin123]" file="tests/auth.py" line="15" .../>

Method 1: Annotations (recommended)

Place one @TestmoId annotation above the parameterized test function. Name normalization automatically strips the parameterized suffix (e.g., [user1-pass1]) before the lookup, so all parameter variants resolve to the same annotation.

# @TestmoId:42
@pytest.mark.parametrize("username,password", [
    ("user1", "pass1"),
    ("user2", "pass2"),
    ("admin", "admin123"),
])
def test_login(username, password):
    assert login(username, password) == True

Result: All three test executions (test_login[user1-pass1], test_login[user2-pass2], test_login[admin-admin123]) link to Case #42.

No special configuration is needed — bracket suffix stripping is applied automatically during name normalization.

Method 2: Explicit Mappings

Config mappings use exact test name matching. For parameterized tests, each parameter variant needs its own entry, or all variants can be listed under the same case ID:

config_mappings:
  - test_name: "test_login[user1-pass1]"
    case_id: 42
  - test_name: "test_login[user2-pass2]"
    case_id: 42
  - test_name: "test_login[admin-admin123]"
    case_id: 42

Method 3: Pattern Matching

By default, pattern matching compares test names as-is, so test_login[user1-pass1] won't match a case titled "test login". Use a regex rule to strip parameter suffixes first:

pattern_matching:
  enabled: true
  rules:
    - type: "name_match"
      pattern: "^([^\[]+)"   # Capture everything before the first "["

With this rule, test_login[user1-pass1] → search term test_login → matches case "Test Login".

Handling Test Case Status with Parameterized Tests

When multiple parameterized instances are linked to the same test case, Testmo shows:

Overall status: The "worst" result (failed > passed > skipped)
Individual results: All instances remain visible

Test Case #42: "Login with various users"
├─ Status: ❌ FAILED (1 of 3 failed)
├─ test_login[user1-pass1]: ✅ Passed
├─ test_login[user2-pass2]: ❌ Failed
└─ test_login[admin-admin123]: ✅ Passed

Migration Scenarios

Moving from Another Test Management Tool

When you migrate test cases from another tool to Testmo, use explicit config mappings to reconnect existing automation tests to newly imported test cases.

Migration Workflow

Export from previous tool - Get list of test case names and identifiers
Import to Testmo - Test cases get new Testmo case IDs
Create mapping file - Build a testmo-linking.yml with explicit mappings
Run linking - Execute testmo-link to create all links

# Prioritize explicit mappings during migration
precedence:
  - config
  - annotation
  - pattern

config_mappings:
  - test_name: "testLoginWithUser"
    case_id: 42
  - test_name: "testLogoutFlow"
    case_id: 43
  - test_name: "testCheckoutFlow"
    case_id: 100
  # ... hundreds or thousands of mappings

Large Migration Support

The testmo-link tool efficiently handles large migrations:

10,000+ mappings supported
Memory efficient: Streaming YAML parser
Progress reporting: Use --verbose to track progress

# testmo-linking.yml with thousands of mappings
testmo-link \
  --instance https://your-instance.testmo.net \
  --project-id 1 \
  --run-id 123 \
  --config testmo-linking.yml \
  --verbose

Gradual Migration Path

You don't need to migrate all at once:

Phase 1: Use config_mappings for all existing tests
Phase 2: Add @TestmoId annotations to new tests
Phase 3: Gradually refactor old tests to use annotations
Phase 4: Remove mappings as tests are updated

The precedence system handles this gracefully:

Annotations take precedence over config (new tests)
Config mappings take precedence over patterns (legacy tests)

Preview mode (dry-run)

Before creating actual links, you can preview what the tool will do using dry-run mode:

$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --dry-run

Example output:

Scanning files for annotations...
Fetching test results for run 456...
Found 500 test result(s)

[DRY RUN] Would link: "testLoginWithUser" (automation_case_id=101) → case 42 [annotation]
[DRY RUN] Would link: "testPasswordReset" (automation_case_id=102) → case 44 [annotation]
[DRY RUN] Would link: "testLegacyAuthFlow" (automation_case_id=103) → case 999 [config]
[DRY RUN] Would link: "testCheckoutFlow" (automation_case_id=104) → case 100 [pattern]
...

Automation linking complete (1.2s):
  Matched/linked:  485
  Already linked:  0 (skipped)
  Unmatched:       15

This is useful for:

Testing your configuration before going live
Identifying unmatched tests that need attention
Verifying pattern matching is working correctly

CI/CD integration examples

GitHub Actions

name: Test and Link

on: [push]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Run tests
        run: npm test

      - name: Submit results to Testmo
        id: testmo-submit
        run: |
          RUN_ID=$(testmo automation:run:submit \
            --instance ${{ secrets.TESTMO_URL }} \
            --project-id 1 \
            --name "Build ${{ github.run_number }}" \
            --source "backend-unit" \
            --results "results/*.xml")
          echo "run_id=$RUN_ID" >> $GITHUB_OUTPUT
        env:
          TESTMO_TOKEN: ${{ secrets.TESTMO_TOKEN }}

      - name: Link automation results
        run: |
          testmo-link \
            --instance ${{ secrets.TESTMO_URL }} \
            --project-id 1 \
            --run-id ${{ steps.testmo-submit.outputs.run_id }} \
            --config .testmo/linking.yml
        env:
          TESTMO_TOKEN: ${{ secrets.TESTMO_TOKEN }}

GitLab CI

test-and-link:
  stage: test
  script:
    # Run tests
    - npm test

    # Submit to Testmo (run ID printed to stdout as a plain integer)
    - >
      RUN_ID=$(testmo automation:run:submit
      --instance $TESTMO_URL
      --project-id 1
      --name "Build $CI_PIPELINE_ID"
      --source "backend-unit"
      --results "results/*.xml")

    # Link results
    - >
      testmo-link
      --instance $TESTMO_URL
      --project-id 1
      --run-id $RUN_ID
      --config testmo-linking.yml

Jenkins Pipeline

pipeline {
    agent any

    environment {
        TESTMO_TOKEN = credentials('testmo-token')
        TESTMO_URL = 'https://example.testmo.net'
    }

    stages {
        stage('Test') {
            steps {
                sh 'npm test'
            }
        }

        stage('Submit to Testmo') {
            steps {
                script {
                    // testmo prints the run ID as a plain integer to stdout
                    def runId = sh(
                        script: """
                            testmo automation:run:submit \
                                --instance ${TESTMO_URL} \
                                --project-id 1 \
                                --name "Build ${BUILD_NUMBER}" \
                                --source "backend-unit" \
                                --results "results/*.xml"
                        """,
                        returnStdout: true
                    ).trim()

                    env.RUN_ID = runId
                }
            }
        }

        stage('Link Results') {
            steps {
                sh """
                    testmo-link \
                        --instance ${TESTMO_URL} \
                        --project-id 1 \
                        --run-id ${RUN_ID} \
                        --config testmo-linking.yml
                """
            }
        }
    }
}

Exit codes and CI/CD integration

The automation linking tool returns standard exit codes to work seamlessly with CI/CD pipelines:

Exit code 0: Success - all tests processed, links created successfully
Exit code 1: Error - authentication failed, configuration invalid, API errors, or unmatched tests when --fail-on-unmatched is set

By default, the tool exits with code 0 even if some tests remain unmatched (they're reported in the summary). To make your CI/CD pipeline fail when tests cannot be linked:

testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --fail-on-unmatched

This is useful for enforcing 100% automation coverage requirements.

Viewing linked results in Testmo

After running the linking tool, your automation results will be visible in your test repository:

Repository View: Navigate to your test case repository in Testmo
Automation Column: You'll see an "Automation" column showing which test cases have automated tests
Latest Status: The "Status (latest)" column shows the most recent automation result for each case
Test History: Click on any test case to view its full automation execution history
Coverage Report: Use filters to identify test cases without automation coverage

Handling unmatched tests

When the linking tool cannot find a matching test case for a test result, it reports these as "unmatched" tests:

Example output:

Automation linking complete (2.1s):
  Matched/linked:  487
  Already linked:  0 (skipped)
  Unmatched:       13

Resolution suggestions for 13 unmatched test(s):
  - Annotation: ensure @TestmoId:<case_id> is on the line immediately above the test function/method
  - Config mapping: add an entry to config_mappings:
      test_name: "<test name>"
      case_id: <case_id>
  - Pattern matching: add or refine a pattern_matching rule

Why tests might be unmatched

No @TestmoId annotation in source code
Not listed in config mappings
Test name doesn't match any case title after normalization
Ambiguous pattern match (multiple cases share the same normalized name)
Test case doesn't exist in Testmo repository
Test case is in a different project

How to handle unmatched tests

Add annotations - Add @TestmoId:123 to test source code
Update config - Add explicit mapping in config_mappings
Create test case - Create the corresponding test case in Testmo repository
Review naming - Align test names with case titles for pattern matching
Accept as unmatched - Some tests may intentionally not map to cases

Advanced configuration

Verbose output

Enable detailed logging to see exactly how the tool processes each test:

$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --verbose

Custom configuration location

You can store your configuration file anywhere:

$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config .testmo/custom-linking-config.yml

Environment variables

Like the Testmo CLI, the linking tool reads authentication from the TESTMO_TOKEN environment variable:

$ export TESTMO_TOKEN=your-api-token-here
$ testmo-link --instance https://example.testmo.net --project-id 1 --run-id 456 --config testmo-linking.yml

Using HTTP proxies

If you need to connect through an HTTP proxy, you can specify it either as a CLI flag or via standard environment variables:

CLI flag (recommended):

$ testmo-link \
    --instance https://example.testmo.net \
    --project-id 1 \
    --run-id 456 \
    --config testmo-linking.yml \
    --proxy http://proxy.company.com:8080

Environment variables:

# Set proxy for HTTP and HTTPS connections
$ export HTTP_PROXY=http://proxy.company.com:8080
$ export HTTPS_PROXY=http://proxy.company.com:8080

# Or with authentication
$ export HTTPS_PROXY=http://username:password@proxy.company.com:8080

# Then run normally
$ testmo-link --instance https://example.testmo.net --project-id 1 --run-id 456 --config testmo-linking.yml

The CLI --proxy flag takes precedence over environment variables.

Complete configuration reference

# Linking method precedence (required if using more than one method)
# Values: annotation, config, pattern
# Default order when omitted: annotation → config → pattern
precedence:
  - annotation
  - config
  - pattern

# Method 1: Annotation scanning (optional)
annotations:
  enabled: true                          # Enable/disable annotation scanning
  base_path: null                        # Base directory for resolving glob paths
                                         # Defaults to current working directory
  paths:                                 # Glob patterns for source files to scan
    - "test/**/*.py"
    - "test/**/*.java"
    - "test/**/*.js"
    - "test/**/*.ts"
  patterns:                              # Regex patterns to search for (each must have one capture group)
    - "@TestmoId:(\d+)"                 # Standard (recommended)
    - "@TestCaseId:(\d+)"               # Alternative alias (optional)

# Method 2: Explicit mappings (optional)
config_mappings:
  - test_name: "testLoginWithUser"       # Exact test name as it appears in Testmo
    case_id: 42                          # Testmo repository case ID
    file: null                           # Optional: source file, disambiguates duplicate names
    folder: null                         # Optional: folder/suite, disambiguates duplicate names
  - test_name: "testPasswordReset"
    case_id: 44

# Method 3: Pattern matching (optional)
pattern_matching:
  enabled: true                          # Enable/disable pattern matching
  rules: []                              # Optional regex transform rules
    # - type: "name_match"               # Only supported rule type
    #   pattern: "^([^\[]+)"            # Regex; first capture group used as search term

# Settings (optional)
# Settings configurable in YAML (all can also be overridden by CLI flags)
# Note: --dry-run, --force, and --fail-on-unmatched are CLI-only flags
#       and cannot be set here.
settings:
  conflict_resolution: error             # How to handle conflicting case IDs
                                         # Options: error (default), warn, silent
                                         # CLI override: --conflict-resolution <mode>
  verbose: false                         # Enable detailed logging
                                         # CLI override: --verbose
  output_file: null                      # Write full debug log to file (path)
                                         # CLI override: --output <file>

Best practices

1. Start with dry-run mode

Always test your configuration with --dry-run first:

# Test configuration
testmo-link --run-id 456 --config testmo-linking.yml --dry-run

# Once verified, run for real
testmo-link --run-id 456 --config testmo-linking.yml

2. Use version control for configuration

Store your testmo-linking.yml file in version control alongside your test code:

project/
├── test/
├── testmo-linking.yml  ← Version controlled
└── .github/
    └── workflows/

3. Start with one method, add more later

Begin with the simplest method that works for your team:

For teams with source access:

precedence:
  - annotation

annotations:
  enabled: true
  paths: ["test/**/*.py"]
  patterns: ["@TestmoId:(\d+)"]

For teams without source access:

precedence:
  - config

config_mappings:
  - test_name: "testLoginFlow"
    case_id: 42

4. Use pattern matching for scale

If you have consistent naming conventions, pattern matching requires zero maintenance:

precedence:
  - pattern

pattern_matching:
  enabled: true

5. Monitor unmatched tests

Review unmatched tests regularly and decide whether to:

Add missing test cases to Testmo
Add annotations or config mappings
Accept some tests as intentionally unlinked

Performance and limits

The automation linking tool is designed to handle large test suites efficiently:

Retrieval: Fetches up to 1,000 test results per API page (default 100, configurable 100–1,000 per page)
Linking: Creates links via a single bulk API request, batching up to 500 links per request. This replaces the earlier one-request-per-case approach and greatly reduces the risk of API rate-limiting on large test suites
Large source files: Files over 10MB generate a warning during annotation scanning. Scanning continues — no files are skipped. If you have generated or binary files at that size, exclude them from your annotations.paths globs.
Large config files: Config files over 1MB generate a warning recommending annotations or pattern matching instead. Files over 10MB generate an additional "High memory usage expected" warning. In both cases the tool continues normally — these are advisory warnings only.

Troubleshooting

"No tests found for run ID"

Cause: The run ID doesn't exist or has no test results.

Solution: Verify the run ID is correct and that test results were successfully submitted.

"Configuration file not found"

Cause: The specified configuration file path is incorrect or missing.

Solution: Check the file path and ensure the file exists:

ls -la testmo-linking.yml

Note: The --config flag is optional. If omitted, the tool runs without any linking rules — no links will be created unless you pass a config file.

"Authentication failed"

Cause: Invalid or missing TESTMO_TOKEN environment variable.

Solution: Verify your API token:

echo $TESTMO_TOKEN
# Should output your token (not empty)

"automation_link_na" validation error

Cause: The project does not have automation linking enabled.

Solution: Contact your Testmo administrator to enable automation linking for the project.

"Invalid annotation pattern"

Cause: A regex pattern in the configuration is malformed (e.g., unbalanced brackets or parentheses).

Behavior: The tool exits with code 1 and reports the exact pattern that failed, e.g.:

Pattern configuration error: Invalid pattern `[[@TestmoId:(\d+`...

Solution: Test your regex pattern separately or use the standard pattern:

patterns:
  - "@TestmoId:(\d+)"  # Note: double backslash in YAML

Unsafe annotation pattern (potential ReDoS)

Cause: An annotation pattern uses constructs with nested quantifiers (e.g., (@TestmoId:(\d+)+)+) that could cause catastrophic backtracking.

Behavior: The tool emits a warning and skips that pattern, then continues normally with the remaining patterns. This is not a fatal error — exit code is 0. The warning looks like:

Skipping unsafe annotation pattern (potential ReDoS): `(@TestmoId:(\d+)+)+`

Solution: Simplify the pattern to remove nested quantifiers. Use the standard pattern as a starting point:

patterns:
  - "@TestmoId:(\d+)"

Many unmatched tests

Cause: Linking methods aren't finding matches.

Solutions:

Run with --verbose to see why tests aren't matching
Use --dry-run to preview without creating links
Add more linking methods to your configuration
Review test names and case titles for consistency

"Case not found" errors

Cause: Configuration references case IDs that don't exist in the Testmo project.

Solution:

Verify case IDs exist in your Testmo project
Check for typos in annotations or config mappings
Ensure you're using the correct project ID

Annotations not matching

Symptom: Annotation is present in source code but the test is reported as unmatched.

Cause 1: Test name doesn't match the extracted function name

The most common cause is that the framework reports a different name than the function name in the source. The most frequent case is JUnit 5 @DisplayName — Surefire reports the display name string, but the scanner extracts the method name:

// @TestmoId:42
@Test
@DisplayName("Login succeeds with valid credentials")
public void testLoginSuccess() {}

JUnit XML reports Login succeeds with valid credentials but the scanner indexes testLoginSuccess — these don't match.

Solution: Use config_mappings for tests with @DisplayName, or remove @DisplayName and rely on the method name.

Cause 2: Annotation is inside a docstring or below the function

# ❌ Wrong — won't be found
def test_login():
    """@TestmoId:42"""
    pass

Solution: Move the annotation to a comment line above the function definition:

# @TestmoId:42
def test_login():
    pass

Cause 3: Ambiguous name (same function name in multiple files)

If two source files both have a function named test_login annotated with different case IDs, and the test result doesn't include a file attribute, the scanner cannot choose and skips the test with a warning:

Warning: Annotation ambiguous for "test_login": same name found in multiple files
(tests/auth/test_login.py→case 42, tests/legacy/test_login.py→case 99).
Include the file attribute in test results, or use config_mappings to resolve.

Solution: Either configure your test runner to include the file attribute in JUnit XML, or add an explicit config_mappings entry for the affected tests.

Unknown config key warning

If you see a warning like Unknown config key "patterns" — not recognized and will be ignored, check your top-level config keys. Valid top-level keys are: config_mappings, annotations, pattern_matching, precedence, settings.

A common mistake is using patterns: (the old name) instead of pattern_matching:.

Getting help

If you need assistance with automation linking:

Check this guide for common scenarios
Review the troubleshooting section
Contact Testmo support with details about your setup
Include the output of testmo-link --verbose when reporting issues

Ready to get started? Install the tool and create your first configuration file to begin linking your automation results to your test cases today.

About Automation Linking

Two-Tier Linking Architecture

Supported languages and frameworks

Design goals

Installation & getting started

Installation

Relationship to @testmo/testmo-cli

How automation linking works

Step 1: Submit test results (existing workflow)

Step 2: Link tests to cases (new step)

Complete workflow example

Before (existing workflow)

After (with automation linking)

Configuration file

Basic configuration

Linking methods explained

Requirements by Linking Method

Method 1: Annotations

Method 2: Explicit Mappings

Method 3: Pattern Matching

Using multiple methods together

Handling conflicts

Overriding settings with command-line flags

Writing output to a file

Re-linking with --force

Parameterized / data-driven tests

Method 1: Annotations (recommended)

Method 2: Explicit Mappings

Method 3: Pattern Matching

Handling Test Case Status with Parameterized Tests

Migration Scenarios

Moving from Another Test Management Tool

Migration Workflow

Large Migration Support

Gradual Migration Path

Preview mode (dry-run)

CI/CD integration examples

GitHub Actions

GitLab CI

Jenkins Pipeline

Exit codes and CI/CD integration

Viewing linked results in Testmo

Handling unmatched tests

Why tests might be unmatched

How to handle unmatched tests

Advanced configuration

Verbose output

Custom configuration location

Environment variables

Using HTTP proxies

Complete configuration reference

Best practices

1. Start with dry-run mode

2. Use version control for configuration

3. Start with one method, add more later

4. Use pattern matching for scale

5. Monitor unmatched tests

Performance and limits

Troubleshooting

"No tests found for run ID"

"Configuration file not found"

"Authentication failed"

"automation_link_na" validation error

"Invalid annotation pattern"

Unsafe annotation pattern (potential ReDoS)

Many unmatched tests

"Case not found" errors

Annotations not matching

Unknown config key warning

Related topics

Getting help