Testmo tracks statistics across every automation run and test result it receives. These statistics surface at multiple levels: the automation overview, individual runs, and individual sources. Understanding where each type of data appears and how it is calculated helps you quickly identify problem areas, spot reliability issues, and monitor the health of your test suite over time.
Statistics on the Runs & Results Page
The Runs & results main page opens with three metric cards at the top.
- Runs this week: shows how many automation runs were submitted during the current week and how many of those runs were successful. It also displays the total number of tests across those runs. This card gives you a quick read on recent automation activity and overall pass/fail health without opening individual runs.
- Most active sources: lists the sources that have submitted the most runs recently. It lets you see at a glance which pipelines are most active, which is useful when you have multiple automation sources in a project and want to focus on the busiest ones.
- Recently run: shows a chart of total run activity plotted by month over recent history. Below the chart, a change indicator compares the current month's run count to the previous month, shown as a positive or negative number, or "No change this month" when volume has stayed flat. This card helps you understand whether automation activity is growing, shrinking, or holding steady.
Statistics on an Individual Run
Opening an automation run takes you to its Summary tab, which presents several layers of statistical data.
Test result breakdown
At the top of the Summary tab, a metrics section shows the overall outcome of the run. The larger card on the left is a chart breaking down all test results by status, including the counts for passed, failed, and any other statuses in use. The status tags below the chart are clickable: each one links to the filtered Results tab showing only tests with that status. This makes it easy to jump directly from the summary to the full list of, for example, only the failing tests.
The smaller card on the right shows:
- Total time and thread count, for completed runs.
- Running and elapsed time, for runs still in progress. The time updates live and the card shows an activity indicator so you can tell the run is still accepting results.
Failures
Below the metrics section, the Failures tab is selected by default. It lists the tests that failed in this run, sorted by their test ID. Each row shows the test's status tag, name, folder, and elapsed time for that execution. The list shows up to 25 tests. When the run has more failures than that, a link at the bottom reads "See all failures (N)" linking to the complete failure-filtered Results view.
If the run has no failures, the tab shows "Good job, no failures in this run!" instead of a list.
Failures are available for both active and completed runs. For archived runs, only the first 50 failing tests are retained, so the list reflects the archived subset.
Two buttons below the inline tabs provide quick navigation for the full picture: View all tests opens the complete Results tab, and Failures opens the failure-filtered Results view with the total failure count displayed next to the label.
Slowest tests
The Slowest tests tab lists up to 25 tests sorted by how long they took in this run, longest first. Each row shows the elapsed time as a highlighted tag, the test name, folder, and its status. Tests with no elapsed time data are excluded.
This tab helps you identify the tests that are dragging out total run time, whether due to inherently slow operations or regressions in performance.
Thread statistics
Below the inline tabs, the Execution section shows the threads that made up the run. Each thread displays:
- The thread name.
- A success or failure status tag.
- Elapsed time for that thread.
- The total number of tests in the thread, linked to the Results view filtered to that thread.
- An inline bar chart showing the status breakdown across tests in that thread.
- A failure count, linked to the failure-filtered Results view for that thread.
For runs submitted from a single CI job, there is typically one thread. For parallel test execution, each parallel job appears as its own thread, letting you compare performance and failure rates across jobs independently.
Statistics on the Sources Page
The Sources main page lists all automation sources in the project.
Source cards
Active sources appear as cards. Each card shows:
- The source name.
- A status indicator reflecting the outcome of the most recent run.
- A mini chart showing the number of successful and failed runs per day over recent history, so you can spot periods of instability at a glance.
- Total runs: the cumulative number of runs submitted from this source.
- Average time: the average execution time across all runs, providing a baseline for detecting performance regressions.
Retired sources
Retired sources appear below the active sources in a table. The table shows each source's name, total run count, last run date, and average time. Retired sources are read-only and can no longer receive new results, but their historical data remains accessible.
The right panel on the Sources page shows a count of active sources with the number of retired sources displayed below it. If retired sources exist, the count card also renders a proportional chart so you can see the ratio at a glance.
Statistics on an Individual Source
Opening a source shows a detailed view of that source's historical data. The page is split into a left pane with metrics and run history, and a right sidebar with the source's About details.
Metrics cards
Two metric cards appear at the top of the left pane.
The growth chart shows how this source has changed over time. It plots two trends: average test count per run and average run time. Depending on how long the source has been active, the chart switches between a daily view and a weekly view automatically. This helps you see whether your test suite for this source is growing and whether execution times are trending in the right direction.
The Average time card shows the current average run execution time across all runs for this source, with the total run count displayed as a sub-label. This gives you the historical baseline to compare against individual runs.
Run activity chart and run table
Below the metrics cards, a bar chart shows the run activity for this source, with successful runs and failed runs stacked by day. The chart is interactive: clicking a day in the chart filters the run table below to show only the runs from that date. A header between the chart and the table shows the active date filter and how many runs match. Click the reset icon in the header to return to the full list.
The run table shows the runs for this source with columns for run name, status, elapsed time, failure count, and creation date.
Most failures, Slowest tests, and Flaky tests
Below the run table, three tabs present aggregated statistics across all runs for this source.
- Most failures: lists up to 25 automated tests sorted by total failure count across all runs, highest first. Each row shows the failure count in a red tag, the test name and folder, and the total number of results recorded for that test. Tests that have never failed are excluded from this list.
- Slowest tests: lists up to 25 tests sorted by average elapsed time across all runs, slowest first. The average time appears in a blue tag next to the test name and folder, alongside the total results count.
- Flaky tests: lists up to 25 tests sorted by flaky percentage, highest first. The percentage appears in an orange tag next to the test name and folder.
These three tabs reflect the full lifetime history of the source, not just recent runs. They are updated each time a run completes, so the data stays current as new results come in.
Clicking a test for its history
Clicking any test in the Most failures, Slowest tests, or Flaky tests lists opens a popup for that test. The popup shows the test's name, total result count, and average elapsed time. Below those, a table lists its most recent results, with each row showing the result status, the run it belongs to (linked directly to that test within the run), and the date. This lets you quickly browse where and when a test has been failing or behaving unexpectedly without leaving the source view.
How Flaky Tests Are Detected
A test is considered flaky when its result changes between runs. Specifically, Testmo increments a flaky counter each time a test's outcome switches: from pass to fail, or from fail to pass. The flaky percentage is calculated using our proprietary algorithm.
A test that alternates between passing and failing on every run reaches 100%. A test that passed consistently for a long stretch before starting to fail occasionally shows a much lower percentage. Tests with no outcome changes are excluded from the Flaky tests list entirely.
Because flaky percentage is calculated across the full history of a source, a test that was historically stable but recently became unstable will appear lower on the list than its current behavior might suggest. The list is most useful for identifying tests with a long pattern of inconsistency, while the run-level failure data and the test's individual history popup are better suited to investigating recent regressions.
How Failure and Slowest Rankings Are Calculated
Failure count per test
Each automated test has a cumulative failure count that increases each time it fails in a run. The Most failures tab on the source view ranks tests by this cumulative count. A test that fails once in every run for a year ranks higher than a test that failed many times in a single run but has otherwise been reliable.
For individual runs, the Failures tab shows tests that failed in that specific execution only, not cumulative counts.
Average elapsed time per test
Each test's average elapsed time is recalculated after each run by averaging the elapsed time across all recorded executions for that test within the source. The Slowest tests tab on the source view uses this average, which smooths out one-off performance spikes and reflects the test's typical cost. The Slowest tests tab on an individual run uses the actual elapsed time from that run, which shows the real performance in that specific execution.