XCTest / Xcode – Testmo

Testmo works with any test automation tool, including the official XCTest unit testing framework from Apple for Xcode. This is also the underlying test framework for XCUITest and iOS or macOS tests. This quickstart guide provides an overview of how to report XCTest/XCUITest test automation results to Testmo.

Run XCTest & generate XML file

You can generate JUnit-style XML files from the XCTest & xcodebuild output, which has become a standard format to exchange test results between tools. You can either use the popular xcpretty or xcbeautify tools for this. Both tools work similar and we will use xcbeautify here because it is faster and more frequently updated.

$ set -o pipefail && xcodebuild [flags] test | xcbeautify --report junit

This will run your XCTest tests and automatically write all test results to an XML file in the build/reports directory. You can learn more in the xcbeautify documentation.

Submit test results to Testmo

To submit your test results to Testmo, you simply use our cross-platform testmo CLI tool. The CLI tool is distributed as an NPM package and is easy to install on any system. Simply install our official @testmo/testmo-cli NPM package:

$ npm install -g @testmo/testmo-cli
$ testmo -h

Usage: testmo [options] [command]
[...]

We can now send the XCTest test results to Testmo. To do this, make sure to generate an API key in Testmo from your profile page. The API key is used to authenticate with Testmo to send the results. We can then use the testmo CLI tool so submit our results (note that we first set the TESTMO_TOKEN variable, which the tool expects):

Full example

Single line

Windows

$ export TESTMO_TOKEN=********
$ testmo automation:run:submit \
  --instance https://<your-name>.testmo.net \
  --project-id 1 \
  --name "XCTest test run" \
  --source "unittests" \
  --results results/*.xml

If you run this example on a single command line, make sure to remove the backspace:

$ export TESTMO_TOKEN=********
$ testmo automation:run:submit --instance https://<your-name>.testmo.net --project-id 1 --name "XCTest test run" --source "unittests" --results results/*.xml

On Windows, the command line uses a slightly different syntax to set a variable:

> set TESTMO_TOKEN=********
> testmo automation:run:submit --instance https://<your-name>.testmo.net --project-id 1 --name "XCTest test run" --source "unittests" --results results/*.xml

See command output

Collecting log files ..
Found 1 result file with a total of 855 tests
Created new automation run (ID: 254)
Created new thread (ID: 608)
Sending tests to Testmo ..
Uploading: [|||||||||||||||||||||||||] 100% | ETA: 0s | 855/855 tests
Successfully sent tests and completed run
Marked the run as completed

That's it! 🎉 This will automatically analyze the XML result file, create a new test run in Testmo, submit all tests & results and mark the run as completed. There's no need to manually create any tests, map tests or IDs or build any custom API code. Everything is handled automatically for you.

➡️ Bonus: Launch XCTest with Testmo CLI

In the above example, we first launched XCTest to generate the XML file and then used our CLI tool to submit the results in a second step. As an improvement to the above example, we can ask our CLI tool to launch XCTest (testmo then starts and waits for XCTest to finish). This has the following additional benefits:

a) Capture full console output and send it to Testmo

b) Accurately measure test times

c) Record the XCTest exit code

Launch XCTest with CLI tool

We can ask our CLI tool to launch XCTest, so we can capture the console output and more:

$ export TESTMO_TOKEN=********
$ testmo automation:run:submit \
  --instance https://<your-name>.testmo.net \
  --project-id 1 \
  --name "XCTest test run" \
  --source "unittests" \
  --results results/*.xml \
  -- set -o pipefail && xcodebuild [flags] test | xcbeautify --report junit
    ^ space!                       ^ adjust!

Also learn more and see other examples in the full Testmo CLI guide.

More resources & references

Now that you are familiar with submitting your XCTest test results to Testmo, you might also find the following additional examples, topics, and references useful for more advanced workflows.