Skip to main content

TestNG

testng.png

Testmo works with any test automation tool, including the popular TestNG unit testing framework for Java and other JVM-based languages. This quickstart guide provides an overview of how to report TestNG test automation results to Testmo.

Run TestNG & generate XML file

TestNG comes with full support for generating JUnit-style XML files, which has become a standard format to exchange test results between tools. You can either run TestNG from the command line or as part of your build workflow. You can learn more in the TestNG documentation.

$ java org.testng.TestNG testng1.xml

The testng1.xml file here specifies one of your TestNG suites. After executing your tests, TestNG will automatically generate JUnit-style XML report files in the test-output directory (if this is not the case, please make sure that the org.testng.reporters.JUnitXMLReporter listener is active).

Submit test results to Testmo

To submit your test results to Testmo, you simply use our cross-platform testmo CLI tool. The CLI tool is distributed as an NPM package and is easy to install on any system. Simply install our official @testmo/testmo-cli NPM package:

$ npm install -g @testmo/testmo-cli
$ testmo -h

Usage: testmo [options] [command]
[...]

We can now send the TestNG test results to Testmo. To do this, make sure to generate an API key in Testmo from your profile page. The API key is used to authenticate with Testmo to send the results. We can then use the testmo CLI tool so submit our results (note that we first set the TESTMO_TOKEN variable, which the tool expects):

Full example
Single line
Windows
$ export TESTMO_TOKEN=********
$ testmo automation:run:submit \
  --instance https://<your-name>.testmo.net \
  --project-id 1 \
  --name "TestNG test run" \
  --source "unittests" \
  --results test-output/**/*.xml

If you run this example on a single command line, make sure to remove the backspace:

$ export TESTMO_TOKEN=********
$ testmo automation:run:submit --instance https://<your-name>.testmo.net --project-id 1 --name "TestNG test run" --source "unittests" --results test-output/**/*.xml

On Windows, the command line uses a slightly different syntax to set a variable:

> set TESTMO_TOKEN=********
> testmo automation:run:submit --instance https://<your-name>.testmo.net --project-id 1 --name "TestNG test run" --source "unittests" --results test-output/**/*.xml
See command output
Collecting log files ..
Found 1 result file with a total of 855 tests
Created new automation run (ID: 254)
Created new thread (ID: 608)
Sending tests to Testmo ..
Uploading: [|||||||||||||||||||||||||] 100% | ETA: 0s | 855/855 tests
Successfully sent tests and completed run
Marked the run as completed

That's it! 🎉 This will automatically analyze the XML result file, create a new test run in Testmo, submit all tests & results and mark the run as completed. There's no need to manually create any tests, map tests or IDs or build any custom API code. Everything is handled automatically for you.

➡️ Bonus: Launch TestNG with Testmo CLI

In the above example, we first launched TestNG to generate the XML file and then used our CLI tool to submit the results in a second step. As an improvement to the above example, we can ask our CLI tool to launch TestNG (testmo then starts and waits for TestNG to finish). This has the following additional benefits:

a) Capture full console output and send it to Testmo 

b) Accurately measure test times 

c) Record the TestNG exit code

Launch TestNG with CLI tool

We can ask our CLI tool to launch TestNG, so we can capture the console output and more:

$ export TESTMO_TOKEN=********
$ testmo automation:run:submit \
  --instance https://<your-name>.testmo.net \
  --project-id 1 \
  --name "TestNG test run" \
  --source "unittests" \
  --results test-output/**/*.xml \
  -- java org.testng.TestNG testng1.xml
    ^ space!

Also learn more and see other examples in the full Testmo CLI guide.

➡️ Bonus: Custom fields

Testmo also supports additional custom fields for test runs, threads and tests. You can learn more here:

TestNG does not yet directly support fields/properties per test, but Testmo can also parse and read fields/properties from the test output. And TestNG lets you specify the output it should write to the results XML file. So we can use Testmo's output property syntax to record any additional custom fields for our tests and submit these as fields to Testmo. Testmo supports many automation field types including strings, links, rich HTML, steps and attachment links.

Add custom fields with TestNG

To log additional fields for a test, simply use Testmo's output property syntax with TestNG's Reporter.log method.

package test.users;

import org.testng.Reporter;
import org.testng.annotations.Test;

public class TestUserProfile {
  @Test
  public void testUpdateUserEmail() {
    Reporter.log("[[PROPERTY|author=Adrian]]\n";
    Reporter.log("[[PROPERTY|language=english]]\n";
    Reporter.log("[[PROPERTY|url:github=https://github.com/project/code.java]]\n");
    
    // Testmo also supports automation steps, including statuses & 
    // sub fields. This example uses multi-line properties.
    Reporter.log("[[PROPERTY|step[passed]]]\nThe first step\n[[/PROPERTY]]\n");
    Reporter.log("[[PROPERTY|step[passed]]]\nThe second step\n[[/PROPERTY]]\n");
    Reporter.log("[[PROPERTY|step[failed]]]\nThe third step\n[[/PROPERTY]]\n");

    // You can also include attachment links for test artifacts
    Reporter.log("[[PROPERTY|attachment=https://ci-server/browser.log]]\n");
    Reporter.log("[[PROPERTY|attachment=https://ci-server/screenshot.png]]\n");

    // [..]
  }
}

You can learn more about fields and types supported by Testmo.

More resources & references

Now that you are familiar with submitting your TestNG test results to Testmo, you might also find the following additional examples, topics, and references useful for more advanced workflows.

Was this article helpful?