Fields for test automation results are useful to include additional meta info, references, and properties for your tests.
Testmo also supports multiple ways to include and submit fields, properties, and references for automated tests. Testmo reads JUnit-style XML files to submit results with any automation tool (see many examples). We support various extensions for this format to include additional properties on a test case level.
Our CLI tool (which you use to submit test results to Testmo) then reads and submits these fields together with your test results. This topic explains how to include fields for your tests and the various field types and conventions Testmo supports.
Submitting fields for runs and threads
Also see our separate topic on submitting fields for runs and threads.
Include fields as properties
If you are using a test automation framework (or reporter module) that can output properties for test cases in the resulting XML files (properties are an extension feature not yet supported by all tools), then this is the easiest way to include additional fields. The resulting XML file should include properties like this:
<testcase name="testCase3" classname="Tests.Registration" time="3.441">
<properties>
<property name="priority" value="high" />
<property name="language" value="english" />
<property name="author" value="Adrian" />
<property name="attachment" value="https://ci/files/22-dashboard.png" />
<property name="attachment" value="https://ci/files/23-users.png" />
<!-- Optional support for properties with text values -->
<property name="description"><![CDATA[
This text describes the purpose of this test case and provides
an overview of what the test does and how it works.
]]></property>
</properties>
<!-- Optional output or error/failure/skipped -->
</testcase>Testmo supports various special field names (for steps and attachments) as well as various additional field types (such as links or HTML). See the additional sections below to learn more.
Some frameworks and their APIs do not support outputting multiple properties with the same name; in this case you would just append a unique index suffix like this:
<properties>
<property name="priority" value="high" />
<property name="language" value="english" />
<property name="author" value="Adrian" />
<property name="attachment1" value="https://ci/files/22-dashboard.png" />
<property name="attachment2" value="https://ci/files/23-users.png" />
<property name="attachment3" value="https://ci/files/24-error.png" />
</properties>
Include fields in outputs
Alternatively, if your test automation tool doesn't support properties on a test case level yet, our Testmo CLI tool can detect and read properties included in the system-out or system-err sections of the XML file. This means that if your automation tool supports capturing the console output of your tests (and many do), you can simply write properties to standard out using the following syntax:
<testcase name="testCase3" classname="Tests.Registration" time="3.441">
<system-out><![CDATA[
Output line #1
Output line #2
[[PROPERTY|author=Adrian]]
[[PROPERTY|language=english]]
[[PROPERTY|browser-log]]
Log line #1
Log line #2
Log line #3
[[/PROPERTY]]
]]></system-out>
</testcase>Testmo can detect and read single line and multi line properties from the output fields. Additionally, Testmo understands the output attachment syntax used by various tools.
[[PROPERTY|author=Adrian]]
[[PROPERTY|language=english]]
[[PROPERTY|browser-log]]
Log line #1
Log line #2
Log line #3
[[/PROPERTY]]
[[ATTACHMENT|https://ci/files/22-dashboard.png]]
[[ATTACHMENT|https://ci/files/23-users.png]]Each property is listed on its own line without any additional text. Each property consists of a name / value pair. Optionally, multi-line properties just specify the name without a value in the opening tag and end with a closing tag on its own line.
Supported fields & types
Testmo supports various field types for automated test properties, which can be useful so Testmo can nicely display various values. For example, you can tell Testmo that a certain field includes an URL, so Testmo can display it as a link. Likewise, you can ask Testmo to render values as console output, or even render HTML for more complex data (including source code highlighting, tables etc.).
To tell Testmo which type a certain field uses, you use so called type hints as part of the field names. Simply prepend the type and a colon before the field name. Testmo currently supports the following types:
-
string- The default for single line fields -
text- The default for multi-line fields -
url- Renders a clickable link -
console- Renders as monospaced console/output text -
html- Renders the value as HTML (reference)
For HTML, Testmo supports a subset of HTML tags and attributes. Please refer to our full HTML reference, including examples on how to render tables, source code, lists, etc.
Here are some examples of how to include type hints with your fields:
You can add type hints in front of the property names. The type is removed from the resulting names in Testmo.
<properties>
<property name="string:priority" value="high" />
<property name="url:github" value="https://github.com/project/code.js" />
<property name="text:description" value="Long text field.." />
<property name="console:log"><![CDATA[
Log line #1
Log line #2
Log line #3
]]></property>
<property name="html:richtext"><![CDATA[
<h1>Title</h1>
<p>Text</p>
]]></property>
</properties>Type hints work exactly the same with properties included in outputs. Simply add the type in front of the field name.
[[PROPERTY|string:priority=high]]
[[PROPERTY|url:github=https://github.com/project/code.js]]
[[PROPERTY|text:description=Long text field..]]
[[PROPERTY|console:log]]
Log line #1
Log line #2
Log line #3
[[/PROPERTY]]
[[PROPERTY|html:richtext]]
<h1>Title</h1>
<p>Text</p>
[[/PROPERTY]]
Steps & attachments
Testmo also supports & understands a couple of special field names, specifically to support steps and attachment/artifact links for test automation.
Test steps
Name: step*
You can optionally include test steps as properties to render all the steps an automated test performs. Testmo optionally also supports a status per step as well as sub fields.
Testmo expects the fields to be named step or, if your tool doesn't support multiple properties with the same name, you can add a unique index like step1, step2 etc. See below for examples on the syntax to include steps and optionally statuses and sub fields.
Simply name the fields step (or use unique indexes such as step1, step2 ..) and optionally include a status in square brackets. You can use all status automation aliases as configured in Testmo under Admin Statuses.
<properties>
<!-- Either use multiple properties named "step" -->
<property name="step" value="This is the first step." />
<property name="step" value="And this is the second step." />
<!-- Or append a unique index if your test framework requires this -->
<property name="step1" value="Testmo will automatically index the steps." />
<property name="step2" value="And this is another test step." />
<!-- Some tools support multi-line properties. Steps always support HTML -->
<property name="step"><![CDATA[
This is a more complex test step with <strong>multiple lines</strong>
of text using a property with a text value.
]]></property>
<!-- You can optionally also include a step result status as part
of the property name in square brackets. All status automation
aliases as configured in Testmo under Admin > Statuses work. -->
<property name="step[passed]" value="This is a test step that passed." />
<property name="step[failure]" value="And this step had a failure." />
<property name="step[skipped]" value="This step wasn't executed." />
<property name="step[error]" value="And this step had an error." />
<!-- Testmo also supports step sub fields using the following special tags. -->
<property name="step[passed]"><![CDATA[
<test-step-subfield name="Step">
This line describes this step. You can also use <strong>HTML</strong>.
</test-step-subfield>
<test-step-subfield name="Expected">
You can optionally include more sub fields, e.g. for the expected results.
</test-step-subfield>
]]></property>
</properties>
You can also include steps via test outputs. The step names and values work exactly the same. You can also use all status automation aliases as configured in Testmo under Admin Statuses.
# Either use multiple properties named "step"
[[PROPERTY|step=This is the first step.]]
[[PROPERTY|step=And this is the second step.]]
# Multi-line properties also work. Steps always support HTML
[[PROPERTY|step]]
This is a more complex test step with <strong>multiple lines</strong>
of text using a property with a text value.
[[/PROPERTY]]
# You can optionally also include a step result status as part
# of the property name in square brackets. All status automation
# aliases as configured in Testmo under Admin > Statuses work.
[[PROPERTY|step[passed]=This is a test step that passed.]]
[[PROPERTY|step[failure]=And this step had a failure.]]
[[PROPERTY|step[skipped]=This step wasn't executed.]]
[[PROPERTY|step[error]=And this step had an error.]]
# Testmo also supports step sub fields using the following special tags.
[[PROPERTY|step[passed]]]
<test-step-subfield name="Step">
This line describes this step. You can also use <strong>HTML</strong>.
</test-step-subfield>
<test-step-subfield name="Expected">
You can optionally include more sub fields, e.g. for the expected results.
</test-step-subfield>
[[/PROPERTY]]
Attachments
Name: attachment*
You can also include links to external attachments and Testmo will automatically render these as artifact links.
Testmo expects the fields to be named attachment or, if your tool doesn't support multiple properties with the same name, you can add a unique index like attachment1, attachment2 etc. Please note: Attachments need to be URLs / links to externally hosted files, e.g. in your CI pipeline service. In the future Testmo might also support automatically uploading local attachments.
Simply name the fields attachment (or use unique indexes such as attachment1, attachment2 ..). Attachments need to be URLs / links to externally hosted files.
<properties>
<!-- Either use multiple properties named "attachment" -->
<property name="attachment" value="https://ci/files/22-dashboard.png" />
<property name="attachment" value="https://ci/files/23-users.png" />
<!-- Or append a unique index if your test framework requires this -->
<property name="attachment1" value="https://ci/files/24-sessions.png" />
<property name="attachment2" value="https://ci/files/25-failure.png" />
</properties>
You can also include attachments via test outputs. Testmo also supports the alternative syntax with ATTACHMENT entries as supported by many tools.
# Either use multiple properties named "attachment"
[[PROPERTY|attachment=https://ci/files/22-dashboard.png]]
[[PROPERTY|attachment=https://ci/files/23-users.png]]
# Or append a unique index if your test framework requires this
[[PROPERTY|attachment1=https://ci/files/24-sessions.png]]
[[PROPERTY|attachment2=https://ci/files/25-failure.png]]
# Or use the common attachment syntax used by many tools
[[ATTACHMENT|https://ci/files/25-admin.png]]
[[ATTACHMENT|https://ci/files/26-login.png]]
Tool examples & guides
We try to make it as easy as possible to include additional test fields. Please see our guides on using test automation with Testmo and many test automation frameworks and tools:
If your tool doesn't support test case properties or console output yet, you can also add fields & artifacts to test runs and threads, which always works.