Test Configuration

Configuration Setting Overview

The configuration is done using a JSON object. Please note the text must use double quotes (") around all strings. The following parameters can be set:

xtcType: this should always be "XTC", unless a custom project template was created by Matrix Requirements staff
cloneSources: this should always be ["TC", "UC"], unless a custom project template was created by Matrix Requirements staff
presetFields: if set the "create test forms" dialog will ask for default values of these to set them in created forms. The available options depend on the project setup. The default are version, tester and executionDate.
render: [] an array defining how test tables are rendered in each category (see below)
defaultTestResultResult: this is the initial value a an executed test case should have when created. Must be a 'code' from the values in 'automatic' or 'manual' section below.
automatic: if the overall test result should be generated from the individual test step results, this can be configured here. It is an array of items defining rules. The first rule which applies determines the status of the test result.

human: a human readable text explaining the test result
code: a unique identifier for the status of the test result
render: whether test result means 'ok' (all good), 'error' (test failed), 'warning' (e.g. not completed): this is used in report renderings
rule: the type of rule to determine from the test steps the overall status. Can be 'one', 'all' or ''. The first rule matching determines the status
param: parameter for the rule, i.e. a code of a test step

manual: These allow the user to set the overall test result manually.

human: a human readable text used in reports explaining the test result
code: a unique identifier for the status of the test result
render: whether test result means 'ok' (all good), 'error' (test failed), 'warning' (e.g. not completed): this is used in report renderings
command: the string in the drop down menu

perStep: Possible test results per test step.

human: a human readable text used in reports explaining the test result
code: a unique identifier for the status of the test result
render: whether test result means 'ok' (all good), 'error' (test failed), 'warning' (e.g. not completed): this is used in report renderings
command: the string in the drop down menu
key: the keyboard shortcut
image: so far only internally used to show icons for common states (passed, fail and docu)

Changing a test configuration:

Most of the changes can be done in the administration client's main UI

In the last setting, there is the possibility to autofill the tester name following three different options:

no, keep it a user choice : it won't be filled in automatically
set the first user as tester: the first user who save the XTC will be automatically recorded, then it won't change automatically
set the last user as tester: the last user who's saving the XTC will be automatically recorded each time a save is made

Note: for this option to work you need to disable the flag "requires content" in case it is set for the tester field

Video:

In the following FAQ, see "how to change the overall test results options".

Changing Test Table Columns

To change the columns of test tables, you need to click on the the advanced button the admin page (1).

When the script editor opens you can click on format (compact) (2) to more easily edit the columns (3).

The render object has one table definition for each category which contains tests, e.g. UC, TC and XTC.

Each definition is an object which defines the columns, e.g. "UC": {"columns": []} would define a table without columns. To add columns you need to specify for each column

a name (shown in user interface)
a field (an identifier under which the values are stored)
an editor (which specifies how data is entered, or none for readonly cells). See Table Configuration for a list of editors.

Note: When generating test forms from tests (e.g. an XTC from a TC) values are copied from the TC if the field identifiers are identical in both tables. So in the example above the column Action (with id action) from the TC is copied into the column Action Result (because it has the same id action).

Example Configurations

Example Configuration

JS

{
    "xtcType": "XTC",
    "cloneSources": ["TC", "UC"],
    "presetFields":[ 
	  {"field":"name", "value":"Version"},
      {"field":"name", "value":"Tester" } ],
    "render": {
      "UC": {
       "columns":[
         {"name": "Action UC", "field": "action", "editor": "text"  },
         {"name": "Expected Result UC", "field": "expected", "editor": "text"  }]
      },
      "TC":{
        "columns":[
          {"name": "Action 1 TC", "field": "action","editor": "text"  },
          {"name": "Action 2 TC", "field": "action2", "editor": "text" },
          {"name": "Expected Result", "field": "expected", "editor": "text"  }]
      },
      "XTC":{ 
        "columns":[
          {"name": "Action", "field": "action","editor": "none" },
          {"name": "Action Alt","field": "action2","editor": "text" },
          {"name": "Expected","field": "expected","editor": "none" },
          {"name": "Passed/Failed","field": "result","editor": "result" },
          {"name": "Comment","field": "comment","editor": "text"}]
      }
    },
    "defaultTestResultResult":"an",
    "automatic": [
        {"human": "passed", "code": "ap", "render": "ok", "rule": "all", "param": "p"},
        {"human": "failed", "code": "af", "render": "error", "rule": "one", "param": "f"},
        {"human": "not started", "code": "an", "render": "warning", "rule": "all", "param": ""},
        {"human": "documentation", "code": "ad", "render": "warning", "rule": "one", "param": "d"},
        {"human": "in progress", "code": "ai", "render": "warning", "rule": "", "param": ""}
    ],
    "manual": [
        {"human": "passed", "command": "passed", "render": "ok", "code": "p"},
        {"human": "failed", "command": "failed", "render": "error", "code": "f"},
        {"human": "", "command": "not executed", "render": "warning", "code": "r"},
        {"human": "to be decided", "command": "to be decided", "render": "warning", "code": "tbd"},
        {"human": "documentation", "command": "documentation", "render": "warning", "code": "d"},
        {"human": "in progress", "command": "in progress", "render": "warning", "code": "i"}
    ],
    "perStep": [
        {"human": "passed", "command": "passed", "render": "ok", "code": "p", "key": "p", "image": "success.png"},
        {"human": "failed", "command": "failed", "render": "error", "code": "f", "key": "f", "image": "fail.png"},
        {"human": "documentation", "command": "documentation", "render": "warning", "code": "d", "key": "d", "image": "docu.png"},
        {"human": "", "command": "not executed", "render": "warning", "code": "", "key": "r", "image": ""}
    ]
}

This example

adds a column to TC where users can select and design item
another column to XTC where user can select a FILE item

example with refs as columns

CODE

// 1) in the traceability of XTC add downrule, e.g. to a NEW category FILE (which you need to create....)
{
  "message": "files", "name": "can have file", "rule":"can_have", "any_of": [ "FILE" ] 
} 
// modify the xtc_config
{
  "xtcType": "XTC",
  "cloneSources": ["TC", "UC"],
  "presetFields":[  {    "field":"meaning",     "value":"Version"   },   {      "field":"name",     "value":"Tester"     } ],
  "render": {
    "UC": {
      "columns":[    
         {"name": "Action",          "field": "action",          "editor": "text"          },
         {"name": "Expected",          "field": "expected",          "editor": "text"          }]    },
    "TC":{
      "columns":[
        {
          "name": "Links",
          "field": "links",
          "editor": "design"  
        },
        {
          "name": "Action",
          "field": "action",
          "editor": "text" 
        },
        {
          "name": "Expected",
          "field": "expected",
          "editor": "text"  
        }]
    },
    "XTC":{
      "columns":[
        {
          "name": " Links",
          "field": "dref",
          "editor": "none" 
        },
        {
          "name": "Action",
          "field": "action",
          "editor": "none" 
        },
        {
          "name": "Expected",
          "field": "expected",
          "editor": "none" 
        },
        {
          "name": "Passed/Failed",
          "field": "result",
          "editor": "result" 
        },
        {
          "name": "Test Results",
          "field": "files",
          "editor": "downrules"  
        },
        {
          "name": "Comment",
          "field": "comment",
          "editor": "text"
        }]
    }
  },
  "defaultTestResultResult":"an|warning|not started",
  "automatic": [
    {
      "human": "passed",
      "code": "ap",
      "render": "ok",
      "rule": "all",
      "param": "p"
    },
    {
      "human": "failed",
      "code": "af",
      "render": "error",
      "rule": "one",
      "param": "f"
    },
    {
      "human": "not started",
      "code": "an",
      "render": "warning",
      "rule": "all",
      "param": ""
    },
    {
      "human": "documentation",
      "code": "ad",
      "render": "warning",
      "rule": "one",
      "param": "d"
    },
    {
      "human": "in progress",
      "code": "ai",
      "render": "warning",
      "rule": "",
      "param": ""
    }
  ],
  "manual": [
    {
      "human": "passed",
      "command": "passed",
      "render": "ok",
      "code": "p"
    },
    {
      "human": "passed with deviation",
      "command": "passed with deviation",
      "render": "ok",
      "code": "pf"
    },
    {
      "human": "failed",
      "command": "failed",
      "render": "error",
      "code": "f"
    },
    {
      "human": "",
      "command": "not executed",
      "render": "warning",
      "code": "r"
    },
    {
      "human": "to be decided",
      "command": "to be decided",
      "render": "warning",
      "code": "tbd"
    },
    {
      "human": "documentation",
      "command": "documentation",
      "render": "warning",
      "code": "d"
    },
    {
      "human": "in progress",
      "command": "in progress",
      "render": "warning",
      "code": "i"
    }
  ],
  "perStep": [
    {
      "human": "passed",
      "command": "passed",
      "render": "ok",
      "code": "p",
      "key": "p",
      "image": "success.png"
    },
    {
      "human": "failed",
      "command": "failed",
      "render": "error",
      "code": "f",
      "key": "f",
      "image": "fail.png"
    },
    {
      "human": "documentation",
      "command": "documentation",
      "render": "warning",
      "code": "d",
      "key": "d",
      "image": "docu.png"
    },
    {
      "human": "",
      "command": "not executed",
      "render": "warning",
      "code": "",
      "key": "r",
      "image": ""
    }
  ]
}