Executing external test scripts from Matrix

Intro

There is a way to automate the calls to external scripts. The general idea is that the tests steps to automate will have a text starting with run:, followed by the name of a script to launch on the client side, and optional arguments.

Overview

Step 1: Define a test case calling your script (mytest.bat)

Step 2: Create an XTC from that

The script will have some extra columns

Step 3: Execute any manual step before the call to your script

Step 4: locally run the test runner script (TestRunner.jar)

The test runner script will

take the XTC id(s) as input and find all the run:command actions.
When ever it finds one it will execute the command
After executing the command it will write the results back into the XTC

Find information below how to use the TestRunner.jar and where to get it.

Using TestRunner.jar

The user set up some test steps with the syntax run: followed by the name of a script to run.
Example in Linux:
CODE
```
run: /home/scripts/script1.sh x y z 
```
Example in Windows:
CODE
```
run: d:\scripts\script1.cmd x y z
```
The user generates a test run containing some of the above. Let's say the test run has tests number from XTC-50 yo XTC-60
The user calls the TestRunner program to create the test file (big list of all the tests to run):
CODE
```
java -jar TestRunner.jar --grab --project=PROJ --category=XTC --start=50 --end=60
```
this will generate a file called test.xml on the local directory, with the test steps detected in items XTC-50 to XTC-60.
The actual run of the tests can now take place. The user enters the following command:
CODE
```
java -jar TestRunner.jar --run --project=PROJ
```
This will read the test.xml file and call each script. The script must generate on return a response file called response.xml (see syntax below).
The TestRunner program will decode the response and send the ok or error code to MatrixALM, including some comments.
It is possible to stop the whole execution at a particular step if a pre-condition is not met by specifying continue=no in the response file

TestRunner syntax

CODE

java -jar TestRunner.jar [--help][--grab|--run][--project=X][--category=X][--start=n][--end=n][--item=XTC-n][--config=configfile][--test=testfile]
  --help: displays this help
  --grab: generates a configuration file from a (set of) test execution items
  --run: read the test execution items from the testfile and run the tests
  --project=P: mandatory
  --category=X: if specified, changes the default category (XTC)
  --start=n --end=n: if both are present, the test will grab all XTC items from XTC-start to XTC-end
  --item=XTC-n: will grab a single item instead of a start-end
  --config=configile: the name of the config file to use (defaults to config.xml on the local folder)
  --test=testfile: the name of the test file to generate (defaults to test.xml on the local folder)

Syntax of the XML response we are expecting in the reponse.xml file

CODE

<execution result='ok' user='xyz' date='2015/10/25' continue='yes|no'>
   <details>(text for details)</details>
   <global>(optional global test result)</global>
</execution>

Syntax of the configuration file

In order to avoid having to pass the same identification data to the test line all the time (and for security) the credentials have to be stored in a configuration file called config.xml by default:

CODE

<runner>
    <instance>https://mycompany.matrixreq.com</instance>
    <user>mylogin</user>
    <password>mypassword</password>
</runner>

Getting TestRunner.jar

Please ask us to get the TestRunner.jar and the requirements for Java environment needed to run it.