Search in Guide:

Type a word/phrase above and press enter to search the page.

Table of Contents

User Guide

for MaxTAF Cloud


Getting Started with MaxTAF

Getting started with MaxTAF Cloud has been designed to be as smooth as possible.

All you need is:

To create an account, go to beta.maxtaf.com.

From there you will be presented with a sign in screen.

Click the ‘register’ tab and enter your details.

Currently there are no further steps and you can start using MaxTAF immediately

The first page you see after logging into MaxTAF is the projects screen.

Projects are essentially unique sets of data, cases, suites and settings. They are useful for partitioning test data, and as the name suggests, should be used on a project to project basis.

All users have access to the community project, “Examples”. If you wish to contribute to this, you may do so, but for testing your own system with your own cases, it is necessary to create your own project.

Creating your First Project

  1. Click the ‘+’ button, to create a new project.

Once a project is created, the next step is entirely up to you, however most users would start out by creating their first test case using the button on the home page

If you already have tests, libraries, Selenium IDE recordings or anything else from an existing set-up you wish to import, follow the relevant instructions here:

Importing Cases or Suites
Adding Libraries
Importing Selenium IDE Recordings

Creating a case

To create a case in MaxTAF Cloud there are 2 ways of going about it, the first way (Using the Cases page) is the easiest:

  1. Navigate to the Cases page using the side navigation menu
  2. Click “Create”
  3. Enter the name, type (language), testing framework and finally the Basic UI template.
  4. Finally, click "Create" to confirm your selection.

You have now created your first test case. This case will navigate to Google, type ‘Hello World’ and then assert the page title is as expected.

You have now created your first test case. As the "Basic UI" Template was used, the test will come with a pre-made script that, when run, will: navigate to Google, type ‘Hello World’ and then assert the page title is as expected.

To find out more about the second way of creating cases (using the workspace), look here.

Note: For your learning purposes, we would recommend that you use one of our templates - templates are user configurable ‘pre-made’ test cases. You can read more about them here.


To run the test case you created, press ‘Run’, and then press ‘Run’ again in the opened dialog.

This will be a very quick test case, so the MaxTAF live preview function will not be able to load in time to see it.

Once completed, a report of the run with key information such as run-time and pass/fail will be produced, and you should be taken straight there.

In the report, a tab named ‘video’ should appear automatically. This tab, as you would expect, contains a video recording of the test case so you can see what happened when the test ran.


All aspects of a test can be altered by accessing the test case from the Cases page. One such aspect is the script for the test, it can be edited by doing the following:

  1. Navigate to the Cases page via the side menu.
  2. Select the test case you wish to edit from the list that is shown to you.
  3. From the following screen, select the tab titled “Script”.

You should now be able to see the java class for the test case. You can edit, compile and even run the test case from this tab.

To get started with using this script tab, try adding the following example lines of code:

    // assert that the search text box on the result page contains "Hello World"

    String searchString = driver.findElement(By.name("q")).getAttribute("value");

    Assert.assertEquals(searchString, "Hello World");

You can now save and compile this file using the respective buttons at the top.

Running the test will automatically save and compile the code, just as compiling automatically saves the code beforehand.

Now you have edited your first test case.

The MaxTAF editor works in the same way as any other IDE editor (in fact it’s based on VSCode’s editor, Monaco) so you don’t need to worry about missing any desired IDE functionality. You can simply create cases as you are used to.


Not only can you create new tests from scratch, but you can also duplicate existing test cases, to edit or run as you please.

To duplicate your case:

  1. Navigate to the Cases page via the side navigation menu
  2. Select the test case from the test list shown to you
  3. Press duplicate in the following screen and confirm
  4. Now an identical test case will present in the test list with “ 1” appended to its name

Do this 3 or 4 times for the next stage of this guide.


Next we will take a look at suites, the way to group up tests for batch runs.


Suites are groups of tests (they can also contain suites, a useful feature). Typically, they are used to organise your cases in a logical manner, such tests of a certain type (e.g., UI/API tests) or tests for a certain area of your application.

Grouping tests in this way is critical for making the most of MaxTAF Allure reports.

To create a suite you need multiple tests.


To create a test suite you must:

  1. Navigate to the Suites Page using the side navigation menu.
  2. Click the “Create” button at the top of the suite list.
  3. Enter the suite name in the following dialog and confirm.
  4. From the following screen, click “add items” to start adding tests/suites to this containing suite.

If you run this suite now, it will run all the cases and suites within. It will also produce a report showing the success/failure of each inner test.


Running a suite is identical to running a test case, you must:

  1. Navigate to the suite page from the list of suites accessed previously.
  2. Click the “run” button and then click “run” in the following dialog to confirm.

The sequence in which the contained tests/suites are run can be edited by changing the sequence number within the suite’s page. If the sequence number is the same, the tests will be run in parallel.

Tests can also be disabled in the same list, this allows you to keep a test in the correct suite, but not run it when the suite is run. This allows you to, for example, temporarily stop a flaky test from corrupting the results of the run until you fix it.

Running a Suite will automatically produce an Allure report on completion.


Allure is a powerful open source test reporting framework. To find out more about it’s intricacies, editability and other details please visit the documentation for Allure here.

Finding the allure report is simple, if you have just run a suite you will be redirected to a run results page automatically. At the top of this page are two tabs, the second tab (titled “Allure Report”) will take you to the allure report.

To find the run results for a previous run you:

  1. Navigate to Runs > Completed in the side navigation menu
  2. From the list of runs presented to you, click the one you want to see the results for.

You will now be directed to the same run result screen as the one you were redirected to when you first ran this test.


Schedules can be created by doing the following:

  1. Navigate to the Schedule page using the side navigation menu.
  2. Click “Create” to open the schedule creation dialog.
  3. Within this dialog, enter the name for the schedule and choose the suite or test you wish to be scheduled.
  4. Click “Create” to confirm.

From the schedule’s page you can edit details such as: the name, description, and the timing of the schedule by clicking the pencil button next to each.

You can also edit parameters at the bottom of the page to provide the schedule with some special functions. We will cover these parameters later in the documentation.


This was a basic guide to using MaxTAF Cloud, covering the essentials needed to understand the platform.

Areas that have not been covered, but will become more essential as you progress through your MaxTAF development journey, are:

  1. Parameters
  2. Data Driven Testing
  3. Workspace

Developing Test Cases

With in a Case Page one of the tabs available takes you to the script editor in which you can edit the script for a test case. Functionality available in the script editor includes:

Every Major IDE has a code completion feature that is very useful when quickly developing test cases. The autocomplete feature should work out of the box as long as the language server status is enabled (green circle on the bottom right of the editor, this will be red if the language server is down).


In order to see the code completion in action, start typing code and you should get suggestions as you type.


Code Lookup is an innovative feature that allows users to have an example script opened alongside a script in development.

This is especially useful for referencing and code-reusage, a critical part of efficient Software Development In Testing.


Where to find it

The Code Lookup button is found on the toolbar within the script tab for a test case.


Pressing the Code Look-up button will allow you to use find and open up any other script, from any of the projects you are a part of, to reference. The script from the code look-up will be presented in a separate script box next to the script for the case you are editing.

Code Snippets can be copied and pasted from the Code Lookup view over to your main script.


Failure line jump is a useful feature when writing test cases, it allows you to jump to and highlight the exact line of code which is causing an error for your test run. To do so, you just need to click on the line number in the error message as shown below.


As you type code for a test case, you are prone to making visual mistakes (like indentation, new lines, etc.). Language server has a great tool which you can use to format your code. This is particularly useful when you want to write well formatted code quickly.

To format your code, either right click somewhere inside the code editor then click on “Format Document”, or use the shortcut “Shift + Alt + F”.


MaxTAF Cloud currently supports 4 different programming languages:

  1. MXML
  2. Java
  3. Python
  4. Javascript
  5. C# (work in progress)

When you are creating a case, you can choose a language in which your code will be written.


MXML is MaxTAF Cloud’s custom language that uses XML tags and makes it easier for non-developers to write code. MXML transpiles into Java code which is then compiled. You can read in depth about MXML in the MXML section of the developers guide.

Java is a standard language for writing test cases. You can also use custom .jar or third party libraries inside your test case.

Python is minimal and doesn’t have the need for compilation. It also supports custom or third party libraries.

Javascript is somewhere between Java and Python and also doesn’t need to be compiled.


Using the Workspace

Workspace gives you access to your project's file structure. From the workspace you can:

You can access the Workspace from the left navigation menu by clicking on the Workspace menu option :


Warning: As with any other file managing system, be extra careful when deleting files as you may cause irreparable damage.

The workspace is organized in two panes:

MaxTAF files are organized in a typical hierarchical manner. Within the project root folder there is a dedicated directory for each of the supported languages.

To view files from other projects that you have access to, press the "Other Projects" tab in the side menu of the File Tree pane. That will open a new file tree in a split view below the main project file tree and from there you can navigate to and open any of the files in other projects.

Note: you may view files from other projects and copy them into the current project but you cannot edit them. In order to edit them you must switch to their project.

Depending on the type of the file, you may have some type-specific actions. For example, a .java file can be compiled and, if it is also the script file for a test, the test can be run from here. Here is a list of some typical actions:

Save Saves your edits.
Properties Shows properties of the file.
Download Downloads the file.
Compile Compiles the file.
Go to case Switches to the Case view.
Refresh Refreshes the editor.
Run Runs the case.
Params Reveals the parameters of the case.
Transform To Case Creates a case from the file.

Each of MaxTAF supported languages has it's own file structure within the Workspace.

Within these folder structures, each language will have a place where you can add any of the external libraries and modules that you need for your cases and files. To add a library or a module, simply find it on the internet, download it to your computer and from there upload it to the dedicated library/modules folder.

Java related files are organized on the following folders:

Each C# test case has a dedicated folder within the CSharp root folder that shares its name with the test case itself. Within this case folder the structure looks like this:

Robot Framework is different to other languages in terms that it sits on top of Python as a layer. For that reason, addition of new libraries boils down to adding new libraries into the Python file structure.

MaxTAF Cloud fully supports a libraries. You can also use custom or third party libraries (for example from https://mvnrepository.com).

In order to use libraries do the following:

  1. Go to “Workspace”
  2. Depending on your java_ext_libs_path project parameter, you should navigate to it’s value in the folder tree (or by default /java/libs)
  3. Once you navigate to Java libraries folder, upload the .jar by right clicking on the folder and selecting Upload


  4. Once the .jar is uploaded, you can use it inside your test case code for compilation and autocompletion support


Earlier it was mentioned that there are two ways of creating a test case, here we show how to create a test case from the workspace:

  1. Navigate to the Workspace page using the side navigation menu.
  2. Find the point in the file system you wish to create your test in.
  3. Right-click the folder in which you want your test to be located.
  4. Click “Add File”, enter the file name with its file extension (e.g. “.java”) and confirm.
  5. Populate the File with the code you need for the test and compile.
  6. At the top of the script editor click “transform to case” (Clipboard icon with an upwards arrow on it).
  7. Select the “Params” button (Icon with sliders on it) and enter the necessary parameters to run the test (Can copy these from a test created using the Basic UI template).

While being more complicated, this method of creating cases is currently the only relatively simple way of creating tests in specific locations within the file system.

The Terminal

Each project in MaxTAF has its own engine. An engine in this context is a linux container that runs the code within the project. The terminal allows the user to directly login to the engine and perform commands directly.

An example of using the terminal in this way would be if you wanted to commit your project code to git. This is currently the only way the user can use git with MaxTAF making the terminal a very useful tool for development teams.

Note: Users must be careful when using the terminal as it has the potential to cause irreversible damage to the file system for the project.


Test Management

Cases are MaxTAF's way of presenting test case data to the user, such data includes:

You can change most of this data to suit your testing needs.


To create a test case you must either:

To create an exact duplicate of a test case follow the instructions here.


From each Case's test page the user can view the run log as a list. That is, they are able to see what is in essence an instance of the completed runs list where all the runs except those that spawned from this test case have been filtered out.

The run log also offers the same filtering functionality as in the Completed Runs Page, more information on this can be found here.


Each test case's page has an accessible tab "Schedule" in which they can create schedules for the case in question. From this list they can also export or import schedules as you would be able to with the list in the Schedules Page.

The first page you see when you access the Cases Page via the navigation menu is a list of all the cases you have on your project.


While developing tests you are likely to either accidentally delete the wrong case or delete a case and only later realise you still need it.

MaxTAF will store any deleted cases under "Trash Records", a page accessible from the Cases Page. In order to recover a test case you have to access the test case from said page and click "Restore".


From the Cases Page you can perform actions on multiple test cases at once. This is done by using the checkbox next to each case you wish to use. Once each case is selected the actions available are exporting and deleting as a group.

If you select multiple test cases from the "trash records" tab, you can recover them as a group by clicking restore.


From the cases list you can use the search input at the top to filter the list presented to you. More information on case filtering can be found here.


Suites are batches of tests, used for running them in groups.

They can be used to either run an entire pack of tests, or to batch tests up according to themes, system areas tested, types of test or any other level of refinement required.

Grouping tests in this way allows for more granular reporting, giving testers and developers a better and more precise insight into system health.


Creating a suite is simple.

  1. Head to the Suites section of MaxTAF, which is found in the left-hand menu

  2. Press ‘create’
  3. Enter name, press create
  4. In the items box, click “Add items”
  5. Select the cases you want to add
  6. Click “add” to confirm selection

Your test runs can be sequenced in any configuration or order that you choose.

In the items table, under the sequence column, the number dictates how early in the sequence tests will run.

If you select the same sequence number for different tests, then the tests will run in parallel.


Running a suite follows the same steps as running cases, found here.

The singular difference between running suites and running tests is that you are working with suites not tests. Other than that, you can do everything else identically. This includes:

To duplicate a suite you have to:

  1. Navigate to the Suites Page via the side navigation menu.
  2. Select the suite to be duplicated.
  3. Click "Duplicate" and confirm your action in the following dialog.

Suite pages also have a schedule tab identical to the one found in test pages which can be used to view and create schedules that only include this particular suite.


Suite pages contain a run log that shows every completed run of that suite in the same format as the completed runs list.


To delete a suite you must:

  1. Navigate to the Suites Page via the side navigation menu.
  2. Select the suite to be deleted.
  3. Click "Delete" and confirm your action in the following dialog.

To restore a deleted suite:

  1. Navigate to the Suites Page via the side navigation menu.
  2. Select "Trash Records".
  3. Select the suite to be restored.
  4. Click "Restore" and confirm your action in the following dialog.


From the suites list users can filter in a similar, but less powerful way as is found in the cases list. More information can be found here.


Just as seen in the Cases list, you can select multiple suites in this list for exporting or deleting.


Cases can share information during a suite run. This is especially needed when one case creates some data object that will then be further processed by another case that runs after it.

The mechanism that MaxTAF Cloud uses is similar to a bulletin board, in the sense that the cases that participate in the same suite run can post information to and read information from a jointly accessible area.

  1. To post a piece of information, a case will use ApiService method setSharedRunValue(name, value)
  2. To read posted information, a case will use ApiService method getSharedRunValue(name)

Even for hierarchically structured suites (when a suite contains other suites), the values that are posted to the bulletin board are available to every case in each suite that belongs to the hierarchy.

Note: It is very important to correctly sequence the cases in the suite as otherwise you may end up reading a shared value before it has been set.

Example:

Suite Structure:

 

Sequence No

Case

10

Create_Invoice

20

Approve_Invoice

Java:

Create_Invoice code:

public class Create_Invoice extends Base {
		private ApiService mxService = super.getApiService();
		@Test
		public void test() {
		mxService.setSharedRunValue("invoice_no", "INV123");
		}
	}

Approve_Invoice code:

	public class Approve_Invoice extends Base {
		private ApiService mxService = super.getApiService();
	...
		@Test
		public void test() {
			String invoice_no = mxService.getSharedRunValue("invoice_no");

_____________________________________________________________


C#:

Create_Invoice code:

[AllureNUnit]
public class Create_Invoice : Base
{
	ApiService MxService;
	...
	[Test]
	public void Test1()
	{
	MxService.SetSharedRunValue("invoice_no","INV123");
	}
}

Approve_Invoice code:

[AllureNUnit]
public class Approve_Invoice : Base
{
	ApiService MxService;
	...
	[Test]
	public void Test1()
	{
		string invoice_no = MxService.GetSharedRunValue("invoice_no");
	}
}

_____________________________________________________________


Python:

Create_Invoice code:

class Create_Invoice(Base):

  def setup_class(self):
    self.mxService = Base.getApiService(Base)

  def testBasepy(self):
    self.mxService.setSharedRunValue("invoice_no","Hello from test!")

Approve_Invoice code:
class Approve_Invoice(Base):

  def setup_class(self):
    self.mxService = Base.getApiService(Base)

  def testBasepy(self):
    invoice_no = self.mxService.getSharedRunValue("invoice_no","Hello from test!")

_____________________________________________________________


JavaScript:

Create_Invoice code:

describe("Create_Invoice", function() {
  let mxService

  before(function() {
    mxService = base.getApiService()
  })

  it("posts invoice no", async function() {
    mxService.setSharedRunValue("invoice_no","INV123")
  })
})

Approve_Invoice code:
describe("Approve_Invoice", function() {
  let mxService

  before(function() {
    mxService = base.getApiService()
  })

  it("reads invoice no", async function() {
    invoice_no = await mxService.getSharedRunValue("invoice_no")
  })
})

Suites of Suites

Suites can be also batched up into meta-suites.

To create a suite of suites, follow the same process for cases, but when you enter the ‘add items’ dialogue, there is a tab at the top in which suites can be selected instead of cases.



Suite Parameters

The guide on suite parameters can be found here.


Scheduling Suites

Check out Documentation on Scheduling here.


Templates are used to create test cases with predefined code and parameters. Each test case in MaxTAF Cloud is created from a template if you decide to use the Cases page rather than the workspace to create them. You can choose from many predefined templates in your project to create a test case.

Each template is a JSON object which contains the following:

Templates are stored in your workspace. You can find them by opening Workspace > Templates.


The templates that are stored by default are:

  1. Default – Creates a mostly empty test class with an empty test method, some basic imports and instantiates an APIService object to allow API calls to MaxTAF.
  2. Basic UI – Creates a runnable UI test with some hello world code and an APIService instantiation with some basic API calls to show some of what it can do.
  3. Basic API – Creates a runnable API test which bypasses the UI by making an API call to google maps and checking that the response code is as expected.

Creating Templates

You can create as many templates as you wish by exporting test cases as templates.

First, you must create a test case (or choose an existing one) with the same combination of language and testing framework as you want your template to use. Then you can write code and parameters according to your needs. When your test case is ready to become a template, click the “Export” button and then click the Template option, give your new template a name and description (the description is optional) and click “Export”.

Can I use my templates in other projects?

Yes, you can. Just download the templates you want to use elsewhere and upload them in the corresponding workspace folders for the projects in which you want to use them.

Exporting allows you to download a test case or suite in either XML or JSON format.

To export a case, follow these instructions:

  1. Navigate to the Cases page using the side navigation menu.
  2. Click the test you wish to be exported (Skip this step if you want to export all cases).
  3. Press the export button near the top of the page.
  4. Choose which form you want to export it in.
  5. Select “Export” and the file will immediately start downloading.

If you decide to export a suite, all the contents or said suite will be within the resulting download file, including the test cases and suites within.

Importing allows you to upload a previously downloaded suite or case file to your project, allowing you to use them as you normally would.

To import test files to MaxTAF:

  1. Navigate to the Cases/Suites Page via the side navigation menu.
  2. Click “Import” and drag the file into the dialog (or click "browse" and find the file) and click “upload”.
  3. If successful, you will now be able to close the dialog and start using the class straight away (You will need to add the necessary parameters that are required for the test to run).

Note: Currently importing java classes is not possible if your class inherits from another class, so the file must be at the top of its own inheritance hierarchy.

MaxTAF Cloud allows users to import Selenium IDE recordings as test cases, find out more information on this here.


Filtering is a tool shared by many components of MaxTAF. It allows you to search tables for the item you need, this is used in the following areas:


Any part of the website that shows a list of cases will allow filtering based on the following attributes:


Any part of the website that shows a list of suites will allow filtering on only the suite name and description.


The parts of the website that display a list of completed runs will allow filtering by the following attributes:


The active runs page will only allow filtering based on the name of the case/suite being run and the run ID.


The parameters tab found under the settings page will allow filtering based on the parameter name and value.


The Users tab found under the settings page allow filtering on only the user email address and name.

The User requests tab only allows filtering based on the email address used to invite a user to the project.


Execution

Running a test case requires pressing the ‘Run’ button from either the script tab or the main tab when you are on the page for said case.

You can also Run a test case directly from the workspace - See information here


When you run a case, you will be presented with the option to either run the case, or to run using advanced settings.


Simply press run again if you want to run the test case with default settings.


If you opt to perform a more advanced run, MaxTAF provides a variety of properties to run your test with:

Data Driven Testing is possible with MaxTAF Cloud.


Data Driven Testing is the process whereby parameterized regression scripts are procedurally run with different data scenarios, provided by rows of data in a file.

In simple terms, when you run a data driven test, you run the same script many times, but different variables are fed into the test for each iteration.

The nature of the data is determined by the data file you supply, with the number of rows being equal to the number of runs (data scenarios) and each column referring to a discrete piece of data in the data scenario.


Data Driven Testing requires preparation in three areas:


Parameters:

In the parameters area in the main tab for the test case, configure the parameters so that those prepared in your script can be fed data from your spreadsheet.

Data is contained in a file in the workspace specified by the "mx.datadriven.file" parameter.

Add a new parameter in the "Main" tab called mx.datadriven.file and set its value to the location of the data driven file relative to your workspace, for example:

This parameter will search for the file in the root folder and then in the folder “dd” where it will find the data driven file, (in this example "google_search.csv").

After that, specify parameters with custom names for columns and take the columns from the data driven file like this (where language and min_hits are the custom names in question):

At run time, there will be a separate run for every row in the data file and each time, the test parameters will receive new values from the columns with the previously specified names, e.g.

When there is no column title, the value can be specified by the column's number, e.g.

The first column in the data driven file is treated as header column and it is used to identify the column by name (example: mx.datadriven.name.language)

Then you need to specify if the rows inside data driven file are to be treated and run as parallel test cases:


Script:

Follow the instructions to parameterize your script where you wish the data to be changed, so that you may modify your script parameters in the ‘main’ tab.

All you need to do in your test case script is get the data driven parameter you set previously in the parameters section (via mxService for example)) and use them like you would with any other parameters:

String query = mxService.getParam("language");
long minExpectedHits = Long.parseLong(mxService.getParam("min_hits"));

Spreadsheet:

The spreadsheet you use can be created in either CSV or Excel format.

Each row will correspond to a different data scenario/test run.

Each column will correspond to the different parameters you have defined.

MaxTAF handles parallel test coordination as a basic feature.

Note: You may be limited by your plan on the number of parallel nodes you can deploy. If you exceed the number of nodes you have available, MaxTAF will use the max number available and sequentially run your test cases in batches.


To run an individual case in parallel, usually for basic performance checks, you must:

  1. Navigate to the Cases page via the side navigation menu.
  2. Search for and click on the row for your chosen test case.
  3. Press ‘Run’ from the resulting page.
  4. Select the ‘Advanced’ dropdown menu.Set the ‘Run type’ option to ‘Parallel’.
  5. Select the ‘Performance’ dropdown menu.
  6. Set the ‘Run type’ option to ‘Parallel’.
  7. Change the ‘Repeat run’ value to reflect the number of times you want to run the test.
  8. Click ‘Run’ to confirm.



Tests within suites can be run in parallel by doing the following:

  1. Navigate to the Suites page via the side navigation menu.
  2. Navigate to the suite within which you want to produce a parallel run.
  3. Within the “items” section of the page you will see a sequence number next to each item.
  4. Set this number to be the same for each test which you want to run in parallel.

Now whenever you run this suite, each inner test/suite with the same sequence number will be run in parallel. Setting up suites in this way allows maximum control over the sequencing of your tests as you can have some tests run in parallel while others do not.


If your case is set up for Data Driven Testing See here, follow the above steps but instead of performance select data driven.

You can change the run type to Parallel and your data driven test will run in parallel for as many nodes as there are available.

MaxTAF Cloud gives the user the ability to both see their test runs either live or as a recording once the test is complete.

Live Preview

The live preview functionality is always enabled for test cases that are run using the MaxTAF Cloud interface. However, it needs time to switch on. The test will not be shown live for the few seconds of the run as the preview software needs time to load. This means that if the test finishes execution in those seconds at the start of the test, nothing will be shown to the user.

Live previews can also be seen by clicking the camera button that appears in the rows in the Active Runs Page. This is useful when you run multiple tests at once, such as in suites and multiruns of individual cases.

Using the Browser.Pause() method provided by MaxTAF, in your script you can pause a test midway through and take control of the browser by clicking on the cursor icon at the side of the preview.

Video Recording

In order to see a video recording of a test run once it is complete, you have to add the parameter "mx.selenium.enable_video_recording=true" to the test's parameters.

Once enabled, you can see this recording in the run details once the script has finished executing. Alternatively, you can find the recording by accessing the run via the Completed Runs Page.

MaxTAF Cloud officially supports:

The browser that we recommend as it was tested the most is Google Chrome.

To use a specific browser with a test, do the following:

  1. Navigate to the Cases page via the side navigation menu.
  2. Select the case which you want to use the browser on.
  3. In the “Params” section click on the pencil icon to edit the parameters for that case.
  4. Set the mx.selenium.browser parameter to “internetexplorer”, “edge”, “chrome” or “firefox” just as it is done in the picture below:


By default, if the “mx.selenium.browser” parameter is not set, MaxTAF Cloud will use Google Chrome.

Since selenium nodes from the common/private pools are run on a linux machine, the only browsers you can use with them are Google Chrome and Mozilla Firefox. In order to use Internet Explorer or Microsoft Edge you need to specify your own selenium hub/node that’s run on Microsoft Windows. You can do that by changing the parameter “mx.selenium.server” to your own selenium hub/node address.

Example:

Email Notifications are available for case and suite runs.

See the parameters to set up notifications here.

You can schedule tests in a few ways.

The schedule section, accessed from the side navigation menu, allows you to create schedules for any test or suite.

To create a schedule, you must:

  1. Navigate to the schedule page via the side navigation menu.
  2. Press “Create”.
  3. Give the Schedule a name in the following dialog.
  4. Select the desired Case or Suite.
  5. Press “Create” again to confirm.
  6. Edit details according to your needs (note, to set the timing for the schedule, press the pencil icon next to Schedule under Schedule Data).

Other details that can be edited from this schedule’s page are as follows:

You can also set schedules from the schedules tab inside individual Suites or cases.

These can be administered from the schedule menu as well.


Network and Infrastructure

API Service provides communication between MaxTAF Cloud and test cases. It is required for passing test case parameters to tests, saving screenshots and generating xml reports among other functions. MaxTAF Cloud has an ApiService class for every supported language.

To use the API Service in a Java test, your test class needs to extend the Base class and then instantiate ApiService by calling the getApiService method from the extended Base class.

package com.maxtaf;
import org.junit.Test;
import com.maxtaf.ApiService;
import com.maxtaf.Base;

public class JavaTestClass extends Base {

	private ApiService mxService = super.getApiService();

	@Test
	public void test() {
		// Your test code
	}

}

If you need to instantiate ApiService within static method you will have to call the static method from the Base class like this:

package com.maxtaf;
import com.maxtaf.ApiService;
import com.maxtaf.Base;

public class JavaTestClass extends Base {

	public static void main(String[] args) {
		Base.init(args);
		ApiService mxService = Base.getApiService();
	}
}

To use the API Service in a C# Test, your test class needs to extends the base class and then instantiate ApiService by calling the getApiService method from the extended Base class.

using NUnit.Framework;
using NUnit.Allure.Core;
using MaxTAF;

[AllureNUnit]
public class Csharptest : Base
{
  ApiService MxService;
  [Test]
  public  void test1()
  {
    MxService = base.GetApiService();
  }
}

To use the API Service in a Python test, your test class needs to extend the Base class and then instantiate ApiService by calling the getApiService method from the extended Base class.

import pytest
from maxtafbase import Base

class TestPython(Base):
def setup_class(self):
self.cdlmApi = Base.getApiService(Base)
def testMethod(self):
	# Your test code

if __name__ == '__main__':
 	Base.init(Base)
 	pytest.main([__file__, *Base.args])

To use the API Service in a JavaScript test, you have to import the Base class and then instantiate ApiService by calling the getApiService method from the imported Base class.

const assert = require("assert")
const base = require("../maxtaf/maxtafbase.js")

describe("JavaScriptTest", function() {
 	let mxService

 	before(function() {
mxService = base.getApiService()
})

it("should be test", async function() {
// Your test code
})
})

JAVA

 

Method name

Input parameters

none

Return type

String

Throws

Exception

Description

This method is used to get all test case parameters as a String divided by \n

Example

String myCaseParams = mxService.getAllParams();

 

Method name

Input parameters

paramName: String

Return type

String

Throws

Exception

Description

This method is used to get a specific test case parameter as a String

Example

String myCaseParam = mxService.getParam(“nameOfMyParam”);

 

Method name

Input parameters

none

Return type

List<SystemParam>

Throws

Exception

Description

This method is used to get all system parameters as a list of type SystemParam

Example

List<SystemParam> systemParams = mxService.getAllSystemParams();



Method name

Input parameters

paramName: String

Return type

String

Throws

Exception

Description

This method is used to get a specific system parameter value as a String

Example

String systemParamValue = mxService.getSystemParam(“someSystemParam”);

 

Method name

Input parameters

none

Return type

List<ProjectParam>

Throws

Exception

Description

This method is used to get all project parameters as a list of type ProjectParam

Example

List<ProjectParam> projectParam = mxService.getAllProjectParams();

 



Method name

Input parameters

paramName: String

Return type

String

Throws

Exception

Description

This method is used to get a specific project parameter value as a String

Example

String projectParamValue = mxService.getProjectParam(“someProjectParam”);






Method name

Input parameters

projectParam: ProjectParam

Return type

ProjectParam

Throws

Exception

Description

This method is used to create/update a project parameter (if a project parameter with the same name already exists then that parameter will be updated, otherwise a new project parameter will be created)

Example

ProjectParam myProjectParam = new ProjectParam();

myProjectParam.setName(“projectParamName”);

myProjectParam.setValue(“this is a value”);

myProjectParam.setAutoIncrement(false);

myProjectParam.setDescription(“This is example project param”);


myProjectParam = mxService.setProjectParam(myProjectParam);


// or


mxService.setProjectParam(myProjectParam);

 

Method name

Input parameters

paramName: String

paramValue: String

Return type

ProjectParam

Throws

Exception

Description

This method is used to create/update a project parameter with “autoIncrement=true”(if a project parameter with the same name already exists then that parameter will be updated, otherwise a new project parameter will be created)

Example

ProjectParam myProjectParam = mxService.setProjectParam(“projectParamName”, “projectParamValue”);


//or


mxService.setProjectParam(“projectParamName”, “projectParamValue”);

 

Method name

Input parameters

paramName: String

paramValue: String

autoIncrement: boolean

Return type

ProjectParam

Throws

Exception

Description

This method is used to create/update a project parameter (if a project parameter with same name already exists then that parameter will be updated, otherwise a new project parameter will be created)

Example

ProjectParam myProjectParam = mxService.setProjectParam(“projectParamName”, “projectParamValue”, false);


// or


mxService.setProjectParam(“projectParamName”, “projectParamValue”, false);



Method name

Input parameters

paramName: String

Return type

String

Throws

Exception

Description

This method is used to increment a project parameter and return the incremented value as a String. If the parameter can’t be incremented, the method will return null

Example

String incrementedValue = mxService.incrementProjectParam(“projectParamName”);










Method name

Input parameters

paramName: String

Return type

void

Throws

Exception

Description

This method is used to delete a project parameter

Example

mxService.deleteProjectParam(“projectParamName”);

 

Method name

Input parameters

progress: int

Return type

void

Throws

Exception

Description

This method is used to set the overall run progress represented as a percentage

Example

mxService.setRunProgress(17);

mxService.setRunProgress(50);

mxService.setRunProgress(89);

mxService.setRunProgress(100);

 

Method name

Input parameters

log: String

Return type

void

Throws

Exception

Description

This method is used to append a new String to the run log

Example

mxService.addLog(“This is a log”);








Method name

Input parameters

log: String

Return type

void

Throws

Exception

Description

This method is used to append a String to the run log with the new line character at the start

Example

mxService.addLogLine(“This is a log line”);

 

Method name

Input parameters

title: String

description: String

testClass: String

author: String

date: Date

Return type

void

Throws

Exception

Description

This method is used to create a new run report. This should be done before any test methods are called. The report can be found after the test run in the run results under “Run Reports”

Example

Date date = new Date();

mxService.createReport("Report title", "Report description", "testClass", "Name of author", date);

Result



Method name

Input parameters

status: String

(PASSED or FAILED)

Return type

void

Throws

Exception

Description

This method is used to set run the report pass/fail status. This should be done after every test method has been called

Example

mxService.setReportStatus(“PASSED”);

Result



Method name

Input parameters

name: String

Return type

void

Throws

Exception

Description

This method is used to create a new test report. This should be done once per test method

Example

mxService.addTestReport(“My test report”);

Result



Method name

Input parameters

testReportName: String (must be same as report name you want to add it to)

stepNumber: int

action: String

expectedResult: String

actualResult: String

status: String

Return type

void

Throws

Exception

Description

This method is used to append a test step to an already created test report. This should be done as many times as you need per test method

Example

mxService.addTestReportStep("My test report", 1, "Some action", "63", "78", "FAILED");

mxService.addTestReportStep("My test report", 2, "Another action", "true", "true", "PASSED");



Result

 



Method name

Input parameters

paramName: String

paramValue: String

Return type

void

Throws

Exception

Description

This method is used to create/update a parameter which is shared between all runs in a specific suite or multirun

Example

mxService.setSharedRunValue(“paramName”, “paramValue”);



Method name

Input parameters

paramName: String

Return type

String

Throws

Exception

Description

This method is used to get a shared parameter value as a String

Example

String sharedParamValue = mxService.getSharedRunValue(“paramName”);



Method name

Input parameters

screenshot: byte[]

Return type

void

Throws

Exception

Description

This method is used to save an array of bytes as a screenshot from the Selenium WebDriver

Example

byte[] screenshot = ((TakesScreenshot) driver).getScreenshotAs(OutputType.BYTES);

mxService.saveScreenshot(screenshot);

Result





Method name

Input parameters

caseName: String

Return type

CaseRunResult

Throws

Exception

Description

This method is used to run a test case

Example

mxService.runCase(“My case”);



Method name

Input parameters

caseName: String

runDetails: RunDetails

Return type

CaseRunResult

Throws

Exception

Description

This method is used to run a case with additional run details

Example

RunDetails runDetails = new RunDetails();

runDetails.setSyncAsync(SyncAsync.SYNC);

runDetails.setRepeatRun(5);


mxService.runSuite("My case", runDetails);



Method name

Input parameters

runId: int

Return type

CaseRunResult

Throws

Exception

Description

This method is used to get the run details of a specific run

Example

CaseRunResult runResult = mxService.getCaseRunResult(1563);



Method name

Input parameters

suiteName: String

Return type

SuiteRunResult

Throws

Exception

Description

This method is used to run a suite

Example

mxService.runSuite(“My suite”);



Method name

Input parameters

suiteName: String

runDetails: RunDetails

Return type

SuiteRunResult

Throws

Exception

Description

This method is used to run a suite with additional run details

Example

RunDetails runDetails = new RunDetails();

runDetails.setSyncAsync(SyncAsync.SYNC);

runDetails.setRepeatRun(5);


mxService.runSuite("My suite", runDetails);



Method name

Input parameters

runId: int

Return type

SuiteRunResult

Throws

Exception

Description

This method is used to get the run details of a specific run

Example

SuiteRunResult runResult = mxService.getSuiteRunResult(135);



Method name

Input parameters

runId: int

Return type

void

Throws

Exception

Description

This method is used to stop an active run

Example

mxService.stopRun(1235);

Common pool is a pool of selenium nodes that you can run tests on. By default, it is a shared pool that all MaxTAF Cloud users can use between themselves. When you run a test on a common pool, the test is added to the queue. After checking for availability the test is sent to the first available selenium node, where it is executed. If all the nodes are busy, the test will wait in a queue for execution. After one of the nodes is available again, the test will continue with execution.

In order to use common pool for your case/suite, do the following:

  1. Go to Cases/Suites and choose a case/suite
  2. In the “Params” section click on the pencil icon to edit parameters for that case/suite
  3. Set the mx.selenium.server parameter to ${mx.system.common_pool} as shown in the picture below: mx.selenium.server=${mx.system.common_pool}

You can also use a common pool to run tests in parallel. The maximum number of parallel cases vary by user, but the default is usually 10. MaxTAF Cloud will give you an error if you try to run too many (more than the common pool can handle).

Same goes for parallel cases/suites. If you run 10 tests and 5 nodes are available at that point in time, the first 5 will be run in parallel and the next 5 will wait in queue. When the nodes become available again, they will continue with execution.

Private pool is another cluster of selenium nodes in which you can run your tests. Unlike the common pool, a private pool is not shared by other MaxTAF Cloud users and is exclusive to the project. Nodes from the private pool are created on demand.

When the test is run on a private pool, a node is created. After some time (usually about 30 seconds) when the node is fully functional, the test continues with execution.

In order to use private pool for your case/suite, do the following:

  1. Go to Cases/Suites and choose a case/suite
  2. In the “Params” section click on the pencil icon to edit parameters for that case/suite
  3. Set the mx.selenium.server parameter to ${mx.system.private_pool} just as shown in the picture below: mx.selenium.server=${mx.system.private_pool}

You can also use a private pool to run tests in parallel just like you would with the common pool. After all the tests are finished, the nodes will automatically shut themselves down.

MaxTAF Bridge is a package that lets you execute the Selenium tests on your computer.

MaxTAF Bridge establishes an internet connection between your computer and MaxTAF Cloud server and then when you run one of your cases, you can have them launch the browser on your computer. A typical use-case for this is, if you need to test a site that is on your side of the firewall, and as such is not visible to the Selenium nodes that are hosted by MaxTAF.


To install MaxTAF Bridge do the following:

  1. Download the MaxTAF Bridge package from < >
  2. Unzip the zip file in a directory of your choice
  3. Modify resources/application.properties and enter your MaxTAF.com username, password and the id of the project that holds the test cases that you want to run. The id of the project can be found by going to Settings -> Project via the navigation menu. Note: if, for security reasons, you do not wish to keep login details in this file, you can leave it empty and then supply the login details each time you run the application.

External Software Dependencies:

Independent of MaxTAF bridge installation, you will also need to have on your computer:

  1. Browsers that you want to use for testing
  2. Selenium Server software
  3. Browser drivers

Browsers that you want to use for testing

When running your cases through MaxTAF Bridge, they will be run on browsers that you have on your computer. Therefore you must install all of the different browsers that you plan to use for testing.


To install and run a selenium server on your computer, follow the instructions from https://www.selenium.dev/downloads/ in the section marked “Selenium Server (Grid)”.

Note that Selenium server can be configured to any port number, but will default to 4444 if no specific port value is specified. MaxTAF bridge will establish connection on port 4444 by default. If this value is changed for the Selenium server, then the app.properties file needs to be modified by adding the following line:


A Selenium Server requires browsers drivers to run tests in the specific browsers you intend to use. These drivers are separate pieces of software that you need to install. Follow the instructions from the “Browsers” section of the https://www.selenium.dev/downloads/ web page. There you will find links to specific drivers (eg, chrome, firefox etc). Follow these links to get detailed instructions for download and installation.

Note: As browser versions change, it is likely that their drivers will change too. Therefore make sure that your drivers are kept up to date.


Preconditions:

Before you start MaxTAF Bridge, make sure that you have first started your Selenium Server (following instructions from selenium.dev website) and that your browser drivers are correctly installed and configured according to the manufacturer's instructions (e.g. that they are visible to the Selenium Server).

A correctly started Selenium Server should look something like this:

To start MaxTAF Bridge, open a terminal (command line interface) and then navigate go to the root directory of the MaxTAF Bridge package. From there enter the following:

For Windows, enter start-bridge.bat

For Linux, enter start-bridge.sh

If started correctly, you should see something like this:

To confirm, if you login to MaxTAF cloud, open your target project, and navigate to Settings - User, you should see the bridge status indicator as green in color:

Now when you run a case from your project, communication will be established between MaxTAF Servers and your Selenium Server

Mapping a drive is a functionality that lets you use your workspace from your personal computer.

Mapping your workspace folder to your computer allows you to use it directly from an IDE like a project.

To map your workspace folder, do the following:

  1. Go to the MaxTAF Cloud Dashboard
  2. Go to Settings => Project
  3. Open the cmd/terminal
  4. Execute the corresponding command (found in the project settings page under “Mount workspace”) to mount the drive
  5. On Windows, the mounted folder should appear in “This PC” and on Linux in “/mnt”


MaxTAF allows users the ability to run their MaxTAF tests through external infrastructure providers. This allows cross-platform testing for browsers and operating systems not supported by MaxTAF.

Each of these external providers will have their own instructions but here are a few examples.

Note: for each example you will need to replace the methods inherited from the Base class as it provides its own capabilities for the set up process. You can even remove the "extends Base" from the test as it does nothing. If you are not using a MaxTAF template to create this test then you can ignore this note.

To use SauceLabs as your infrastructure provider with MaxTAF tests (you need to register an account with them for this):


To use BrowserStack as your infrastructure provider with MaxTAF tests (you need to register an account with them for this):


To use LambdaTest as your infrastructure provider with MaxTAF tests (you need to register an account with them for this):

The process varies between different infrastructures but it usually requires getting the desired capabilities required to run a test on different browsers and operating systems.

After doing this for any website You should now be able to see the test results in the infrastructure website used to run the tests. If its not there, it is likely an error will have been returned to MaxTAF saying why it didn't run.


Reports

MaxTAF has basic run results that are accessible immediately after a case or suite has finished running.

You can access this in a few ways:


You should be taken straight to a basic report after your test has begun running. When completed it will update with the final information of that test run.


To view all of the previous runs from your project, navigate to the Completed Runs Page via the side navigation menu (listed under "Runs" as "Completed").

This page contains a list of all runs completed on the system. It can be useful for understanding the entire scope of testing that is ran at any one time across your entire project. Accessing any of the runs on this page will bring you to the same run details screen as if you had run the test directly from the test's case page.

The list has the option of viewing the list of suite runs instead. The sutie list will not show the runs that came from running the same test multiple times using the advanced runs menu. Those will be found in the cases list.


To see all the runs that are currently in progress, navigate to the Active Runs Page via the side navigation menu (listed under "Runs" as "Active").

This page shows a list containing every run that has not finished its execution. Just as with the Completed Runs Page, each list item contains a test case from which you can access the run details page for said run. One unique aspect about this list that next to each run is a camera icon which, when clicked, will show the live preview in a new window.

Any test runs found in this list can be stopped by either clicking stop all at the top or just pressing the stop button at the end of each row if you want to stop them individually.


Most of the details are self explanatory here, but there are some things you could miss:

Failures: If a test fails, details of the failure are presented.

This includes whatever failure message has been prompted, as well as the stack trace for the failure in the script code.

Pressing the Line number (e.g Line 19 in the image below), will open the code tab lower in the report, and take you to the precise line where the test/script failed.

Log: if the script prints anything to the log, this is shown in the log dropdown. Note: this will update live as the test runs.

Console Output: Details the output of the browser console during the script run.

Params: details any params used during the test run.

Code: presents the code used during the run. Useful for version control/record keeping etc.

Allure is an open-source framework designed to create test execution reports that are clear to everyone in the team. You can read more about allure on Allure official website.

MaxTAF Cloud has integrated support for Allure reports. Each run automatically generates an Allure report.

MaxTAF Cloud automatically generates an Allure report for each run. You can view the generated report from the Run details page by clicking the Allure report button.

Can I generate an Allure report for multiple test cases?

Yes you can. First, you have to create a Suite with the test cases you want. When the Suite is ready, run it. After the run is completed you will be able to click Allure report button.


Collaboration

The examples project is a community driven project of example tests.

It is intended for MaxTAF users to refer to for their own automation problem solving tasks.

Every user is included in the examples project by default.


Any user can upload new tests to the examples project, in the same way they would do for their own projects.


Example tests can be found by accessing through the project page or by switching projects using the menu at the top of the screen.

You may also use the Code Lookup in the script tab for a case and find other cases to lookup from Examples.


External Tools

MaxTAF Cloud allows importing cases from any external source as long as they export as a selenium test file for one of the languages we support. However, of the many external tools available we provide extra support for Selenium IDE recordings.

Selenium IDE provides a low code alternative to writing test cases which lets users trace the steps they want their test to take by recording their path through a website. This is great for people who have less programming experience or just need to write a functional test quickly.

For more detailed information on Selenium IDE, the documentation can be found here.


To add Selenium IDE to chrome you need to:

  1. Navigate to the chrome extensions page and search for selenium IDE.
  2. Add Selenium IDE to Chrome by clicking "Add to Chrome".

To use selenium IDE you must:

  1. Open Selenium IDE via the extensions tool bar at the top.
  2. From the dialog that is immediately shown, select "Record a new test in an existing project.
  3. Enter the name for your project and provide the base URL you wish your tests to start from.
  4. An instance of chrome should open from which you can perform all the steps you wish to be replicated in your test case.
  5. Once you have finished these steps, open the selenium IDE window again and click stop recording.
  6. From the following dialog enter the name of your test and confirm.
  7. Finally Click Save and choose where it will be stored.

You have now recorded and stored your first recorded test case with Selenium IDE.


  1. Use the Selenium IDE to create a test.
  2. In the Selenium IDE window, right click on the test you wish to import and right click.
  3. Click on “export” to download the recording as a test file.
  4. Navigate to the Cases section of MaxTAF Cloud and select “Import”.
  5. Drag and drop the file into the file section of the import dialog (or click “browse” to find it in your file system).
  6. Click “Upload” to finish the import process.

You will now be able to find the imported test case in the "Cases" page. In the script tab you can see that MaxTAF has altered the set-up method to use its own driver instantiation method, as well as instantiating an instance of the MaxTAF API Service.


Settings

Projects are administered from the project settings menu, accessed under settings.


Multiple Users can be added to a single project, allowing for frictionless collaboration on a shared workspace.

In Settings -> Project -> Users, users can be added to the project

Pressing ‘Add User’ will allow you to add users by email.

Note: Users do not need a MaxTAF account to be added, but must make one if they wish to use MaxTAF.

From the Users Tab, selecting a user will display a pop-up dialog in which their role can be altered by the administrator of the project.

This list of Users has filtering capabilities.


Users who you have added to your project but have not yet created an account appear here.

This list of Users has identical filtering capabilities as the Users tab does.


You can administer parameters on the project layer here.

To create a project param:

  1. Click "Create" above the list of parameters.
  2. Assign a name and value for the parameter.
  3. Optionally you can also set a description.
  4. If you check the "Auto increment" box, the parameter will increment its value by 1 if it is an integer every time it is used.
  5. Confirm by clicking "Add" at the bottom of the dialog.

This tab has filtering capabilities.


Individual Users have a few unique settings that can be set for them.


Parameters are a powerful and flexible tool for system configuration in MaxTAF Cloud. They provide test cases with key-value pairing which the test can then use.

For example, adding a parameter "name=john" to a case will allow the test to fetch the name "john" by calling the APIService method getParam("name") in its script.

Below is a list of all reserved parameters. Reserved parameters can have custom values but the key cannot be used for any other task:

mx.test.##

mx.datadriven.file

mx.selenium.server

mx.suite.##

mx.datadriven.runtype

mx.selenium.enable_video_recording

mx.user.##

mx.datadriven.name.##

mx.selenium.browser

mx.project.##

mx.datadriven.col.##

mx.selenium.mode

mx.system.##

mx.selenium.slow_down_level

mx.time_to_live

mx.bridge.address

mx.selenium.proxy

mx.params.source

mx.notify

  

mx.params.method

mx.notify.email

  

mx.params.method.source

mx.notify.on_outcome

  

mx.mxml.schemaFormat

mx.params

All three parameters need to respect their own rules. That means that if you set them like this:

mx.params=user-suite-test

mx.params.method=intersection

mx.params.method.source=test

They are not going to be applied inside the “project” layer, because the “project” layer (where they are defined) is not going to be taken into consideration when setting parameters. Look at “mx.params.source” as an example.

If the parameters were defined like this:

mx.params=user-suite-test-project

mx.params.method=intersection

mx.params.method.source=test

Then they are not going to be applied inside the “project”, “suite” or “user” layer, because these layers will not be in intersection if not defined so in “test”. Look at “mx.params.method” and “mx.params.method.source” as an example.

Default value

mx.params.source=user-suite-test-project

Possible values

‘test’, ‘suite’, ‘user’, ‘project’, ‘system’, separated by -

Meaning
  1. Determines which layers will be used to set parameters for run.
  2. Determines which layer has bigger priority.

Example: ‘mx.params.source=user-test-project’

‘user’ layer has top priority, then ‘test’ and then ‘project’ layer. That means that if the same parameter is defined in the “user” and “test” and “project” layer, the one in the “user” layer will be used.

Examples:

layers

user

suite

test

project

system

fruit=cherry

fruit=banana

fruit=apple

fruit=pear

fruit=grapes

..

..

..

..

..
1) If “test” layer had the parameter “mx.params.source=test”, the result would be:

fruit=apple

Because the parameter “mx.params.source” is set to “test”, the “test” layer will be used for parameter setup. Other layers will be ignored.

2) If “suite” or “test” layer had the parameter “mx.params.source=suite-test”, the result would be:

fruit=banana

Because the same parameter is in the “suite” and “test” layer, the parameter from the layer with bigger priority will be used. In this case the “suite” layer has a bigger priority than the “test” layer.

3) If “user”, “suite”, “test” or “project” layer had the parameter “mx.params.source=user-suite-test-project”, the result would be:

fruit=cherry

Because the same parameter is in different layers, the parameter from the layer with the biggest priority will be used. In this case the “user” layer has the biggest priority.

4) If “test”, “suite” or “project” layer had the parameter “mx.params.source=test-suite-project-test”, the result would be:

fruit=apple

You can mention one layer multiple times.

Because the “test” layer has the biggest priority, the value of the parameter is “apple”

layers

user

suite

test

project

system

fruit=

fruit=banana

fruit=

fruit=pear

fruit=

..

..

..

..

..
5) If the “test” or “project” layer had the parameter “mx.params.source=suite-test-project”, the result would be:

If suite is run:


fruit=banana


Because the test is run from a suite, we have a “suite” layer. Because “suite” layer has bigger priority than “test” and “project” layers, parameter will take value from the “suite” layer

If test case is run:


fruit=pear


If the test case is run individually, “suite” layer doesn’t exist, so we compare “test” and “project” layers. “test” layer has bigger priority, but the parameter “fruit” doesn’t have a value in the “test” layer, so in that case if the value is not set, the layer with smaller priority will be picked. If the parameter exists, we would be using it.


If we wanted to not set the parameter value, we could’ve done that in two ways. One way is to delete the parameter “fruit” from the “project” layer, or we could’ve changed the value of the parameter mx.params.source to mx.params.source=user-suite-test, so the “project” layer is not included.

For more examples, look at the bottom of the “mx.params” category, because in order to understand everything, you need to know what “mx.params.method” and “mx.params.method.source” parameters do.

Default value

mx.params.method=intersection

Possible values

‘union’, ‘intersection’

Meaning 

Explains the way in which layers will merge

Examples:

layers

 

 

user

suite

test

project

system

fruit=cherry

fruit=banana

fruit=apple

fruit=pear

fruit=grapes

   

food=pizza

   
     

seasons=winter

  
  1. If “test” or “project” layers had the following parameters: ‘mx.params.source=test-project’ i ‘mx.params.method=union’ the result would be:

    2. fruit=apple

    food=pizza

    seasons=winter

  2. If “test” or “project” layers had the following parameters: ‘mx.params.source=test-project’ i ‘mx.params.method=intersection’ the result would be:

    3. fruit=apple

    food=pizza

    If you wonder why the result is not

    fruit=apple

    Look at the explanation of parameter ‘mx.params.method.source’

    For more examples look at the end of mx.params category, because in order to understand everything you need to know what “mx.params.source” and “mx.params.method.source” parameters do.

Default value

mx.params.method.source=test

Possible values

‘test’, ‘suite’, ‘user’, ‘project’, ‘system’

Meaning 

Determines what is intersection for layers when parameter ‘mx.params.method’ has value ‘intersection’

Default value

mx.params.method.source=test

Possible values

‘test’, ‘suite’, ‘user’, ‘project’, ‘system’

Meaning

Determines what is intersection for layers when parameter ‘mx.params.method’ has value ‘intersection’

Examples:

layers

 

user

suite

test

project

system

fruit=cherry

fruit=banana

fruit=apple

fruit=pear

 
   

animal=cat

animal=dog

 

seasons=summer

    

seasons=winter

  
   

food=pizza

   
       

instrument=guitar
  1. If the test parameters were ‘mx.params.source=user-suite-test-project’, ‘mx.params.method=intersection’ and ‘mx.params.method.source=test’ the result would have parameters:

    2. fruit=cherry

    animal=cat

    food=pizza

    Because the value of parameter ‘mx.params.method.source’ is ‘test’ we will have only “test” layer parameters.

  2. If the test parameters were ‘mx.params.source=user-suite-test-project’, ‘mx.params.method=intersection’ and ‘mx.params.method.source=suite’ the result would have parameters:

    3. fruit=cherry

    Because the value of parameter ‘mx.params.method.source’ is ‘suite’ we will have only “suite” layer parameters.

  3. If the test parameters were ‘mx.params.source=user-suite-test-project’, ‘mx.params.method=union’ and ‘mx.params.method.source=user the result would have parameters:

fruit=cherry

animal=cat

seasons=summer

food=pizza

Because the value of parameter ‘mx.params.method’ is ‘union’, parameter ‘mx.params.method.source’ is ignored and result contains union of parameters from all the layers that are mentioned in “mx.params.source” parameter.

Examples:

layers

 

user

suite

test

project

system

fruit=cherry

fruit=banana

fruit=apple

fruit=pear

  
   

animal=cat

animal=dog

  

vegetables=carrot

vegetables=pepper

      

seasons=summer

    

seasons=winter

  
     

drink=beer

drink=water
   

food=pizza

    

color=green

color=red

  

color=pink

  
 

sport=basketball

sport=

sport=tennis

  
       

instrument=guitar

 

  1. All 3 parameters have default value and they are not defined in any layer (we can do that in test parameters)

    2. mx.params.source=user-suite-test-project

    mx.params.method=intersection

    mx.params.method.source=test

    The result would be:

    fruit=cherry

    animal=cat

    vegetables=carrot

    food=pizza

    sport=basketball or tennis

    - The result has only parameters that “test” layer has, because parameter “mx.params.method” has value “intersection”, which means that the intersection of the layer will be made in relation to “test” layer that we see from parameter “mx.params.method.source=test”

    - The value for fruit is “cherry” because parameter “mx.params.source” has a “user” layer, then “suite”, then “test” and, in the end, “project” layer. When the same parameter exists in multiple layers the one with highest priority compared to the “test” layer will be used (mx.params.method.source=test), and in this case that’s “user” layer.

    - The value for animal is “cat” because that parameter is not defined in any layer with higher priority (“user” and “suite”) compared to “test” layer (mx.params.method.source=test). It is however defined in a layer with lower priority but will not be applied because the parameter has value in the “test” layer.

    - The value for food is “pizza” because that parameter is not defined in layer with higher priority (“user” and “suite”) compared to “test” layer (mx.params.method.source=test)

    - The value for sport depends if the test is run inside the suite or on it’s own. If it’s run inside a suite, the value will be “basketball” because the “suite” layer has higher priority than the “test” layer. If it’s run on its own, the “suite” doesn’t exist and the parameter sport exists in two places: in the “test” layer and in the “project” layer. In that case the “test” layer has higher priorities, but because parameter sport in the “test” layer doesn’t have value, the value will be looked in layers that have lower priority and if it exists it will be used. In this case, parameter sport exists in the “project” layer and has value “tennis” so that value will be used.

  2. If we use “union” instead of “intersection” for mx.params.method and we define these 3 parameters somewhere in “user”, “suite”, “test” or “project”:

    3. mx.params.source=user-suite-test-project

    mx.params.method=union

    mx.params.method.source=test

    The result would be:

    fruit=cherry

    animal=cat

    seasons=summer

    drink=beer

    food=pizza

    color=green

    sport=basketball or tennis

    Parameter ‘mx.params.method.source’ is ignored because the value of the parameter ‘mx.params.method’ is ‘union’.

    Because the value of the parameter “mx.params.method” is “union”, the result contains all parameters that are defined in layers which are specified in “mx.params.source”. In our case that’s “user”, “suite”, “test” and “project” layers. Parameters from the “system” layer are not included in the utmost result because that layer is not defined in parameter “mx.params.source”, therefore we don’t have parameter “instrument”. That’s good because “system” layer parameters should be avoided in “mx.params.source”.

    - The value for fruit is “cherry” because parameter ‘mx.params.source’ has a ‘user’ layer, then ‘suite’, then ‘test’ and in the end, ‘project’ as the last layer. When the same parameter exists in multiple layers, the one with the higher priority will be used, and in this case that's the “user” layer.

    - The value for animal is “cat” because the layer with the highest priority is ‘test’ layer.

    - The value for seasons is “summer” because parameter ‘mx.params.source’ has a ‘user’ layer, then ‘suite’, then ‘test’ and in the end ‘project’ as the last layer. When the same parameter exists in multiple layers, the one with the higher priority will be used, and in this case that's the “user” layer.

    - Value for food is “pizza” because the layer with the highest priority is “project” layer.

    - The value for color is "green” because parameter ‘mx.params.source’ has a ‘user’ layer, then ‘suite’, then ‘test’ and in the end, ‘project’ as the last layer. When the same parameter exists in multiple layers, the one with the higher priority will be used, and in this case that's the “user” layer.

    - The value for sport depends if the test is run inside a suite or on it’s own. If it’s run inside suite, the value will be “basketball”, because “suite” layer has bigger priority than “test” layer, but if it’s run on its own, then “suite” layer doesn’t exist and parameter sport exists in two places: in “test” layer and in “project” layer. In that case the “test” layer has higher priority, but because parameter sport doesn’t have value in the “test” layer, it will be searched for in layers that have smaller priority, and it will be used if it exists. In this case parameter sport exists in the “project” layer and has value “tennis” so that the value will be used.

  3. If the parameters are defined inside suite parameter:

    4. mx.params.source=test-suite

    mx.params.method=intersection

    mx.params.method.source=suite

    Then the result would be:

    fruit=apple

    vegetables=pepper

    color=red

    sport=basketball

    This makes sense only for suite runs, so we will cover that case only.

    - The result has only parameters that “suite” layer has, because parameter “mx.params.method” has value “intersection”, which means that intersection will be applied to layer in relation to “suite” layer, based on parameter “mx.params.method.source=suite”.

    - The value for fruit is “apple”, not “banana”, because in parameter “mx.params.source” we specified that the “test” layer parameter has higher priority than “suite” layer. Also because the value “mx.params.source” doesn’t contain a “user” nor “project” layer, their parameters will be ignored, so the value can’t be taken from them.

    - The value of vegetais “pepper” because in parameter “mx.params.source”, we specified which layer will be looked at and which priority the layers have. There is no layer with higher priority that has that parameter. Other layers that have it aren’t included because we didn’t specify them in “mx.params.source”.

    - Value of parameter color is “red” because in parameter “mx.params.source” we specified which layer will be looked at and which priority the layers have. There is no layer with higher priority that has that parameter. Other layers that have it aren’t included because we didn’t specify them in “mx.params.source”.

    - Value for sport is basketball though the ‘test’ layer has higher priorities, but because parameter sport in ‘test’ layer doesn’t have value, it will be looked for in a layer that has lower priority and if exists, will be used. In our case, parameter sport exists in the “suite” layer and has the value “basketball”, so that the value will be used.

Default value

mx.time_to_live=300

Possible values

Number between 0 and 300

Meaning 

Specifies maximum allowed time for a test case run in seconds

- If we don’t have the parameter set, the default value will be used

- Maximum time for every test is 300 seconds (5 minutes) and can’t be higher than that.

- The user can put in a smaller value than 300 seconds, for example 60 seconds. If the test would finish in 20 seconds, everything would be fine, but if the test carries on for more than 60 seconds it will fail with a message that the maximum time exceeded.

Examples

mx.time_to_live=60

mx.time_to_live=360

mx.notify

Default value

mx.notify=false

Possible values

‘true’ or ‘false’

Meaning 

Specifies if we should get an email notification about test run results.

 

Default value

Email address of the owner of the project

Possible values

Email address, or email addresses separated by ‘,’ (comma) character

Meaning 

If the parameter “mx.notify” has value “true”, then this parameter specifies to which email addresses should the notification be sent.

- If we want the notification to be sent to email@example.com, we would do that in the following way:

mx.notify.email=email@example.com

- If we want to send the notifications to multiple email addresses, we would do that in the following way:

mx.notify.email=email.1@example.com, email.2@example.com

- If parameter “mx.notify” has value “false”, parameter “mx.notify.email” will be ignored.

 

Default value

mx.notify.on_outcome=fail

Possible values

‘fail’ or ‘pass’ or ‘always’

Meaning 

If the parameter “mx.notify” has value “true”, then this parameter specifies in which cases would the notification be sent.

- If the value is ‘fail’ the notification will be sent only if the test run fails.

- If the value is ‘pass’ the notification will be sent only if the test run succeeded.

- If the value is ‘always’ the notification will be sent when the test run finishes.

- If the parameter “mx.notify” is set to “false”, the parameter “mx.notify_on_outcome” will be ignored.

Examples:

1) If the parameters for notify were set in the following way:

mx.notify=true

The email would be sent to the owner of the project, only if the test failed.

2) If the parameters for notify were set in the following way:

mx.notify.email=email@example.com

mx.notify.on_outcome=always

Because the parameter “mx.notify” has the default value “false”, the other parameters will be ignored, and the notification won’t get sent.

3) If the parameters for notify were set in the following way:

mx.notify=true

mx.notify.email=email@example.com

mx.notify.on_outcome=pass

If the test run succeeds, the notification will be sent to the email address: email@example.com

4) If the parameters for notify were set in the following way:

mx.notify=true

mx.notify.email=email.1@example.com, email.2@example.com, email.3@example.com

Because parameter “mx.notify.on_outcome” has default value “fail”, if the fails, the notification would be sent to the following email addresses: email.1@example.com, email.2@example.com, email.3@example.com

mx.datadriven

 

Default value

  

Possible values

Location of the data driven file
Example: mx.datadriven.file=/dd/data.csv

Meaning 

Specifies test as a data driven test and points to a location of the data driven file.

- Supported file types (extensions) are .csv and .xls (.xlsx)

- If the value of the parameter is “mx.datadriven.file” value is “/dd/data.csv”, but before we run a test we pass a data driven file from UI, the parameter will be ignored, and the file that was passed from the UI will be used.

- If the parameter “mx.datadriven.file” has a set value, the test will count as a data driven test.

Example:

1) If the parameter ‘mx.datadriven.file’ was set in the following way:

mx.datadriven.file=/dd/data.csv

That would mean that the test is data driven and that the data driven file is on the location “/dd/data.csv”

 

Default value

mx.datadriven.runtype=serial

Possible values

‘serial’ or ‘parallel’

Meaning 

Specifies in what way would we execute the rows from the data driven file.

- If the value of the parameter is set to “parallel”, but we set it to “serial” from UI, the data driven rows would be executed in serial because the UI dialog has higher priority.

 

Meaning 

With the help of that parameter, we can get the value of the column from a data driven file by a column name.

- This parameter is always used as a variable.

Examples:

  1. If we want to get value for parameter “place” for a column named “city” inside the data driven file, we could do that in the following way:

    2. place=${mx.datadriven.name.city}

    For “${}” usage, refer to our documentation.

     

    Meaning 

    With the help of that parameter, we can get the value of the column from a data driven file by a column number.

    - This parameter is always used as a variable.

    Examples:

  2. If we want to get value for parameter “place” from 3rd column inside the data driven file, we could do that in the following way:

    place=${mx.datadriven.col.3}

    For “${}” usage, refer to our documentation.

 

Values

If the bridge is active, the value of the parameter is bridge address, if not, it has no value

Meaning 

Specifies bridge address

- It’s specification is in the “user” layer.

- It exists only when MaxTAF Bridge is active (Look at MaxTAF Bridge documentation).

mx.selenium

 

Default value

  

Possible values

“${mx.system.common_pool}”,

“${mx.system.private_pool}”

or selenium server address.

Example: “http://1.1.1.1:4444/wd/hub”

Meaning 

This parameter specifies the selenium server address on which tests should be executed

- If the value is “${mx.system.common_pool}”, the common pool will be used for running the tests. Common pool is a pool of selenium nodes that everyone using MaxTAF Cloud shares. See more in the common pool section.

- If the value is “${mx.system.private_pool}”, the private pool will be used for running the tests. Private pool is a pool of selenium nodes that only you use, and are created on demand. See more in the private pool section.

- If the bridge is active, the parameter will have the following value: ‘mx.selenium.server=${mx.user.mx.bridge.address}/wd/hub’. The configuration for this is in the “user” layer.

 

Default value

mx.selenium.enable_video_recording=false

Possible values

‘true’ or ‘false’

Meaning 

This parameter specifies if the video of the test run should be recorded for later analysis

- If the parameter “mx.selenium.enable_video_recording” is set to “true”, the video will be recorded during run, and when the test finishes, the video will appear inside the “video” tab which can be expanded.

 

Default value

mx.selenium.browser=chrome

Possible values

‘internetexplorer’, ‘edge’, ‘chrome’ or ‘firefox’

Meaning 

Specifies which browser should be used when running a test

 

Default value

mx.selenium.mode=remote

Possible values

‘remote’ or ‘local’

Meaning 

Specifies if the remote selenium server should be used for running the tests, or if the tests are going to be run locally.

 

Default value

mx.selenium.slow_down_level=0

Possible values

Any number greater than 0

Meaning 

How much the tests should be slowed if they are too quick for you to monitor them by eyes (in seconds)

 

Default value

  

Possible values

Address of any HTTP proxy server

Meaning 

Specifies which HTTP proxy server should be used in order to route requests

- Pretty useful if you want location consistency. For example a test that does Google Searches in the United Kingdom only.

In order to understand these parameters, it is necessary to understand layers and variables.

mx.test.### mx.suite.### mx.user.### mx.project.### mx.system.### are not parameters but prefixes, and they are used for retrieving values of the parameters that exist in some other layer.

“mx” stands for “MaxTAF”, and next word specifies the layer and can be test, suite, user, project or system

Example:

  1. If we have parameter “time” inside “project“ layer and we want to get its value, we could do that in the following way:

    2. timeValue=${mx.project.time}

  2. If we have a parameter “time” in “user” layer and we want to get its value, we could do that in the following way:

    3. timeValue=${mx.user.time}

  3. If we have parameter “time” in “suite” layer and we want to get its value, we could do that in the following way:

    4. timeValue=${mx.suite.time}

    That will only work if we ran a test case inside a suite, because if we do not, there will be no “suite” layer

user

suite

test

project

system

Project layer is available only to ADMIN users