Visual E2E Testing Reference

Screener End-of-life

The Screener visual testing solution is going to be discontinued on May 31st, 2024. You can migrate to the new Sauce Labs Visual Testing solution by following the integration steps.

If you have any questions, please reach out to your Customer Success Manager or Sauce Labs Support.

Visual E2E Commands

Screener's Visual E2E commands can be integrated into existing WebDriver test code simply and safely. Each command is simply a JavaScript comment placed in a WebDriver execute command — no need to install anything.

Init command

The Init command (/*@visual.init*/) is used to initialize and name a Visual test. This command must be added before any snapshot commands. It can be used multiple times in a browser session to initialize multiple visual tests.

Arguments

Argument	Type	Required	Description
name	String	Yes	Name of Visual test
options	Object	No	Init command options. Options available: ignore: comma-delimited list of css-selectors to ignore in all snapshots in test. Example { ignore: '.selector' }

Code Snippets

JS/WebdriverIO

browser.execute('/*@visual.init*/', 'My Visual Test')
browser.execute('/*@visual.init*/', 'My Visual Test', { ignore: '.selector' })

Java

((JavascriptExecutor) driver).executeScript("/*@visual.init*/", "My Visual Test");

Python

self.driver.execute_script('/*@visual.init*/', 'My Visual Test')

Ruby

driver.execute_script('/*@visual.init*/', 'My Visual Test')

C#

((IJavaScriptExecutor) driver).ExecuteScript("/*@visual.init*/", "My Visual Test");

Snapshot command

The Snapshot command (/*@visual.snapshot*/) is used to capture a visual snapshot. This JS comment can be added into your code wherever you want a snapshot to be taken, and can be used multiple times.

The above Init command must be called first before any snapshots are taken, or an error will occur.

Arguments

Argument	Type	Required	Description
name	String	Yes	Name of Snapshot
options	Object	No	Snapshot command options. Options available: ignore: comma-delimited list of css-selectors to ignore in snapshot. cropTo: single css-selector to crop the snapshot to. scrollAndStitchScreenshot: boolean option to capture a full-page screenshot using a scrolling and stitching strategy instead of using native browser full-page screenshot capabilities.

Code Snippets

JS/WebdriverIO

browser.execute('/*@visual.snapshot*/', 'State Name')

Example with ignore option:

browser.execute('/*@visual.snapshot*/', 'State Name', { ignore: '.selector' })

Example with cropTo option:

browser.execute('/*@visual.snapshot*/', 'State Name', {
ignore: '.selector',
cropTo: '#header'
})

Example with scrollAndStitchScreenshot option:

browser.execute('/*@visual.snapshot*/', 'State Name', {
ignore: '.selector',
cropTo: '#header',
scrollAndStitchScreenshot: true
})

Java

((JavascriptExecutor) driver).executeScript("/*@visual.snapshot*/", "State Name");

Example with ignore option:

HashMap options = new HashMap();
options.put("ignore", ".selector");
((JavascriptExecutor) driver).executeScript("/*@visual.snapshot*/", "State Name", options);

Python

self.driver.execute_script('/*@visual.snapshot*/', 'State Name')

Ruby

driver.execute_script('/*@visual.snapshot*/', 'State Name')

C#

((IJavaScriptExecutor) driver).ExecuteScript("/*@visual.snapshot*/", "State Name");

Example with ignore option:

var ignoredElement = new Dictionary<string, object>();
ignoredElement.Add("ignore", "#login_button_container");
JsExecutor.ExecuteScript("/*@visual.snapshot*/", "Ignore on Snapshot", ignoredElement);

Example with cropTo option:

var croppedElement = new Dictionary<string, object>();
croppedElement.Add("cropTo", ".bot_column");
JsExecutor.ExecuteScript("/*@visual.snapshot*/", "cropTo", croppedElement);

End command

The End command (/*@visual.end*/) is used to wait and get visual results. This command should be the last visual command in your browser session, used after all your visual snapshots.

The response will contain the following properties:

passed	Whether or not the test passed.
status	The test status. Possible values: success, failure, error, timeout, cancelled.
totals	Visual regression result totals.
states	List of all snapshots, including name, status and URL. NOTE: URLs are not permalinks; URL will direct to the latest results for the snapshot.
message	Error message.

Code Snippets

JS/WebdriverIO

const result = browser.execute('/*@visual.end*/')
assert.ok(result.passed, result.message)

Java

Map response = (Map)((JavascriptExecutor) driver).executeScript("/*@visual.end*/");
Assert.assertTrue((Boolean)response.get("passed"), (String)response.get("message"));

Python

result = self.driver.execute_script('/*@visual.end*/')
assert result['passed'] is True

Ruby

result = driver.execute_script('/*@visual.end*/')
expect(result['passed']).to be_truthy, result['message']

C#

var response = ((IJavaScriptExecutor) driver).ExecuteScript("/*@visual.end*/") as Dictionary;
Assert.IsTrue((Boolean)response["passed"], (String)response["message"]);

Example Responses

Success

{
  passed: true,
  status: 'success',
  totals: {new: 0, changed: 0, accepted: 2, rejected: 0, all: 2},
  states: [
    {name: 'State 1', groupName: 'Test 1', status: 'accepted', url: '...'}
    {name: 'State 2', groupName: 'Test 1', status: 'accepted', url: '...'}
  ],
  message: null
}

Failure

{
  passed: false,
  status: 'failure',
  totals: {new: 0, changed: 1, accepted: 1, rejected: 0, all: 2},
  states: [
    {name: 'State 1', groupName: 'Test 1', status: 'accepted', url: '...'}
    {name: 'State 2', groupName: 'Test 1', status: 'changed', url: '...'}
  ],
  message: '1 visual regression found. Test failed.'
}

sauce:visual Capability Options

Below are the available options that you can define with the sauce:visual capability.

Key	Type	Required	Description	Default
projectName	String	Yes	Project name.
apiKey	String	Yes	API Key for user's Screener account.
viewportSize	String	No	A <width>x<height> representation of desired viewport size. Example: 1024x768.	"1024x768"
branch	String	No	Branch or environment name. Example: main.
baseBranch	String	No	Branch name of project's base branch. Used for baseline branching and merging. Example: main.
diffOptions	Object	No	Visual diff options to control validations. Options available: structure: enable or disable structural changes. Defaults to true. layout: enable or disable layout changes. Defaults to true. style: enable or disable style changes. Defaults to true. content: enable or disable content changes. Defaults to true. minLayoutPosition: Optional threshold for Layout position changes (in css pixels). minLayoutDimension: Optional threshold for Layout dimension changes (in css pixels). minShiftGraphic: Optional threshold for pixel shifts in graphics (in css pixels).	{ structure: true, layout: true, style: true, content: true, minLayoutPosition: 4, minLayoutDimension: 10 minShiftGraphic: 2 }
ignore	String	No	A comma-delimited list of css selectors to ignore when performing visual diffs. Example: #some-id, .some-selector.
failOnNewStates	Boolean	No	Option to set build to failure when new states are found, and to disable using new states as a baseline. This option defaults to true, and can be set to false if user wants new states to automatically be the visual baseline without needing to review and accept them.	true
alwaysAcceptBaseBranch	Boolean	No	Option to automatically accept new and changed states in base branch. Assumes base branch should always be correct.	false
disableBranchBaseline	Boolean	No	Option to disable independent baseline for each feature branch, and only use base branch as baseline. Must be used with "baseBranch" option.	false
scrollAndStitchScreenshots	Boolean	No	Option to capture a full-page screenshot using a scrolling and stitching strategy instead of using native browser full-page screenshot capabilities.	false
disableCORS	Boolean	No	Option to disable adding CORS headers. By default, CORS headers are set for all cross-origin requests.	false
iframesBeta	Boolean	No	Option to enable the capturing and comparing of iframes content. Pseudo states on iframes like hover, focus, focus-within, and active are not supported.	false
iframesOptionsBeta	Object	No	Options to control the capturing and comparing of iframes content.	{ maxFrames: Infinity }