Saving time and resources through automated visual testing

From smartphones to desktop, Firefox to Safari, our customers access nsw.gov.au across different devices and browsers.  

The challenge for the teams creating the websites is to ensure the pages are presented correctly, no matter which device and browser they are being displayed on. This is important to ensure we are offering the best digital customer experience. 

Top 5 browsers for nsw.gov.au - September 2021

Overcoming this challenge is particularly important as more and more people seek information via their phones. In September 2021, 69% of customers accessed nsw.gov.au on a mobile, 28% on a desktop and 3% on a tablet. 

In the past, developers and Quality Assurance (QA) teams have relied on manual checks to look for visual issues. However, with so many sites and features added to the website, this has been time and resource intensive. 

The problem with manual checking

Websites need frequent updating as bugs are fixed and new functionality is added. 

The issue is that sometimes changing the code of one page of a website can result in things looking different elsewhere. 

For example, updating the font size on the home page may affect image sizes on another page. A new font could also present well on a desktop computer but cause text to go out of alignment on a tablet. It is not possible to know this will happen without looking at the page. 

Previously, ahead of a release of a new feature on the website, QA Engineers would go through a long process. They would load the same content on different browsers and devices and go through a checklist to make sure everything from text alignment to logo placement was consistent. It is easy to overlook things and difficult to capture every issue.

A new solution

With the OneCX program in place to migrate hundreds of NSW Government websites onto nsw.gov.au, there was a clear need to find a more efficient way to ensure consistency. 

After searching for suitable solutions, we discovered Visual AI software, which provides the ‘sweet spot’ between human and machine when it comes to visual QA testing. 

This tool takes a screenshot of the staging environment, which can be viewed after a web page has been updated, but before the changes are released. It then checks how the page would look on an iPhone, Android phone, desktop computer, tablet etc., comparing the difference between the ‘live’ and the ‘staging’ site. 
 

Visual AI testing report

The automated visual testing identifies differences a human eye would pick up on. For example, if an image has shrunk, if a button has disappeared from the page or if the text has dramatically changed size, the tool will highlight the difference. The tool also has the capability to avoid ‘false positives’, for example by ensuring the testing ignores dynamic data that changes daily, e.g., COVID-19 related information.

Visual testing highlighting an icon on the portrait tablet view is squashed

While visual testing is underway, additional technology can also be used to test links work and the site functions the way it is supposed to.  

Improving staff and user experience

Visual AI software is incredibly effective when rolling out new changes on nsw.gov.au. 

We can ensure customers receive a consistent user experience with pages displaying correctly. 

Using this solution as well as other functional automation testing, QA teams can cut down on repetitive tasks and the need to endlessly scroll through the same webpage looking for problems. As a result, we have captured several critical bugs before they hit the production environment. QA team also saved around 60 hours on manual testing efforts in each weekly release cycle via this automated visual testing.

By using automated visual testing, releases can happen faster and with more confidence behind them. Our QA team is then able to concentrate on more complex and value-adding tasks to improve the digital experience for our customers.

Have you ever experienced a large manual task or process that became automated? Where did you invest the time saved?