“Visual immediately saw an uptick in usage. Average daily snapshots increased from 400 to 5,800, indicating that users were adding Visual to their test plans at scale.”

Overview

In 2019, Sauce Labs acquired a visual component testing application (Visual) to expand its suite of testing capabilities. Since then, the Visual team made some improvements, but it was clear that Visual was not meeting customers’ needs:

• Limited test framework support
• No mobile support
• A different architecture than Sauce
• Its purchasing experience remains separate
• A different user experience from the rest of Sauce

Sauce is losing market share and revenue opportunities according to CSMs, Sales, feature request tickets, and user interviews. Many Sauce customers are interested in Visual but do not purchase a plan because of the above limitations and pain points.

Project Goals

I took on the task of reimagining Visual as an integrated part of the Sauce platform through a user-centered design approach.

• Visual is framework and device agnostic. Users can run visual tests in the framework of their choice, for desktop or native mobile.
• Visual is integrated into the Sauce Labs platform. User can manage their account, teams, and testing in the same place.
• Visual aligns with the Sauce UI and UX. Users no longer need to switch contexts when using Visual and Sauce. Both now utilize the same components and behaviors.

The Team

• My Role: Lead Designer
• Product Manager
• Engineering Lead
• Developers (4)

Planning

Our product triad — Product, Engineering, and Design — developed an Opportunity Tree to communicate how our work aligns and contributes to company objectives. The exercise also helped with scope and prioritization. Integrating the Visual UI into Sauce will help us accomplish several objectives — better UX, Alignment with Sauce, and unified user accounts — but we can’t build for every framework at once.

Our first objective was to get existing customers to use Visual, many of whom had been asking for an integrated Visual product. Our support tickets and customer feedback showed strong interest for Visual to work in the Cypress framework, by far the most requests.

Who are we designing for?

Research

Feedback from current Visual users was that it works well, but its UI could be more intuitive. There was no visual hierarchy, and it’s unclear what actions users need to take. From user interviews, we learned that:

• Visual provides value — “It’s such a cool tool.” — “The amount of things it’s caught…” — “…Adds confidence to our team.” — “Helps to provide more tests…easy to add tests.”
• Actions are not always clear — “Controls at the side-by-side view should be more obvious. What are the actions I can take?” — “At the overview, the logs should also be easily visible, not buried.”
• Some critical information is missing from the UI — “Who did the change approval? I don’t know. Maybe different team members are working on the same branch? Either way, it’s not clear to me.”
• The dashboard and filtering need help — “…current screener dashboard is not user-friendly. I want an overview of all my baselines. Make maintaining baselines a little easier.” — “Are we able to ignore the browser version? Need to see baselines from each browser easily. Have a search projects capability. We have a lot of projects. Always have to ‘click more.’”
• We need to improve workflows — “Can we take a screenshot to include in a Jira ticket?” — “Can we run functional and visual tests together in the same test?” “Some workflow is needed for generating different reports, UI and visual.”

Process Flow

We reviewed and simplified the workflow for how baselines snapshots are created, and how users review, accept, and reject changes.

Test Status Alignment

Most of the test status names between Visual and Sauce were identical or similar enough not to be problematic. However, while most functional test results are pass/fail, Visual results are change/no change and require review (by a human) to determine if a change is ‘accepted’ or ‘rejected.’ We ensured the Sauce status model and design accommodated the additional Visual statuses needed.

Nomenclature Alignment

An essential part of this project was aligning the nomenclature and object models between Visual and Sauce. What parts of the Visual model need to change, and what should inform Sauce? Are there terms and layers unique to Visual? Does the model change between batch and session-based frameworks?

UI Exploration

New Designs

Visual Test Results, Builds View: A build is the result of a collection of tests that a user ran at a particular point in time.

Build Detail View: Here, a user can see all the test results associated with a particular build.

Build Detail View, Expanded: Once clicked and expanded, the user sees the snapshots generated as a result of running the visual test. These snapshots are compared against baselines (previously accepted snapshots) to see if any visual elements have changed (color, style, text, position). A user can click on a snapshot thumbnail to see details of changes.

Snapshot Detail View: Users can review, and then accept or reject any changes. If accepted, the latest snapshot becomes the new baseline. If rejected. the user has the ability to add notes, create an issue, and share with others, automatically notified through Gitlab or Github.