User Testing

Overview

Testing with users with a disability, gathering their feedback and thoughts, and observing them undertake tasks on a website is a vital cog in assuring real-world accessibility.

User Testing is an important part of the Australian Government's new Digital Service Standard (DSS).

Expert Skills & Knowledge

We have the essential and unique skill set required to run accessibility user testing successfully. We combine expert knowledge of:

• WCAG guidelines
• Assistive technologies (JAWS, ZoomText etc)
• Assistive strategies (the way people use their assistive technologies)

Knowledge of all three is critical for successful accessibility user testing.

me2 have the skills - see why

All our consultants are technical and practical accessibility experts with years of experience and have undertaken multiple user tests.

The People

me2 recruit suitable, representative, people from the following categories:

• Vision impaired (blind & low vision)
• Physically impaired
• Mild cognitive impairment (e.g. dyslexia)
• For our standard user test, we use between 4 and 12 users.

The Script

me2 work closely with the client to craft a set of tasks for users to undertake. The tasks include:

• Searching & navigating to a page
• Locating specific content
• Interacting with the site’s functionality (e.g. completion of forms)
• Using the website features (e.g. tabs, accordions, etc)
• Accessing different types of content (e.g. images, video, tables, etc)

Test & Record

An experienced facilitator will observe and conduct the test. The reactions and feedback of the participants during the test are collected using the “thinking aloud” test protocol.

In addition, both the user and the screen are recorded and watched together, with picture-in-picture.

Mobile: Responsive Websites

80% of blind and low vision users access websites with a smartphone. Our user testing includes testing on desktop and mobile devices.

We test the site with TalkBack on iOS and Zoom with Android.

Analyse & Document

The outputs of the user test are converted into valuable, actionable recommendations by the me2 team and documented in a comprehensive and easy-to-read User Testing Dashboard report and User Testing Findings report. (see below).

Knowledge Transfer & Support

Handover of the audit report, a presentation of the audit findings and user testing support.

Our User Testing consists of two reports

One for the project managers & team leads and the other for designers, developers and content authors

Dashboard Report

A summary report highlighting the overview accessibility.

Prioritising the issues in terms of severity and content type.

Findings Report

A report that lists the websites accessibility issues, the impact on users and technical recommendations on how to resolve them.

Including bespoke code snippet, examples and screenshots.

Our reports are easy to read and understand, while offering technical recommendations and bespoke code solutions.

By grouping issues by type and assigning them to the correct resources for fixing, our reports greatly speed up any re-development time and effort.

Issue Description

We provide a detailed outline of the issue encountered by the user during the test session.

WCAG Fail or Not

We outline if the issue is a WCAG fail or not - an important distinction.

Impact of the Issue

A description of the impact the issue had on the test user.

Bespoke Recommendations

Tailor-made recommendations including code snippets and screen grabs to illustrate each issue.

We Identify Who Should Fix an Issue

• Content Editors
• Developers
• Designers
• Multimedia

We Categorise Each Issue

• Image, Multimedia & Colour
• Structure & Content
• Navigation
• Forms & Interaction

Related Success Techniques

A list of associated WCAG Success Techniques.

Severity

All issues are not equal. We rate each issue from Minor (minor issue that is a matter of technical compliance and has little impact) through to Problematic, Serious and Catastrophic.

Prevalence

We identify how prevalent the issue is across the sample (Low, Medium, High, Very-High).

Test Method

How developers can replicate each test & ensure an issue is fixed.

Presentation of Findings

Handover of the user testing report, and a presentation of the findings and phone workshop.

Support

Each user testing evaluation comes with an extended four hours' email and phone support over 60 days.

We also provide support on an ad-hoc basis at an hourly rate if required, learn more.

Flexible Recommendations

One size doesn’t always fit all, especially with web applications and complex websites. If a recommendation can't be implemented, we are happy to work with your team to identify an alternative.