Finding out what’s broken – Usability Testing

BACKGROUND

LifeWorks is a cloud-based employee wellbeing platform. At every company that uses LifeWorks there would be a select amount of people (usually HR managers) that are in charge or doing tasks such as adding colleagues, posting onto the newsfeed on behalf of the company. This was done on a separate platform, the Admin Panel.

 

MY ROLE

At LifeWorks, I worked on the Data and Analytics team as the only designer. Aside from myself, the team was made up of: 1 PM and 5 developers.

My role on this project was to find out what was wrong with the colleague management section of the Admin Panel

THE PROBLEM

Company admins contact LifeWorks to do tasks they should really do themselves.

Data showed that company admins were constantly contacting the customer success managers at LifeWorks to complete simple tasks, like removing and adding new employees to the platform. Through user research I established that this was mostly due to admin users not knowing how to complete these tasks.

Using this as a foundation, I did some usability tests to work out where the holes were exactly and how best to fix them.

EXISTING COLLEAGUE MANAGEMENT

Here are some screenshots of the existing colleague management section of LifeWorks, at the time of testing.

OBJECTIVE

The goal of this research effort was to assess the overall usability of the colleague management section in the LifeWorks admin panel. Specifically the Data & Analytics team were interested in learning whether the colleague management section was intuitive to all users regardless of expertise.

TASK SELECTION

I used Realtime Board to list and prioritise all the tasks a user could do on the admin panel. The top 50% were identified as key user tasks. I then created scenarios to give users based on these key user tasks

METHODOLOGY

12 users participated in individual usability studies of the colleague management section in the LifeWorks admin panel. During the test session, users were asked to perform several tasks including adding a new colleague, archiving a colleague, making a colleague an admin and deleting a colleague invite. Users’ errors, comments and satisfaction ratings were observed and recorded during the study. Inevitably, users’ can feel tired as a result of acting under stress, which could bias the results. To prevent this the tasks were split into 3 sets and the users into 3 groups. This enabled the testing sessions to stay under 45 mins.

An effort was made to recruit users that had different admin panel expertise with the belief that the admin panel should be intuitive to all and not some.

RESULTING SCENARIOS

I created for each of the tasks that we decided to test. Some examples are:

5 new colleagues have joined your company. Their names are [example] and these are their details. Show me how you would give them access to LifeWorks.

 

These colleagues recently left the company and need to be removed from LifeWorks. Show me how you would do this?

 

A colleague has let you know that their contact number has changed. Show me how you would update their contact details on LifeWorks

 

These two people have recently left your company and didn’t join LifeWorks whilst they were here. Show me how you would prevent them from joining, now that they’ve left.

QUALITATIVE FINDINGS

For each task, I gave the participants a realistic scenario and looked out for common patterns. Here’s an example of how they found the task where they had to archive a single colleague.

‘Yesterday, this colleague approached you and told you that they don’t like LifeWorks and want to be taken off. Show me how you would take them off LifeWorks?’

  • 42% of users didn’t notice the archive option in the drop down menu
  • Once they had noticed the archive option, 85% of users didn’t associate the word ‘archive’ with the action of removing or deactivating a colleague.
  • The single user who didn’t experience this is a someone who uses the admin panel on a day to day basis.
  • One user double checked to see if the colleague had been archived after completing the task, this could indicate that they are uncertain as to whether they completed the task correctly or not.
  • One user didn’t notice that they hadn’t archived a colleague and actually thought that they had done something wrong, this could be due to the notification banner being red.
QUANTITATIVE FINDINGS

Pain Rating

After each task, users rated how painful the journey to their goal was on a scale from 1-5. Here is a breakdown of the different ratings given. As can be seen in the table, ‘Add colleagues via CSV’ was the task that took the longest

Time Taken

This is a breakdown of how long it took each user to complete their given task. As can be seen in the table on the right, ‘Add colleagues via CSV’ was the task that took the longest on average.

Success Rate

I made note of whether the task was completed correctly. The percentage that follows denotes the success rate. Eg if ⅘ users completed the task correctly, that then translates to a success rate of 80%. In many cases the user thought they had completed the task correctly when in actual fact they had not.

I gave each task a score based on how high they appeared on each leaderboard & then combined all the scores to reach a grand total, based on time taken, pain rating, and success rate.

TIME TAKEN
PAIN RATING
SUCCESS RATE
GRAND TOTAL
RESULTS

Adding colleagues via CSV upload scored the worst! Followed by archiving colleagues and adding multiple colleagues via email.

RECOMMENDATIONS

Based on these results I made some recommendations that we could focus on to improve the product (especially in the 4 areas that scored worst). Below are some of my recommendations for the archiving colleagues task.

Archiving Colleagues

  • The first column in colleague directory could be changed to ‘All colleagues’ to avoid users tripping up when looking for inactive colleagues
  • The copy ‘Archive’ could be changed to ‘Remove’ with the final option being to delete a colleague.
  • We could introduce the ability to archive a single colleague directly from colleague management section.
  • We should show the user a positive confirmation after a colleague has been successfully archived.
IMPLEMENTATION VS USER PAIN

I took the recommendations linked to the most problematic tasks to the developers on the team and together we were able place the recommendations on a matix that considered both user pain and implementation difficulty.

NOTES

1.1 An algorithm could be created whereby it recognises the data beforehand by figuring out formats of the date. Under the assumption that the data is consistent there would need to be 2 discriminators (at least one record where the day is above 12, at least one record where the year is above 31)

2.1 Easy, if we use buttons as that section of colleague directory isn’t legacy code (library is Angular not React)

2.2 Difficult to maintain selected items after search

2.3 Difficult because columns in each section are different

NEXT STEPS

After this research task, I sat down with the whole team and together we decided which problem to tackle first, what success would look like (metrics), and based off that I began iterating solutions!