8 Tips for Doing Usability Testing at a Fast-Paced Company

At HubSpot, we have one of the fastest development teams around. Our dev team continuously deploys code, up to 100 times per day, so our product is constantly changing. This leads to several challenges for us on the UX team, whose job it is to ensure that the software is easy and enjoyable to use.

One big challenge we have is to conduct usability testing in this crazy fast environment. As you may know, usability testing is often one of the first things dropped from the "must have" list of product release schedules. There are several reasons for this, common assumptions that are made about usability testing:  

• Usability testing takes too long or is too slow
• Usability testing is too hard to get right
• We can get the same data in other, easier ways

All of these assumptions are common but do not stand up to scrutiny. When done well, usability testing doesn't slow down the release process, it's not too hard to do, and it provides uniquely valuable information about your product that you simply can't get in other ways. 

So at HubSpot, we've refined our usability testing process to be as fast as possible. We continually test our software with every product team, being sure to implement testing in a way that does not slow any of our developers down. 

Here are eight things we've learned: 

1. Do a complete round of testing in one day

In order to keep up with the pace of the software and those developing it, we execute each usability testing sprint in one day. We schedule all the tests in one day, even if it becomes a really long day. This usually means four to five one-hour usability tests back to back, where we can quickly and clearly see the trends among the testers and provide the development team with key takeaways immediately. By keeping the testing to one day, we keep the development teams moving quickly and everyone can schedule around it easily. In addition, by consolidating all the testing to one day we make it easier for the product team members to attend the tests without disrupting other projects.

2. Offer different levels of service

Sometimes even one full day of testing is too long or too much on some projects, so we offer several levels of service to our product teams:

• Support Evaluation - A couple hours of talking and very fast usability testing with several support reps to attack a known technical issue with the software. Our Support folks watch and help customers interact with the software all day long and often see issues earlier than anybody.
• Consultant Interview - A couple hours talking with several Marketing Consultants to discuss any domain or conceptual issues around the software. Our consultants are helping people do marketing every day and so they are familiar with any issues that crop up.
• One day sprint - Our typical one day usability testing process that tackles both technical and domain-related issues.

Every so often, there won’t be enough time to commit to a complete testing sprint before releasing something new, so offering different levels of service before we ship can make all the difference.

3. Set up a reliable remote testing environment

At HubSpot we conduct remote usability testing most of the time. While we do the occasional in-person test when the situation calls for it, we test with our customers all over the globe and remote testing is the only way to do that quickly. It is valuable to get insight and feedback from users from places as diverse as Massachusetts, Ireland, and New Zealand. By remotely testing our users, we’re able to extract the most value out of our one day testing sprints. Some of the most valuable benefits of our remote testing sessions are:

• Being able to quickly set up three or four tests in a row
• Saving time and money by not coordinating in-person testing
• Setting up a test in less than 24 hours
• Everyone who should view the test can view the test (even those who are working remotely)
• No shows are easy to overcome

Rather than having to coordinate in-person usability testing, which takes days to plan and conduct, we focus on remote testing because it speeds things up considerably.  

4. Keep the best users ready at a moment's notice

We are in a unique situation at HubSpot: we have first-hand access to all of our customers. We can reach out directly to the people actually using the software at any time. We have a detailed database of testers that we keep current notes on and know which features they use most often. The customers in our usability testing database are ready to help us at a moments notice, so the recruiting process is quick and painless. By creating a database of the best, easily-accessible users we shave lots of time from the testing process. We email them a day or two before the test, and they block off an hour in their schedule. We work with only the most helpful, eager customers for these sprints. Without this ability to recruit awesome users quickly, our testing process would last days rather than hours.

5. Agree on high level tasks and use cases ahead of time

A testing sprint cannot be successful without carefully set high-level tasks and use cases prior to the test. Instead of creating the tasks from scratch every time we test, we've created what we call "product research pages" to record and archive the use cases for each product. These documents are rich with information and research about each product we have, including things like use cases, user research, competitive research, direct and indirect competitor information, as well as the results of any previous testing we've done. By taking a quick glance at this page, anybody (even those unfamiliar) can quickly get up to speed about everything we know and don't know about a product. So when we need to do any sort of usability testing, a quick glance at a product research page gets us moving quickly. Our product and usability teams work to maintain the product research pages over time so they are always up to date. By having these pages we can draw up a task list for usability testing extremely quickly.

6. Make sure the whole team is in the room

During our testing sprints, we make sure the product managers, interface designers and front-end developers are in the room observing and asking questions. The team then has the ability to ask the testers questions about their pain points, expectations and needs. The value for the development team when experiencing the user encountering issues and bugs far exceeds them reading a page of notes or listening to a recording. There is no second-hand information being shared when everyone is present, so the team can immediately move to recommending actionable takeaways after testing.

7. Debrief immediately after testing and create a shared plan

To help make the one day of testing even more effective, we hold a debriefing meeting immediately after the last session. In this half hour we discuss observations, trends, bugs, usability issues and any inferences we make about why users did what they did. We also develop actionable takeaways and assign them to different members of the team. By figuring out the next steps after the tweaks, we can decide if more testing is needed or if the feature is ready for the beta group to test out. The whole group is kept in the loop as development progresses. This debrief allows the team to get started immediately implementing the fixes and recommendations decided on the same day of the test. 

8. Compile notes & videos

Even though we make sure all the right people are in the room for testing, we still often want and need to share the results with the company as a whole.  Once the actionable takeaways are established, it is important to make the findings available to those who weren’t able to attend the testing, usually within 24 hours. At HubSpot, we have an internal wiki where we post all of our findings. A member of the UX team is taking detailed notes during the testing sessions, and those are summarized to show the main points (e.g. 3/3 people could not find the ‘save button’) of the testing, as well as the next steps. We put together a quick video highlighting the key insights for those who don’t have the time to read the notes, which only takes an hour or two. This wrap-up makes it easy for anyone in the company to see the results within a day of the test.

Bringing it all together

Testing in a fast-paced environment isn’t easy. We often don't have weeks or even days to plan a usability test. Having a successful process in place and knowing how to manipulate it when under pressure is key to releasing the best possible product to customers. While we’re getting pretty good at performing extremely fast usability sprints, there is always room to grow. Stay tuned.