How Our Frontend Teams Use CI/CD to Deliver Value to Customers
It was back in 2011 when then-HubSpotter Yoav Shapira wrote a blog post to celebrate an important milestone: more than 1,000 releases of our ...
When it comes down to building and deploying quality software, don't be misled into thinking that the classic "QA Team" is the best solution for your main line of defense. For an early-stage start-up that may be the only viable option. For anyone looking to be a major player in their ecosystem, there's a smarter way, and that involves evolving your QA team(s) into User Experience (UX) and Test Automation teams.
Traditionally, manual QA teams have been responsible for several tasks such as:
1. Creating a test plan based on the product requirements
2. Manual testing / Finding unexpected product behavior
3. Writing down results of the manual testing
4. Filing bugs in an issue-reporting tool
5. Following up on “fixed” bugs to verify they were actually fixed
6. Updating the test plan as new features are added or removed
The Classic QA approach isn't good enough anymore, for a few major reasons:
1. QA people tend to be bug-focused. While this may help you figure out if your product does what it's supposed to do, UX will tell you something far more valuable: Does your product perform the way that it should to be attractive to target users? Focusing on the latter will put you at a significant advantage, and that involves several stages such as field research, focus groups, usability testing and beta testing. If you don't have one already, you can form a UX Team to focus on these issues.
Here's some info on how we do UX at HubSpot:
2. Manual testing is repetitive and tedious, so QA people might have to go through the same sequence of steps over and over again for multiple variations and software changes. While manual QA may hold up for small-scale systems, this doesn't scale well for more complex ones due to the limitations of humans. Compared to machines, humans think and act slowly. Humans are also more expensive, take up more space, can get distracted, can't stay awake 24 hours a day, can't network with other humans as quickly as machines can with other machines, need to eat, have emotions, etc. Because of these limitations, it can be far more effective to automate most (if not all) of the tasks that humans would normally do manually.
Now somebody still has to write the test automation for computers to perform (which may include browser-based integrated tests) and the focus of that can handled by an Automation Team. You may already have developers writing unit tests for your product, which is very smart. Sometimes you may need to create non-test automation to perform various tasks from within a web browser, and you can have your Automation Team write those as well since they'll already be familiar with your collection of automation tools.
If you're trying to transition from classic QA into UX and Automation, you can break up and modify the six tasks at the top as follows:
On the automation side, steps 1 and 6 translate directly into writing the tests. Steps 2, 3, and 5 translate into running those tests. That automation is then able to find issues on its own and do all the logging/reporting for you. As for step 4, you can decide how you want to handle that. You could automate the filing of bugs in a reporting tool, but that has the potential of being a bad idea since you may be filing false positives. At that point, it may be best to verify the test failures manually so that only the real bugs get sent to developers. And make sure you have lots of unit tests in addition to all your integrated ones.
As for UX, if they can answer the following six questions about site pages, it's a good start to making useful design improvements:
1. Who is this page for?
2. What problem does this page solve for the user?
3. How do we know they need it?
4. What is the primary action we want users to take on this page?
5. What might point the user to take this action?
6. How will we know this page is doing what we want it to do?
Once you put the pieces of the puzzle together, it becomes clear that obtaining product quality requires more than just sending a "QA team" on a bug hunt. This is the future. To stay ahead of the competition you have to continue to be innovative and rewrite the traditional rules of getting things done. A good place to start is by improving the process of making quality software.
