Three Key Ingredients to Make Your Business Experiments Better

Experimentation can be scary for business teams. That is why a lot of business experiments end up being multi-year projects that are followed by very heavy change management processes. While that can give a sense of security and risk mitigation, it makes it very hard to predict,_, measure, and control the success of the changes.

On the Flywheel Product team we are always tweaking HubSpot’s go-to-market operating model to get desired business results. We apply a product and engineering approach together with a business approach to increase the productivity of our sales, marketing, and services team and help our customers grow better.

In order to do that we constantly need to balance business experimentation with the needs of an increasing go-to-market team and customer base. Businesses of HubSpot’s scale need to keep a sense of predictability, as well as an ability to forecast revenue and reduce disruption on the go-to-market team.

So, how can you constantly keep experimenting with your go-to-market operating model to get the desired business results as your company continues to scale?

These are some of the main lessons we’ve learned from our experience with business experiments:

1. Experiment design

For a business experiment to be successful, you need clarity and alignment. A lot of complexity and issues stem from not having a good understanding of the problem, the goal, and the experiment itself. That is why it is crucial to have a clearly articulated problem statement, objective, hypothesis, north star metric, and KPI breakdown.

Having a good north star metric, though, is not enough. North star metrics drive alignment but they don’t provide clarity on what the experiment is about. That is because north star metrics are lagging indicators and impacted by many factors. For example revenue might be impacted by a recent change in quotas, and at the same time the marketing team might change their advertising spend for the quarter. So it is essential to have a good breakdown of that high-level outcome indicator to input metrics you can control and influence with the experiment.

Here are some steps you can take to clearly communicate your experiment and align teams around it:

Build a Business Experimentation Framework: This doesn’t have to be a rigid process but rather a collection of steps and artifacts that incentivize healthy discussion, clarity, and alignment.

Some main items the framework could have:

• Initial Research (Quant/Qual): This should be mix of qualitative and quantitative insights that highlight the problem and opportunity area.
• Experiment Template: This is the main artifact and the heart of the experiment. It includes the objective, hypothesis, prediction, etc. While simple, this tool is a very powerful way of incentivizing conversations and alignment.
• Questions: This is the greatest strength of the template. It's not only about filling the template, it's how you do it. It's the discussions you have with the team, the questions you ask to get you where you want to go.

A few examples of powerful questions:

• What is the problem here?
• What are we trying to achieve?
• Is that really our objective?
• What evidence do we have of this problem?

These questions should lead to the sections below:

• Clear Objective and Hypothesis: Agreeing on a clear objective and hypothesis for your experiment is critical. It will drive most of your decisions down the line.
• North Star Metric: This is the main outcome we want to drive. It’s usually a high level KPI like revenue, productivity, or acquisition rate. While this can be easy to agree on, the difficulty is in the next step:
• KPI Breakdown and Proxy Metric: This is crucial for an experiment to be successful. This should lead to a collection of input metrics that are more controllable and that we can experiment with to quickly see impact.

Here are a few experiment templates you can use:

• HubSpot Flywheel Product Business Experiment Template
• Ash Maurya's Experiment Canvas
• Ben Yoskovitz, Lean Analytics

Takeaway:

There is much more to the Experiment Framework, but building and testing an experiment template is a good start. The template itself should then drive and highlight most other areas of the process.

2. Project management: Start small

Business experiments tend to be more complex and require some level of project management. This gets harder the bigger your change and experiment is. But instead of trying to nail project management, we asked ourselves if we can start smaller.

Starting small might be more applicable than you think at the beginning. There are multiple ways to define small. For example, the number of regions you are running the experiment in, the number of users in the experiment, or the size of the change being implemented.

We found that in most cases we were good at starting small on the region and users aspect, but we didn't always start with the smallest change in the product.

Here are some steps you can take to make sure you are starting small:

A. Start with the smallest change possible: This shouldn’t be limited to the region or people impacted but also to the product changes.

• Mockups: This is the first potential change that should be considered. This could be a simple product design, a simple dashboard or sales process, a Looker report, etc. Anything that can give an indication of the change and that can solicit feedback from users.
• Manual changes: When mockups can’t do the job, try to use the tools you already have in place with some manual steps is an option.
• Small changes with limited user base: Implementing the most impactful and riskiest change to a small user base is another option.

B. Define small: While the above might sound fine, the difficulty is how to define if a planned change is small enough and what the smallest change for a given project is.

• List and rate your assumptions: Identifying the riskiest and highest impact assumption will force you to focus and build the smallest thing to validate that assumption, versus building something that validates multiple assumptions.
• Ask "Is this small enough?": Making sure we ask this question is a good start. It's easy yet powerful. The higher the uncertainty, the smaller and lower fidelity the change should be. This graphic from Sam McAfee illustrates this:

Here are a few tools you can use to start small:

• Riskiest Assumption Canvas
• Sam McAfee on Identifying your Riskiest Assumption
• Dan Olsen's legendary MVP Framework

Takeaway:

Starting small is crucial. When your level of uncertainty decreases and you've got lots of learnings from your experiments, you may be ready to implement change globally. This is when you start using your Project Management toolkit.

3. Stakeholder and strategy alignment

When you change the go-to-market operating model, there is a good chance a lot of people in the business will get interested in it. Whether they are curious about the opportunity or trying to make sure on how they are impacted, they will want to get involved.

While high interest is great, it may lead to a working group which is too big to manage effectively, making it difficult to make decisions. You will need a lot of pizzas in those meetings. Worse, it could make it unclear who the decision makers are, leading to even less clarity.

Here are some steps you can take to avoid this:

• Create a small decision-making group: While you cannot avoid involving a lot of people in a business experiment, it’s crucial to have a small group that can make decisions independently and quickly. As former COO JD Sherman wrote of HubSpot, “If we want to really make a difference, and truly achieve our mission of helping millions of organizations grow better, we have to be both big AND fast! That requires us to constantly evolve HubSpot’s operating system, and it requires a bunch of smart folks who get fired up about data and dashboards.”
  
  Make it clear who has the final say: Having a small decision-making group is a good start. As the need to make final calls and trade offs is always there, having someone whose job it is to make those decisions is helpful.
• Communicate what is being experimented: The Experiment Template can help a lot here. What's most important is knowing who we need to communicate our process to. It’s important to note here that the smaller you start, less disruptive the experiment will be, which should ease the need for communication to a wide audience.

Takeaway:

Starting small and having the experiment template filled should help with stakeholder and strategy alignment, as well as communication. Otherwise, when enlarging the scope of the experiment, having a small decision-making group who has the final say will make things go faster.

These steps will help your product team and business move faster in improving the go-to-market operating model. It will also help foster an environment of learning and a thirst for experimentation.

If you are excited about solving complex business challenges and optimizing go-to-market strategy through product, head over to our product careers page.

At HubSpot we are constantly learning and improving our product craft. I’d love to learn how you run business experiments in your company. You can connect with me on LinkedIn. I’d love to connect and continue the discussion.