When new technologies are introduced, it takes experimentation, collaboration and problem-solving to determine how best to apply it to your customer’s challenges. Jo Crossick, Senior Engineering Manager, recently hosted an AI Hackathon with the HubSpot Flywheel Technology team to explore how ChatGPT may be able to be used as a helpful tool when solving for HubSpot’s internal use cases. Learn more from Jo below!
_________
Often, the final point of the Agile Manifesto is perhaps the most difficult of the four: “Responding to change over following a plan”. AI is a great example of such a change. On November 30th, 2022, OpenAI launched ChatGPT and the world of software engineering changed. The question was, how to respond? At HubSpot, we use iterative and agile approaches across our entire Product & Engineering org from our company leadership and down through every team, and we’ve been pulling on our experience in this area to find a path forward.
The broader field of generative AI has come on leaps and bounds, and ChatGPT made natural-language functionality suddenly easy and accessible. HubSpot hasn't been slow to deliver AI functionality for our customers - our AI teams have already launched features such as Content Assist and ChatSpot.
HubSpot Founder and CTO Dharmesh Shah introduces ChatSpot.ai.
Still, we want to go much further than that. AI is no longer something limited to a few specialists, but something that can be woven into the everyday life of every engineer. The Flywheel Product and RevOps teams are in a particularly good position to experiment and learn with this new technology, since we work directly with internal users like HubSpot’s customer-facing representatives. This means we can find out first-hand what’s useful and valuable to our customers.
It turns out it’s pretty hard to know where to start when you’re learning a new technology. Even with encouragement from leadership for teams to experiment, it was hard to get the ball rolling. There were an almost overwhelming number of ideas, but we just didn’t have the embedded expertise to know which ones would be feasible, let alone valuable and quick to deliver. Everything from seemingly practical suggestions to wild ideas were met with the same trepidation. It was clear that the approaches and methods we were used to applying to our business and technology domain to explore and prioritize work weren’t going to serve us well in this situation.
On any learning journey, I often find that getting going is the hardest part. And it was clear that this was the situation for our teams. We needed to get out of a delivery mindset and into a learning mindset, and a hackathon was the ideal way to achieve it.
Structuring and Launching an AI Hackathon
Teams were given just 6 days to build their hack projects and were going from a standing start - virtually none of the 80 participants had prior experience of using the ChatGPT endpoints. I always love to see how quickly learning happens, especially when it’s undertaken collectively and with dedicated time and practical application. Within 2 days, not only were teams starting to get some code working, but we were also developing a collective language and shared understanding to explain our ideas and approaches.
Many produced incredible results in short amounts of time. With 20 different projects worked on, I can’t mention them all! Driving productivity using technologies like sentiment analysis, forecasting, easy in-app contextual help, language translations, text summarization and recommending next steps were all potentially valuable areas we’re going to carry on exploring further for our reps. While some things were easy to slot into our existing tools, some things need a lot more refining before they’re usable. We learnt that setting the HubSpot specific context for ChatGPT isn’t a trivial task, but it’s something we’re already making good progress on, and other places we’re working out where we need to combine generative AI with predictive models to deliver the best value.
Equally valuable were the experiments that didn’t work, highlighting the boundaries and limitations of this new technology. And, as with any new technology, the devil is in the details. Even though we were wowed with what ChatGPT can do, we now need to go through a deeper level of investigation into where the gotchas are. Still, the most important thing is we’re now able to work through that initial list of ideas with a much better idea of which ones we should be tackling and how we can approach them. Some teams have already started work to build and assess production-ready versions of their hackathon projects.
Responding to change is difficult to do in isolation, and I’m excited about the way that the whole organization is working together to respond. We couldn’t have gone nearly so far and so fast if it weren’t for our legal and security teams coming up with practical, easy-to-follow guardrails and guidance for teams, our ML engineering specialists rapidly iterating on shared capabilities to keep our teams focused on end-user functionality, and our senior leadership for giving us the direction and ambition to keep us working together.
The teams are going to be running in learning-mode for some time to come. When it comes to projects in this space, we’ll be staying close to the hackathon mentality - the best way to see if something works is to give it a go! Keeping projects small, focused and self-contained is going to allow us to experiment and learn quickly and effectively. Not every experiment will be a success - if it is, we’re not being ambitious enough! And when things don’t work out we won’t hesitate to throw away the code and move onto the next idea.
ChatGPT in the future
What’s the longer-term future for the backlog of ChatGPT ideas? Ultimately we’re here to deliver value for our users, not play with new technology. Our expectation is that ChatGPT becomes an essential tool in our engineering toolbox to solve problems and deliver previously-impossible value. We have to fully understand how to use that tool before we get there, so we’ll keep running ChatGPT-driven experiments for a while yet. Responding to change needs a willingness to take risks and try new things, and I’m excited to see where we’ll be this time next year.
The Flywheel Product group's mission is to provide the technology solutions required to delivery on our GTM strategy, enabling HubSpot to drive growth. You can learn more about what we do here.
Do you like solving hard problems like this? We're hiring - check out our Careers Page for your next opportunity! And for a behind-the-scenes look at our culture, visit our Careers Blog or follow us on Instagram @HubSpotLife.