Most technical students are familiar with the concept of a test bank. For those who aren't, test banks are collections of old tests and homework. They can be passed down informally from older students to younger ones or they can be complex digitized systems spanning years worth of material. The basic idea is that if you have an idea of the format in which the professor will ask you to demonstrate the knowledge you've attained in class you'll be able to get a higher grade.
When I was in school, all the professors knew our fraternity system maintained elaborate test banks called "cribs". They chose to handle it one of three ways: ignored it, acknowledged and fought against it or embraced it. Those who ignored it passed students that then flopped when asked about the material later. Those who fought against it spent untold amounts of time massaging their material to be crib proof. Their students often tried to absorb everything and often times weren't able to properly demonstrate their knowledge. The professors that embraced it focused on their old tests and changed minutia that required true knowledge of the concepts. They were open about the fact that they knew where you'd be looking for guidance and encouraged you to practice as much as possible. Their students were most often able to both accurately demonstrate knowledge of the material and the professors were able to quickly and easily identify those who didn't measure up.
Interviewing is in many ways like a test. Glassdoor is a great website that has many useful features, including a test bank of sorts where candidates can share information about the questions they were asked during their interview. As Glassdoor grew in popularity and as HubSpot interviewed more and more people the inevitable happened, our tests made their way into the bank.
We had a choice to make: What kind of testers where we going to be? Sticking our head in the sand and carrying forward, besides being very much against the very fiber of HubSpot, seemed like it would inevitably lead to some poor hires. Poor matches have the potential to hurt our business and they eat up a ton of time. We could work really hard, constantly monitor Glassdoor and spend a bunch of time coming up with new questions. The chances of us being able to generate radically new questions that tested for the knowledge and skills we most value at the same pace they leak out seems very unlikely and in the mean time, we may miss out on candidates that will have the potential to be great HubSpotters.
Ruling out the first two options left us with one obvious choice. Moving forward we're going to embrace the transparency that Glassdoor provides. We've going to answer the questions as they come out. We want candidates to know exactly what we've done in the past to identify our current team. We hope that by doing research into what questions we've asked and why, they'll come prepared with their skill set best tuned for us and we'll be able to quickly and accurately identify the candidates that best fit our needs and capabilities.