Lessons and Projects From The HubSpot Open Source Hack Night

Have you ever run into a challenging technical issue and found the perfect answer over on Stack Overflow?  Or, come across an open source library that already does almost exactly what you were about to spend a bunch of time on?  Yep.  Me too.  Every day.  In fact, if there's a day that I don't have this experience, it means I'm freakin' working on PowerPoint slides instead of writing code. In any case, that's the beauty of an open system.  I'm continually amazed and grateful for the culture of openness that permeates the development and technical community.

For the past year, we at HubSpot have been rewriting our entire development stack. (Side note:  this is not for the faint of heart).  Many times, we've been able to take advantage of open source projects like Hadoop or Backbone.js, but we've also been able to build on top of these to forge new, better (at least for our purposes) solutions. Companies like Twitter, Square and Netflix have open sourced some really useful things recently, and we felt it was our turn to give back as well. So we decided to get together a handful of developers, some Indian food and had an open source hack night. Oh, and by "we", I mean Jonathan Kim who organized it all, and is awesome. 

We learned a few lessons from this experience:

1. Open sourcing code is not as easy as doing a git push. A good open source project also needs documentation, a build/bundling system, integration with the appropriate package manager (maven, pip, npm, etc.) and some community building. Those things actually took us more time than expected, mainly because we had very little experience with it. Getting build-systems setup for Javascript projects was particularly time consuming. Modern front-end development is amidst a renaissance, and a new build/package system is created every day (e.g. npm, bower, ender, component).  If you're planning to host an open source hackathon at your company, take the time to learn about open source distribution and make that easy for your team. Spend fewer calories setting up Grunt compilers and more calories on cleaning up your open source code.

2. Emotional hurdles are as challenging as the technical ones.  Though most developers believe in open source, they're often a little reluctant to put some of their code out there.  They're not trying to protect intellectual property -- they're just a little apprehensive of complete strangers peering into their code.  Oddly, even though code is code, developers often see it as a very personal expression.  And, some of us (like me) want "just a couple of days" to clean up the code to prepare it for release.  You need to help people get over this emotional hurdle.

3. Make it easy.  Invest in infrastructure before-hand.  Have a repo ready, a build system figured out and some basic license info for people.  This gives contributors more time to focus on the code.

4. Get some marketing help.  We should have invited a marketer or two to help with the blog post portion of the night. It was something that people put off until the last minute, and having a professional around to help each engineer break the ice would probably have made that process less painful.

5. Don't expect a blockbuster hit.  Although it's possible that a project you open source becomes an instant, world-wide sensation that thousands of developers use -- that's unlikely.  Especially in the early days.  But, that's OK.  Open source is about a culture of sharing and collaboration.  Not every project is going to be a hit.  The idea is to get people into the habit of sharing.  And, I'd argue that knowing a specific tool or project might be released as open source someday gets people to write better, cleaner code.  

We're just getting started on this process and "open" to learning as much as we can.  If you have any experience taking internal tools and projects at your company and releasing them as open source, would love to discuss them in the comments.

Meanwhile, here's a list of the projects we released as part of this open hack night. We hope you find some of them useful.

Projects we opensourced:

astack (Shell)

astack is a command line tool we use to quickly get thread dumps from the JVM. There's no complicated setup, and you can get stack traces from the JVM even under heavy load. astack comes with a set of flags, for formatting, finding and aggregating the thread dumps.

EasyHibernate (Java)

EasyHibernate is a set of base classes and utilities for quickly setting up a Hibernate database layer. It also takes care of common operations, like count, sum and average so you can write DRY code.

executr (Javascript/Coffee)

Ever wanted to see a code example run live? executr is a jQuery plugin that executes coffeescript in the browser. It comes with a handful of options, but it's extremely simple to get started.

Humanize (Javascript/Coffee)

Humanize is a javascript object formatter that's been optimized for speed, written in coffeescript and backed by a suite of jasmine tests. Where other libraries only let you manipulate strings, Humanize gives you methods for translating numbers, arrays, and more into human-readable terms.

Messenger (Javascript/Coffee)

Messenger is the ajax wrapper you've always wanted. It comes with customizable error messages, automatic retries (with a backoff algorithm), and some sexy css styling. Plus it integrates with jQuery and Backbone, so you don't have to modify your existing ajax requests.

sanetime (Python)

Sanetime was written to DRY up all the common date/time manipulation while offering the most simple, versatile, and intuitive API possible. Give Sanetime a date-like object, and it'll automatically return a consistent time format of your choosing.

Sprocket (Python)

Sprocket makes it easy to expose normal python functions in a web API through a restful interface. It works out of the box with Django models and provides a mixin and events system for easily adding in reusable, self-contained components into API.

teeble (Javascript/Coffee)

If you're using Backbone and displaying tabular data, we built teeble for you. Teeble is a light-weight abstraction that allows you to build tables using Backbone views and collections.

virtualenvchdir (Shell)

The virtualenvchdir tool makes switching python virtualenvs much more pleasant. virtualenvchdir will automatically switch in and out of the appropriate virtualenv as you change directories.

What do you think?  Any of the above projects interesting?  Any tips on how we might better run an open source hack night?  Would love to read your thoughts in the comments.