JavaScript Modules at HubSpot

Trevor is the author of Async JavaScript, now available from PragProg.

We deal with a huge amount of JavaScript in the HubSpot front-end: over 1,000 files split across dozens of projects. In this article, I'll talk about how we're organizing this code into modules now, the pros and cons of that approach, and how we're starting the transition to a radically different approach.

The Present: Static Builds

Each top-level directory of app.hubspot.com (like "/content" and "/contacts") has its own JavaScript bundle. These bundles are built with an internal tool called Static3. Static3 takes our raw JavaScript, CoffeeScript, and Handlebars files, then compiles, concatenates, and minifies them into bundles using Sprockets. We upload the bundles to a CDN for fast delivery. Each bundle gets a unique version number and is served with far-future HTTP headers, so browsers will cache the bundle until there's a newer build.

The concatenation means that our customers get reasonably fast script load times, since there's only one HTTP request involved, and very fast load times when moving around between builds. On the other hand, we do about 20 builds a day, and those JavaScript bundles are big enough to add noticeable lag. (The bundle for "/contacts," for example, weighs in at about a megabyte. Minified.)

A bigger problem with the static bundle approach is that it's made sharing front-end code across projects difficult. Static3 lets us bring in JavaScript modules from other projects, but modules written for a particular project typically rely on libraries, templates, and cached data that live in that project—and those dependencies aren't listed explicitly within the module. Instead we have a single project-wide namespace, and we do something like this:

/* contactModel.js */
hubspot.contacts.Contact = Backbone.Model.extend({
  // ...
});

/* contactView.js */
hubspot.contacts.ContactView = Backbone.View.extend({
  model: hubspot.contacts.Contact
  // ...
});

/* contacts.js */
//= require backbone.js
//= require contactModel.js
//= require contactView.js
// ...

Now if we want to display a ContactView in another project, we need to bring in Backbone.js, the Contact model, and ContactView, in that exact order. That means a lot of duplication and, worse, makes it easy for the Contacts team to break the dependent project by adding new requirements on their end.

We can ameliorate this by using Sprockets to list dependencies directly in modules, like so:

/* contactModel.js */
//= require backbone.js
hubspot.contacts.Contact = Backbone.Model.extend({
  // ...
});

/* contactView.js */
//= require backbone.js
//= require contactModel.js
hubspot.contacts.ContactView = Backbone.View.extend({
  model: hubspot.contacts.Contact
  // ...
});

/* contacts.js */
//= require contactView.js
// ...

But that's only a partial solution. For one thing, there's no enforcement: A coder can easily forget a dependency that the project already brought in, and they won’t notice until other projects break. Plus, Sprockets only works for purely static dependencies. What about, say, data that needs to be fetched from the server before a ContactView can be rendered?

That’s why we’ve started to move toward a pattern that’s been sweeping the JavaScript world: Asynchronous Module Definition (AMD).

The Future: AMD Modules

AMD is a simple but powerful idea: Each JavaScript file has a “define” function that names the module, lists its dependencies, and a callback that runs when all of those dependencies (and their dependencies in turn) are loaded. The most popular AMD implementation is the async loader library RequireJS.

Using AMD discourages you from using a global namespace. Instead, you take what you need as callback arguments at the top of the module, and return the module’s export from the callback (called the “factory”). Here’s how we’d rewrite our last example:

/* contactModel.js */
define('contactModel', ['backbone'], function (Backbone) {
  var Contact = Backbone.Model.extend({
    // ...
  });
  return Contact;
});

/* contactView.js */
define('contactView', ['backbone', 'contactModel'], function (Backbone, contactModel) {
  var ContactView = Backbone.View.extend({
    model: contactModel
    // ...
  });
  return ContactView;
});

/* contacts.js */
define('contacts', ['contactView'], function (Backbone) {
  // ...
});

This is more verbose than the Sprockets approach. Also, AMD definitions for some libraries (like Backbone) have to be patched in manually. But those are small prices to pay for increased reusability and maintainability. AMD is a terrific long-term solution to our JavaScript organization problem.

There are a couple of hurdles we’ll have to get over before we can go full-on AMD, though. First, how do we concatenate modules without Sprockets? We don’t want to put the burden of loading all of our JavaScript on RequireJS, because that could mean hundreds of Ajax requests just to render the page. We need to combine those scripts before serving them to the user. RequireJS has an optimizer that can do just that, but that’s only going to work after we’ve converted all of the modules in a project to AMD. And relatedly, how can we have AMD-style modules that depend on modules defined in the old style?

The Transition: hubspot.define & hubspot.require

A month ago, our own Brad Osgood solved this problem by writing simple define/require functions that create, and look for, both global namespace definitions and AMD-style definitions. So we can replace

hubspot.contacts.ContactView = Backbone.View.extend({
  model: hubspot.contacts.contactModel
  // ...
});

with

hubspot.define('hubspot.contacts.ContactView', ['backbone', 'hubspot.contacts.ContactModel'], function (Backbone, ContactModel) {
  return Backbone.View.extend({
    model: ContactModel
    // ...
  });
});

without having to make any other changes. That hubspot.define definition looks for an object called hubspot.contacts.ContactModel and creates an object called hubspot.contacts.ContactView. We’re still using Sprockets for concatenation. So for now, this is mostly window dressing. (We do get some small benefit from only executing modules when they’re needed.) But once every module in a project uses hubspot.define, we’ll be able to drop Sprockets and enjoy all the flexibility that AMD has to offer.