Why We Built Our New CMS on a File-Centric System Instead of a Database-Centric One

Earlier this year, HubSpot announced the launch of our new CMS Hub, an integrated, deeply powerful, and easy-to-use content management system. The goal going into creating this product line was deceptively simple: give marketers the ease of use they want, and developers the tools they needed to make a powerful web experience.

First, a little context. In the large field of CMS products available to companies, there’s often a stark tradeoff between power and ease-of-use. Options run the gamut from page builders with minimal developer tools to complex enterprise solutions that require dedicated developers and are so complicated that few companies are able to get their full value from them. In between those extremes, there are platforms that offer set themes and plugins more friendly to marketers, as well as static site builders that developers love because they can build in them in the tools and languages of their choice.

The issue is that none of those solutions offer both simplicity and power. It’s that combination that developers and marketers need to create great business websites. The magic lies in serving both highly technical ‘backend’ users and less technical people who need to comfortably be able to work with content without worrying about "breaking the website.”

Recently, we wrote about how HubSpot CMS brings this magic to solving for marketers’ needs. Now, it’s time for us to delve into the changes that we made to solve for developers. 

First up: the implementation of the developer file system.

The building blocks of content

Websites in the HubSpot CMS are built off of several types of building blocks: Modules are a way for marketers to use a codeless UI to build their page in a WYSIWYG editor, Templates provide the base files for a page, be it HubL/HTML, CSS, or JS. All of these get bundled up into a Theme, which provides common styling across your website and works as a top-level package for the other asset types:

Pages on a website are represented as objects in our Content database -- these object instances contain references to themes, templates, and modules used on a page as well as the configuration of those objects. All these asset types as well as the Content objects are all represented as rows in our large MySQL Content database and references are handled, essentially, by foreign key ids.

While this is a common pattern for web applications, it doesn't match the way developers are used to working with their web assets. Our solution to this had historically been requiring developers to use an online IDE to build sites on the CMS, called Design Manager. But as engineers ourselves, we knew what developers really wanted: to use their local machine to write code, have source control, use their favorite IDE and build tools, etc.

So the solution was pretty clear: if we wanted to create a better experience for developers on the HubSpot CMS, we needed to start representing these building blocks as actual files. And although that doesn’t sound like too crazy of an idea, we’d built years of code off of the object system and we couldn’t completely overhaul the internal workings of the CMS.

Finding a solution in files 

Our solution to this came to be known as the ContentFileMapper. We created canonical definitions for how to represent Templates and Modules as files, and created a service to turn local files into the objects understood by the rest of the CMS.

Some things were more straightforward than others. Templates were pretty close to real files already, just source code plus some metadata. We created a system of annotations to set the important metadata and then you can create a Template with just an HTML file:

Image is from our opensource boilerplate repo: https://github.com/HubSpot/cms-theme-boilerplate/blob/master/src/templates/about.html

Modules, however, were much more complicated. A module is built out of module-specific CSS, JS, HTML/HubL, and it provides specific fields to be set in the UI. We needed a way to define all of these components locally, plus have them bundled together.

An example module in the Design Manager IDE shows all of the different components that need a way to be defined

So the mapper defined a special folder suffix (.module) and specific file types contained within that folder: module.css, module.html, module.js, meta.json, fields.json. Combined, the mapper can turn these local files into a full module object for the rest of the HubSpot CMS.

We kept building out these local file definitions and it unlocked a world of possibility for us.

Because we now represent everything with files in a traditional folder tree, it means we could stop using foreign key IDs to cross reference assets and use their paths. It means that developers can store their files in GitHub and have real source control. It means that assets can actually be pulled down from one HubSpot portal and uploaded to another without needing any shenanigans with changing references to IDs that would have been automatically regenerated in the old world. 

Creating this mapping layer from files to objects has allowed HubSpot CMS developers to have much more power with the HubSpot CMS.