johno
Writing
Notes
Contact

I'm often asked about MDX, why I built it, and what differentiates it from Markdown, JSX, and existing React + Markdown solutions. This is my mega-post that illustrates my thoughts that will be later condensed into something shorter on the mdxjs.com docs.

Documenting a design system

Firstly, I think it's important to talk about the state of the times when I began working on early versions of MDX. It was November 2017 and I was in Spokane, Washington at my in-laws for Thanksgiving. Nana and Papa were spending time with my daughter so I suddenly found myself with a bunch of time to play around and experiment.

I figured it was time to pull an issue off the backlog that I'd been procrastinating for a while.

I decided to take a whack at generating component documentation since Brent Jackson and I were working on Compositor Lab at the time. We were well aware that documentation is always out of date, so we were seeking a way to generate as much as possible to ensure that it was in sync with it's implementation. The problem was somewhat simple at the top level: given a component specification and it's implementation, how can you best document it?

Parsing the metadata out of the component was quick work since react-docgen, styled-system, and a theme.js made a lot of implementation details self-documenting. However, using that metadata in something friendly for cross-functional teams was proving to be difficult.

NOTE: I'm working on a new project called DesignQL that seeks to standardize the tooling and data structures present in design systems.

Of course we wanted the generated content to be Markdown. It needed to be accessible to all team members including designers and product managers that might not be able to write React components but had important usage guidelines to add.

However, combining Markdown and components felt like a hassle. There were a few things I was seeking to achieve:

  • Render React components for elements in Markdown
  • Importing and exporting metadata (component name, prop types, etc.)
  • Being able to render a code playground for the component
  • The ability to import and render a component at the document level

Existing solutions

This problem, and the issues I encountered were by no means bespoke. There were many projects that were trying to solve these same problems. The libraries that were the closest in functionality were "Markdown in JSX", but what I really wanted was "JSX in Markdown".

That might sound like an arbitrary distinction, but I think it addresses one of the fundamental differences in MDX. When you're seeking to write long form content, like a blog post or documenting a component, you want to start with the content and then layer in some custom content when necessary (like a component). MDX seeks to make content-driven documents easier to develop and that content easier to write.

Guillermo Rauch summarizes it best in his MDX proposal.

Content and visual treatment

At this same time, I'd recently finished building out Clearbit's book series. This was an awesomely ambitious project in content marketing, so I helped build out the template to allow the rest of the team to write out the chapters. The chapters often had specially designed aspects like lists, blockquotes, and other sections. I opted for using [[content]] blocks since they were one of the primary ways to introduce custom elements to a Markdown document. This worked okay, but as soon as you needed a one-off design of a list for a chapter I had to build a new component or add custom syntax.

It just didn't scale.

Composition over configuration

For me, React was the first thing that made components and composition "click". I learned about "composition over inheritance" in school, and the gorilla and the banana problem.

“The problem with object-oriented languages is they’ve got all this implicit environment that they carry around with them. You wanted a banana but what you got was a gorilla holding the banana and the entire jungle.” - Joe Armstrong

An authorable format

I've always been a huge fan of Markdown. I sincerely think it's one of the most powerful ways to write for the web. Being able to avoid HTML syntax for long form content is more natural and ensures that your writing receives the focus.

Unfortunately, Markdown doesn't speak components.

An open format

At Compositor we've often talked about the power of open formats. I think this is important for a few reasons:

Firstly, it ensures that there's no lock in. Similarly to other languages like HTML, CSS, JavaScript, and Markdown, MDX is open and portable between frameworks and tools. This interoperability is important. After all, the web has a very short memory.

Thirdly, JSX is a syntax that's open and powerful I've heard many critiques on JSX, but with its usage and adoption in React/Vue one can't deny it's potential. UI is best expressed declaratively, and with JSX that's the pit of success.

This brings us to the notion of "authorable". I realized we were onto something pretty quickly with MDX because I suddenly found myself writing again. MDX quickly became my journal for sketching out concepts and ideas because I could write out a problem that needed to be solved, and then code up a solution in line.

The implementation

After spending some free time over the next few weeks with remark and how to combine it with JSX/React I finally began to start a real implementation.

https://mobile.twitter.com/jhooks/status/1103156653967306752

The first commit was small and consisted of nothing but an AST snapshot test.

The MDXProvider

Keep the web quirky

As a kid I spent many hours building whacky sites for hobbies in GeoCities and tweaking the styling of my MySpace page. With the growing popularity of content platforms like Medium, we're losing the quirkiness and personality that the earlier web had.

Seamlessly interleaving components with your content allow you to express yourself more creatively.

Move slow to move fast

I'm really bullish on MDX in the long term, so we've been really careful about when and how we add new features. Simple APIs are difficult to achieve and require a lot of investment and planning.

What's next?

I think this is where things get most exciting. There are already awesome things being built with MDX like mdx-deck and docz. This is just the beginning.

An MDX editor

In the near future I'd love for an MDX-based WYSIWYG, and I've built a prototype I will hopefully open source soon. It's something that feels so natural when spiking content to a page. Later on I can address the same post in my code editor while I work on components I've imported. I think this is an important next step because being able to expose a way for less technical folks to edit MDX is of the utmost importance. It's also important for an open web as well.

There are a lot of CMSes and editors that have wonderfully powerful editors that can support components. But I think they miss the mark in a very big way, their "outputs" aren't an authorable format. This might seem like a weirdly nuanced distinction, but it's an important distinction.

An editor that only speaks JSON schema will never see buy-in from the developer community. Part of what makes Markdown so powerful is its ability to be translated between a raw format (an .md file) and a WYSIWYG. One of my initial motivations when creating MDX was implenting a format that was just as useful for devs as users of a futuristic WYSIWYG editor. This would let all members of a team to edit in the way that's comfortable to them. Vim, WYSIYWYG, VS Code, CodeSandbox, anything.

Generative documentation

Thanks

I'd also like to explicitly thank jxnblk and mrmrs for exchanging ideas and motivating me to start working on a prototype.

Huge thanks to rauchg, timneutkens, and the ZEIT team for helping to formalize the specification, getting it production ready, and ensuring the project is battle tested.

Thanks to the many projects that have served as inspiration to this project.

Most importantly we can't forget about wooorm and the unified team. remark's elegant plugin architecture allowed me to build MDX on top absurdly quickly.

And :heart: to all users of MDX and those that've expressed their joy with this project. I'm excited to see where this goes.