Building a Webpack plugin to generate localized emails
At Dashlane we build most of our transactional emails and some marketing emails using basic HTML/CSS with a sprinkle of EJS templating for rendering variables and conditional statements on server-side. Unfortunately, HTML emails can be a frustrating thing to work on since email clients typically render using outdated standards and require nested tables and other things that the much more progressive web has moved past in recent years.
Our existing approach to building email templates was lacking from a technical standpoint too. We had a simple template build system that handled some things like i18n and A/B tests, but all of our emails were barebones HTML with no tooling to help developers build out each email. Each template was basically it's own unique file, with poor reusability. We wanted components, we wanted TypeScript, and importantly we wanted a drop-in replacement so we could seamlessly move from the legacy system without a hitch.
The goal: Create a new build pipeline for email templates
We had some important qualities going into our prototype:
- Ideally leverage React and TypeScript, which our other web tech stacks are built on
- Build static output in HTML with EJS template scripts
- Simplify process of building for email clients
- Support i18n/Localization
- Handle A/B Testing
After some initial prototyping, we landed on using MJML, an email-centric markup language from Mailjet. It solved a lot of the headaches involved with building specifically for email clients, which is an area where we didn't want to reinvent the wheel or spend too much time worrying about email client rendering differences. We expanded it to a proof-of-concept showing emails built in React, written in TypeScript, with a backend of MJML, and rendering to static HTML with full EJS still intact. This covered most of our goals so we knew this was the way to go.
To put it all together, we felt that Webpack could handle a lot of the compilation steps, such as loading TypeScript sources, gathering dependencies, and outputting the final assets. So we set off to build out a Webpack-powered build system. One problem: At the start of this project, I had never built a plugin for Webpack! I had very little knowledge about Webpack outside of some fighting with configs to get my projects to build correctly. Not only that, we weren't using webpack in one of the more common ways, so finding documentation and examples was difficult.
The Plan: Build an email compilation engine
For our email build system we needed to do a few things:
- Build one html email for each template, for multiple languages
- Build zero or more variant templates for A/B tests, each of which may also have custom translation strings
- Include metadata information about each template and include that in the build output (useful as a contract with server to know what to expect to render)
The goal is to end up with a build system that looks something like the image above. All components will be built on React/MJML with some JSON metadata. Webpack will drive compilation and take that React code and data to build the files and we will end up with static HTML and some other necessities like metadata and plaintext versions of the emails. Most of the work will happen in a custom built Webpack plugin which we call ReactMJMLWebpackPlugin. This plugin will handle a few different phases of compilation:
- Loading email template properties
- Evaluating React and rendering to MJML/HTML
- Outputting additional assets
- Render for each supported language
And with that, we have enough high level info to get started. Here on out we'll be going deep in the technical details of setting up a Webpack plugin to drive the build system including lots of code examples and config files! First, a detour to explain what a Webpack plugin is, for the unfamiliar.
The Webpack lifecycle: A quick primer
A simple outline of how Webpack does this:
- Define entry points, where Webpack starts
- Specify loaders, which define how to load and transform various files
- Integrate plugins, which customize how each file is compiled and bundled
- Define output, where the final bundled assets are exported
I recommend reading the Webpack Concepts page in their documentation for a bit more in depth outline of how webpack works from a conceptual level.
Before getting into the cool stuff, there's some setup required to get Webpack to work with our stack. This is probably the area most developers who use Webpack are familiar with. Most projects just need to use existing plugins and setup a few config options to get up and runnning.
In our project, we want to use TypeScript so we'll need to add a TypeScript loader called
ts-loader. And we want all our builds to output to the
export directory. Let's also add in
file-loader and use it on images. This will load all images with their absolute location on the local disk so we can import images and use them in our React components with ease. In production you'd want to specify the URL of your file hosting here or in the case of a web app, use something like
url-loader to load Base64 images inline. For testing emails locally in the browser however, a file url works just fine.
This gets us rolling with TypeScript and React! Toss a
main.tsx file in the codebase and it'll generate a
main.js file just fine! However, we're not building a single typescript file, we'll need to build for a large number of email templates!
Entry points are where Webpack starts building out it's dependency graph.
The simplest example of this would be a single JS file that is the sole bundle for a Single Page Application, where all imported dependencies are loaded into this main entry point.
If we wanted to generate multiple bundles, we would add new entry points. For example a static website might contain a single bundle for each page. The general rule of thumb is "one entry point for each HTML document".
In our case we decided to treat it like a static website since we have many templates each with their own build output and each generates a different HTML page. Since we have potentially multiple variants for each template, each A/B test variant and the control will all be their own entry point. For localization we just insert different content based on language, we'll handle that in the plugin itself, which we'll touch on later.
With this config we actually get pretty far towards our goal! Webpack takes our entry points and makes a new dependency graph for each. We'd be able to use React and import any other dependencies and images. Unfortunately we're still just generating a
.js bundle for each template. That might work well for a website, but we want fully build static HTML with no JS to use in emails.
Building the React MJML Plugin
Our plugin is going to need to handle a few things:
- Load and validate email properties (per template)
- Render each template component from React to MJML to HTML (one per variant)
- Render each HTML output to a plaintext version
- Output a metadata file (subject, from email, etc)
Structure of a Webpack plugin
To build a webpack plugin, make a new class that has a method
apply that takes a compiler object.
From the apply method, we have access to the entire Webpack compiler lifecycle. Super cool! But also super complex! The Webpack compiler does a lot. It builds the dependency graph, loads source files, optimizes bundle output, and a lot more. For this plugin we're going to focus on a few specific things: loading properties files not normally picked up in the dependency graph, replacing the JS bundle output with static html, rendering extra output (plaintext email, metadata file).
In the apply method, we hook into different stages of the lifecycle. Here's a simple example, hooking into the compilation object, then again into the "optimize assets" stage of compliation.
This example will print "Compilation object created!" then list each of the assets. In our case this will be the already compiled JS bundles, since optimizeAssets occurs after the modules have been bundled, and will look something like:
Notice that the asset keys are actually the files that will be outputted, and they are
.js! that's because we transform the tsx files using
Phase One: Loading email template properties
For our email templates, even though we're using this new build process, we actually still have more templating to do once the emails are built! We use EJS inserted inside
<mj-raw> tags so that the final output of our build can still have variables inserted, such as a user's name or a dynamic link generated on the server before sending. We have a properties file we use to define this contract with the server as well as A/B information and i18n language data. So we need to load that file, parse it's data, and validate we have everything we need.
The main question now is: Where do we do this? What part of the Webpack lifecycle should handle this?
We actually aren't importing the properties json file into each component, so Webpack won't be automatically adding it to the bundle output.
Some solutions considered:
- Import the properties file, just to export it within each template
- Add the properties file to entries
- Dynamically modify the file dependencies in Webpack plugin to include properties files
- Load the properties file within the plugin, add to assets manually
Option 1 required developers to manually add extra code to each template, whereas Options 2-4 handled the properties file automatically. Option 2 felt like it was a misuse of entries, breaking the best practice of "one entry point for each HTML document" (more on that in Addendum on extra dependencies). And Option 3 felt just as hacky as Option 4, but knowing adding manual assets wouldn't have any side-effects, we opted to just load properties manually using node's
The initial thought was to add this to
additionalAssets, but we also wanted to use this data to validate the components. Also we were unsure if
compilation.assets had the full list of assets at this point, whereas
optimizeAssets did. Since that's where we were doing the rendering (detailed in the next section), we simply added this file loading there.
One thing we'd really like to do is not assume that every JS file is a template and instead flag each one with metadata, but there doesn't seem to be a clean way to do that in Webpack. What we can do, which is how we've built the production version of this, is exclusively store all templates in a
templates/ directory, so we know all the JS files we find will be the template files, and only look there when doing transformations in the plugin. That process has been left out for the scope of this post.
Phase Two: Evaluating React and rendering to MJML/HTML
Note: We do this phase and the last phase in
optimizeAssets, admittedly not a great place to hook into to do this, but there's not really a better place to do so. Webpack v5 introduces a new hook called
processAssets that has stages that are made specifically doing what we're doing here where we can add extra assets and made assets from existing assets. We're looking to do a refactor now that Webpack v5 is released.
We're going to take our bundled React components and run them through eval (can be done using node-eval or jsdom). We then render to static markup using React DOM Server. That'll give us some static mjml content. MJML has it's own renderer, it's a simple function called
mjml2html, so we'll call that next.
Note: A word of warning! You don't want to use eval client-side, we only ever run eval as part of the build process of our own React components since we only have them as bundled JS as strings. See: Never use eval!
Our render looks like: Raw JS string to React component to MJML to HTML.
Fantastic! now we have a bunch of new html assets that look something like:
These will all get taken by the final steps of the webpack compiler and give us some lovely static HTML emails rendered out to our export directory!
Phase Three: Outputting additional assets
A few additional things to do before we wrap up. That solution is great but we also want to automate a few things. One is our plaintext emails. Here we simply piggy back off the asset definition we did at the end for the HTML and add a new step.
Phase Four: Handling i18n
Finally, we want to handle translations with ease, so we have a bunch of i18n keys at the ready for all text content inside the emails. The way we handle this is simply to loop over each language, and render our output one extra time for each language.
Great! No we have a solid build pipeline all powered by Webpack in one custom plugin. For a high level view of how this plugin is structured take a look at this diagram which shows what different compilation process we hooked into and what steps we took to build each piece of the build system puzzle. There's plenty more to do, and we've built a bunch more on top of this system, like CLI tools for generating and sending emails, reusable components, and more. This should hopefully demystify Webpack plugin development and also show how to tackle some of the less straightforward setups that Webpack can handle, but is lacking in examples or documentation.
There were a few tips and implementation details left out of the main part of the post since it would have complicated the example code a bit too much. We still wanted to include them as they are useful to know for anyone stumbling through Webpack plugin development.
Uncaught errors may cause Webpack to crash, meaning that it won't rebuild automatically when the errors are thrown. You can surface these errors as compilation errors by catching them and adding them to the
compilation.errors array, rather then letting them bubble up. We made a little pretty print function to make this clear and encourage our plugin maintainers to provide helpful hints to the other devs!
Any entry file and it's dependencies will be automatically watched by Webpack when you run a watch build. Unfortunately since we do some parsing and file access outside of these dependencies, changes to files like i18n data will not rebuild webpack in dev mode.
Luckily you can add to fileDependencies manually, and Webpack will properly watch those files like any other dependency!
Addendum on extra dependencies
While writing this article I found Advanced entries and feel that would be a cleaner way to do Phase One Option 2 ("Add the properties file to entries"), and probably the best solution. I explored a refactor that uses this instead, but Webpack didn't handle the extra entries very well, and screwed up the way modules were bundled, breaking our React eval step (will go over that in the next section), and I ended up having to ditch the idea.
I wouldn't have made it through this without a few really helpful resources and open source projects:
- Inspiration from Mark Dalgleish's static-site-generator-webpack-plugin
- Official docs for Writing a Plugin
- Webpack Compiler Hooks documentation
- Webpack Compilation Hooks documentation
- Sean Larkin's Everything is a plugin! talk