At ClassDojo, our frontend web apps are large single-page applications running in React and Redux. Millions of teachers, parents, and students use these apps every day, and we want to ensure a quality experience for them. However, effectively testing frontend apps is hard.
We have a good number of unit tests, but found that apart from a few complicated components with lots of stateful interactions, these tests are tedious to write and usually ineffective at preventing bugs. This is because most of our frontend bugs have occurred in the boundary between components or layers of our app - for example, between the API and the data layer, the data layer and our components, or between a parent and child component. These boundaries are explicitly not tested by standard unit tests, which mock out external dependencies. It's always very frustrating to find that an API change or refactor had broken our app, when all of our unit tests continue to pass.
To address this issue, we decided to look into integration testing our frontend apps. The standard way to do this is with a browser automation tool like Selenium. However, these tools are legendary for their difficulty of use and unreliability. Tests written this way are brittle, breaking when class names change or if API calls or renders take too long. We wanted an easier, more efficient method. Luckily, React and Redux provided us the tools to do so.
Difficulties
The primary difficulty when writing integration tests for web apps is that the browser provides only a limited set of options for interacting with your app. Unit tests can mock out dependencies, wait for promises to resolve, and inspect internal app state as needed, but browser automation tools can only access what appears on the screen.
For example, consider the process of testing the signup flow - ensuring that at each step, the correct information is displayed on the screen. To properly test this flow, we would like to be able to inspect what has been rendered at several points: the initial state, the state after inputting information, the state after the user clicks “sign up”, and the state after the request succeeds or fails. If the only input we receive is what displays on the screen, it's hard to know what assertions we should make at which times.
Our method
There are a few components of our integration tests: initializing and rendering the app, tracking API requests, writing multi-stage tests, and managing the router. I'll review each of them in turn.
Enzyme, Mocha and JSDom
AirBnB has created the excellent testing tool Enzyme to make assertions about React components. In our case, we mount our entire React app in Enzyme using its mount method, which fully renders a tree of React components and properly registers component lifecycle hooks and event handlers. We can then use Enzyme to traverse the rendered tree and make assertions about what's present.
We create and mount a new app instance before each test, to remove any accumulated state. Luckily, Redux makes this very easy, since it's standard practice to create a new store as part of your app root component's creation. However, if your app has any singleton modules, you'll need to reset them manually.
Enzyme's mount method requires that window and document objects be available in the global scope. While this can be accomplished by running your tests in a real browser (using Karma and PhantomJS, for example), this would require running Webpack to compile and build our application before every test run. This is cumbersome and time-consuming.
Instead, we use JSDom to create a mock document, and run our tests in Node using Mocha. Note that we don't actually mount anything in our mock document; it's simply there to provide document and window objects. And babel-register allows Mocha to run the same compiled code Webpack will bundle into our app. While there are some differences some between node and the browser even after a pass through Babel's compiler, Webpack papers over most of them, and clever use of if (__TESTING__) blocks can handle the rest. So far, we haven't had any major issues from test/browser environment inconsistencies.
Sample initialization code for Enzyme and JSDom is below:
global.document = require("jsdom").jsdom("<!doctype html><html><body></body></html>");
global.window = document.defaultView;
global.navigator = global.window.navigator;
// Both these must be required after a DOM is initialized.
const mount = require("enzyme").mount;
const React = require("react");
Tracking API Calls
As mentioned above, waiting for the results of asynchronous operations is a common difficulty when writing frontend integration tests. Our solution is to wrap all of our app's API calls with code that allows us to hook into their completion. The code to do this looks something like this:
let callsInProgress = 0;
let callbacks = [];
// When making API calls, this function should be called instead of directly
// calling `window.fetch`, `superagent`, or whatever other library you use.
export function makeTrackedApiCall (params) {
callsInProgress += 1;
// assume this delegates to your API call library.
return makeApiCall(params).finally(() => callsInProgress -= 1);
}
// This function can be called in your tests in order to wait for all API calls
// to finish before continuing. If no API calls are in progress, it immediately
// returns a resolved promise. Otherwise, it returns a promise that will be
// resolved after all calls have finished - see below.
export function waitForApiCallsToFinish () {
if (callsInProgress === 0) {
return Promise.resolve();
} else {
return new Promise((resolve) => callbacks.push(resolve));
}
}
// In a standard redux app, the results of any API call will be dispatched
// in an action. Using a middleware similar to the one below, we can check
// after each action is dispatched to see if we've finished all API calls.
// We mount this middleware only in testing.
export function resolveAPICallbacksMiddleware () {
return (next) => (action) => {
// After this line, the store and views will have updated with any changes.
// If we don't have any API calls outstanding at this point, it's safe to
// say that we can continue with our tests.
next(action);
// `setTimeout` is in case the any code that runs as a result of this
// action dispatches another API call.
setTimeout(() => {
if (callsInProgress === 0) {
callbacks.forEach((callback) => callback());
callbacks = [];
}
}, 0);
}
}
mocha-steps
Integration tests are often long, requiring a series of steps that each depend on previous results. To make writing these tests cleaner, we've used the mocha-steps library, which allows you to write tests that will halt if any step fails. An example test would look like this:
// this imports `step` as a global, similar to `it` and `describe`
import "mocha-steps";
describe("My test", function() {
step("My first step", function() {
// Do stuff, assert stuff. Can return a promise for async steps.
});
step("Oops!", function() {
throw new Error("Something went wrong.");
});
step("This step will never be run", function() {
// ...nor will its output be displayed in the console.
});
});
Routing
If you're using React-Router to control your app's navigation, congratulations! Page transitions will work just fine. If you want to set the page manually (for example, at the beginning of a test), you'll want to do the following:
import {browserHistory} from "react-router";
browserHistory.replace("/myPage");
Note that React-Router’s API has changed substantially between versions; we’re on v3, so if you’re using a different version, method names may be different.
The API
We manually start a local instance of our API before starting the test suite. An instance is also kept running on our CI server. Before each test, we make a special request to the API that causes it to reset its testing database fixtures.
Putting it all together
We mount our app using Enzyme, Mocha, and JSDom, setting the initial route using react-router and resetting our API's database before each test. We execute a series of test steps using mocha-steps. When a step needs to wait for an API call to finish, it simply calls the waitForApiCallsToFinish function and waits for that promise to resolve. We use Enzyme to trigger events and make assertions about the what’s being shown.
Example test
Here's an example test - in this case, for a user signing up. The code is relatively long, but quite easy to read and write.
import "expect.js";
import "mocha-steps";
import {browserHistory, getCurrentLocation} from "react-router"
import getAppRootInstance from "app/root";
describe("Signing up works", function () {
step("Set page to signup", function() {
// We do this before initializing the app, which simulates the user
// entering this URL into the address bar.
browserHistory.replace("/signup");
});
let appRoot;
step("Initialize app", function() {
// Assume this function initializes an instance of the app and returns
// it to you. We can then make assertions about that instance
return getAppRootInstance().then((root) => appRoot = root);
});
step("Assert submit button is disabled", function() {
// We've found making assertions about the props of well-tested low-level
// components is usually fine. However, we consider it bad practice to make
// assertions about higher-level components - instead, make assertions
// about the lower-level things that they render.
const signupButton = appRoot.find("#signupButton");
expect(signupButton.props().disabled).toBe(true);
});
step("Fill in email and password", function() {
const emailInput = appRoot.find("#emailInput");
const passwordInput = appRoot.find("#passwordInput");
emailInput.simulate("change", {target: {value: "peter@classdojo.com"}});
passwordInput.simulate("change", {target: {value: "hunter2"}});
});
step("Click the submit button", function() {
const signupButton = appRoot.find("#signupButton");
expect(signupButton.props().disabled).toBe(false);
signupButton.simulate("click");
})
step("Loading state should be shown", function() {
// At this time, the signup request has been submitted, but has not
// returned yet. We can make assertions about the loading state.
const loadingIndicator = appRoot.find("#loadingIndicator");
expect(loadingIndicator.exists()).toBe(true);
});
step("Wait for signup request to succeed", function() {
// This returns a promise, need to return it to mocha. Once it resolves,
// the signup request will have finished.
return waitForApiCallsToFinish();
});
step("Assert we are on the welcome page and can see the tour", function() {
// The signup request has finished, and all state changes and rerenders
// that result from this are completed. We can make assertions about the
// success state.
const currentRoute = getCurrentLocation().pathname;
expect(currentRoute).toBe("/welcome");
const tour = appRoot.find("#tour");
expect(tour.exists()).toBe(true);
});
});
Shortcomings
While we’ve had great success with our integration tests, we’ve found a few shortcomings that they do present.
First, since we’re using JSDom as our document, we don’t have the ability to test in multiple browsers. Luckily, babel’s generated code tends to work in all browsers that we support (IE9+), and you can use eslint to keep you from using unsupported browser methods.
Second, this approach to frontend testing doesn’t catch visual bugs in your CSS or HTML. A different approach is needed to test visual changes.
Third, non-API side effects can present some difficulties. Our wrapped API module allows tests to wait for calls to complete. However, to test other asynchronous side effects - for example, writing to local storage - we would need to write a similar wrapper that allows us to wait for the effect’s completion.
Conclusions
We’ve gotten a lot of benefit out of writing integration tests for our React apps. They’ve caught a lot of bugs and edge cases before reaching production, without requiring that much additional time to write.