At ClassDojo, we're working on building amazing classroom communities at scale. To succeed in this mission, we have to run lots of product experiments, and just as importantly, accurately measure their results. In this post, I'll outline how we built our A/B testing system, and how we used it to run one of those experiments.

We build our software with a focus on Continuous Integration and quick deployment lifecycles. Naturally, speed is a critical component, allowing faster iterations and quick feedback.

Going from a 20 minute build to a 11 minute build time on our CI servers makes a dramatic difference in day-to-day work on our android app. The quick build time helps us stay focused and sharp. We've still got a long way to go until we get to a build time we're happy with, but we've made some good steps in the right direction.

At ClassDojo, our frontend web apps are large single-page applications running in React and Redux. Millions of teachers, parents, and students use these apps every day, and we want to ensure a quality experience for them. However, effectively testing frontend apps is hard.

We have a good number of unit tests, but found that apart from a few complicated components with lots of stateful interactions, these tests are tedious to write and usually ineffective at preventing bugs. This is because most of our frontend bugs have occurred in the boundary between components or layers of our app - for example, between the API and the data layer, the data layer and our components, or between a parent and child component. These boundaries are explicitly not tested by standard unit tests, which mock out external dependencies. It's always very frustrating to find that an API change or refactor had broken our app, when all of our unit tests continue to pass.

To address this issue, we decided to look into integration testing our frontend apps. The standard way to do this is with a browser automation tool like Selenium. However, these tools are legendary for their difficulty of use and unreliability. Tests written this way are brittle, breaking when class names change or if API calls or renders take too long. We wanted an easier, more efficient method. Luckily, React and Redux provided us the tools to do so.