About four months ago, our API tests took about 12-15 minutes to run on a decently powered linux machine. Test runtimes on Macs were slightly slower even with comparable hardware. Today it takes us 1 minute and 55 seconds to run the entire test suite including setup & teardown saving us ~50 hours of development time every week.
The number of tests we run for API has more than doubled (from ~2000) since we last talked about it on our engineering blog
At ClassDojo, we're working on building amazing classroom communities at scale. To succeed in this mission, we have to run lots of product experiments, and just as importantly, accurately measure their results. In this post, I'll outline how we built our A/B testing system, and how we used it to run one of those experiments.
We build our software with a focus on Continuous Integration and quick deployment lifecycles. Naturally, speed is a critical component, allowing faster iterations and quick feedback.
Going from a 20 minute build to a 11 minute build time on our CI servers makes a dramatic difference in day-to-day work on our android app. The quick build time helps us stay focused and sharp. We've still got a long way to go until we get to a build time we're happy with, but we've made some good steps in the right direction.