ClassDojo has a fleet of services that work together to serve user requests as well as complete other data operation tasks. Most of these services are set to auto-scale to better utilize our resources. Part of doing this involves synchronizing a lot of metadata like tags and service ports to keep our configurations up to date. We started thinking about how we could improve distributed runtime changes to our infrastructure-wide config and service discovery as servers join and leave the fleet.

About four months ago, our API tests took about 12-15 minutes to run on a decently powered linux machine. Test runtimes on Macs were slightly slower even with comparable hardware. Today it takes us 1 minute and 55 seconds to run the entire test suite including setup & teardown saving us ~50 hours of development time every week.

The number of tests we run for API has more than doubled (from ~2000) since we last talked about it on our engineering blog