One aspect of ClassDojo’s engineering culture that tends to surprise people is that we don’t have dedicated QA engineers or testers. We use a variety of strategies to keep our end-to-end feature development fast, continuous, and high quality. These strategies include a strong culture of writing automated tests, along with regular playtesting sessions.
What do we do instead of QA?
At ClassDojo, we prioritize truly continuous deployment of our backend monolith, which means removing as many manual blocking steps as we can, and we’ve built systems to do so safely. Engineers have end-to-end ownership of features, and we rely heavily on writing automated unit and integration tests to ensure that we’re not pushing code that causes unexpected behavior or errors. Our canary containers help automatically roll back any commits that cause errors, and we monitor our logs carefully.
However, as thorough as we try to be, automated tests and monitoring can’t catch everything. To supplement our automated systems, we hold regular playtesting sessions to serve as manual testing as well as context sharing.
What is playtesting?
Playtesting is simply manually testing the products and features the team has built. You may have also heard of the term “dogfooding,” which refers to actually using your product as a real user. Some folks on the team are able to dogfood as legitimate parent users since their kids’ teachers use ClassDojo. However, since the majority of us are not teachers in a classroom nor parents connected to a real classroom, we do intentional playtesting sessions instead.
How do we run a playtesting session?
Who should join?
Individual teams might playtest new features right before shipping, in which case the whole team should join. We also schedule sessions that are open to anyone at the company so teams can share their recent features and solicit feedback from the wider group. For the best insight and context sharing, we encourage including folks from various functions, such as engineers, designers, PMs, marketers, and customer success agents. Playtesters don’t need to prepare anything ahead of time, or even have any knowledge of the flows. They just need to have the right type of device if it’s a mobile feature that’s only released on one platform.
The process
The first step is scheduling a time for the group of testers to get together and play through certain flows. Usually 30-45 minutes is sufficient, depending on how large or complex the feature is. Ahead of time, the product owner should prepare a set of loose guidelines for how to run through the flows, including any necessary setup steps such as installing an alpha build or turning on certain feature flags.
At the beginning of the session, the facilitator or product owner gives a brief overview of the flows and setup, then all the playtesters simply go through the flows independently on their own devices and accounts. Doing this synchronously instead of async lets us quickly identify whether an issue is widespread or a particular edge case, and answer any questions on the spot. We typically use an Asana board where playtesters can add cards for anything that comes up — not just bug reports but also general product feedback and points of confusion.
We reserve 5-10 minutes at the end of the session to go through the cards and make sure the issues are clear. From there, the product owner and team can prioritize them at their next prioritization meeting.
What are the benefits?
Our playtesting often finds bugs in obscure edge cases not covered by automated tests. With an app and user network as complex as ours, it’s nearly impossible to cover all use cases with test fixtures, so it helps to have a variety of people testing on their own real-world accounts.
Playtesting is one of our best methods for cross-team and cross-functional context sharing. Screenshots, product specs, and demos can only go so far in conveying what a new feature really involves. Having teammates actually get their hands on the features is a great way to share what’s being built. It’s also valuable to get fresh perspectives from folks who perhaps hadn’t explored that product area before. It’s like having an in-house focus group to give feedback.
If you have ideas on how we can improve our playtesting or manual testing strategies, reach out! We’d love to hear from you.
Our teams at ClassDojo have the freedom to choose how they want to work. Many of our teams have started spending a few hours each day mobbing because we've found it to be an effective form of collaboration. Here's how we do it!
What is Mob Programming?
Mob programming is similar to pair programming, but with more than two people working together. One person, the driver, does the actual typing but everyone is involved in the problem solving. Mob programming is often defined as “All the brilliant minds working on the same thing, at the same time, in the same space, and at the same computer.” We don’t follow the strict definition of mobbing, especially since we are a fully remote team, but we are continuously iterating on an approach that works for us.
Why do we mob?
Woody Zuill has a great writeup about how a whole range of issues just faded away once his teams started mobbing, including fading communication problems and decision making problems, without trying to address those issues directly. We’ve found similar benefits, and I’ll call out just a few:
Focus
When the team is working together on a single task, it means we’re focused on the top priority for our team. Although it may sound more productive to have multiple engineers working in parallel on separate tasks, that often means that the top priority is delayed when waiting for answers to questions. Having the whole team focused on the same thing greatly decreases the amount of context switching we need to do.
Knowledge Sharing
Without mobbing, it’s easy to develop silos of knowledge as individuals become experts in specific areas. Others might gain context through code reviews or knowledge sharing meetings. However, when the whole team works together on a piece of code, it almost eliminates the need for code reviews since all the reviewers were involved in writing it, and everyone already has shared knowledge. Mobbing is also really useful for onboarding new teammates and getting them up to speed.
Quality
More time is spent debugging and refactoring code than writing it. If you mob, you have more eyes on the code while it’s being written, rather than during code review or later when it needs to be updated or refactored. You increase the quality of your output, and that quality increase leads to long-term speed.
Collaboration
Especially with a fully remote engineering team, it can be isolating to only work on individual tasks. There is also the challenge of communication and having to wait for answers to blocking questions. By having everyone attend the mob, we eliminate that waiting time. Questions can be answered immediately and decisions are made as a group.
What does remote mobbing look like at ClassDojo?
Who: Most often, we have all the engineers of the same function (e.g. all the full-stack engineers) on a team join a mob. Depending on the task it can be helpful to have other functions like client engineers or product managers join as well, to quickly answer questions and unblock. The group will naturally include engineers of varying skill levels, which is a good thing! We rotate drivers often, but like to have the less experienced engineers drive as it keeps them engaged and learning.
When: This depends on the team’s preference and availability as well as the nature of the task, but we may schedule mobbing time for anywhere from an hour to almost the entire day, most days of the week. It’s important to block the same time off on each person’s calendar and protect that time from other meetings. During longer sessions, we set a timer to remind ourselves to take breaks often. We generally take a 10-15 minute break after every 45 minutes of focused mobbing.
What: We pick one task to focus on, and it should be the highest priority task for the team. It’s easy to get derailed by PRs that need reviewing, bugs that get reported, questions on slack, etc, but we make a conscious effort to avoid starting anything new until we finish the current task. The one exception we have is for P-now bugs, which we drop everything else for.
How: No special tools or complex setup required! We simply hop on a Zoom call and the driver shares their screen. If we’re coding, the driver will use their own IDE and when it’s time to switch drivers, the driver pushes the changes to a branch so the next driver can pull the latest. There are tools for collaborative coding, but we’ve found that they don’t offer much benefit over simply having someone share their screen. If we’re in a design phase, we often use Miro as a collaborative whiteboard.
As with everything we do, we have frequent retrospectives to reflect on what’s going well and what could be improved with how we mob, and we are open to trying new ideas. If you have any thoughts, we’d love to hear from you!