Engineering Dojo, Episode 1: Building Payment Systems

As you might know, ClassDojo offers a paid subscription for parents, to give them extra tools for interacting with both their teachers and their kids. For several years, we built and supported a system for managing those subscriptions. That’s non-trival work: it involves integrations with several app stores and payment providers and wrangling different data models for each.

In the last year, we’ve switched to a third-party solution: RevenueCat. It’s not easy switching payment systems when you’ve got several years of data in the old system, and need to keep it running until the new integration is ready to go.

I recently sat down with Urjit and Sarah, two of the engineers who led this project, and asked them about the challenges they faced and the lessons they learned as they replaced the wheels of our bus without making a pit stop. You can listen to the conversation below:

Listen to Episode 1, Building Payment Systems

Note: we're still in the process of setting up a podcast feed, so stay tuned for that!

  • Engineering Dojo Podcast

Bob Nystrom's What Color is Your Function does an amazing job of describing why it can be painful when programming languages have different rules for calling synchronous and asynchronous functions. Promises and async/await have simplified things in JavaScript, but it's still a language with "red" (async) and "blue" (sync) functions, and I consistently see a few understandable errors from red vs. blue function confusion. Let's go through some of the most common mistakes – none of these are bad things to get wrong, they're just a symptom of how confusing this can be!

Omitting await from try/catch blocks

The most common mistake I see is omitting await from try/catch blocks with async functions. The code looks reasonable, but the catch block will only be able to catch synchronously thrown errors. To make matters worse, error handling logic is often less well tested than the happy path when everything works, which makes this pattern more likely to sneak its way into production code.

async function throwsError () {
  throw new Error("alas! an error");
}

try {
  return throwsError();
} catch (err) {
  console.error("Oh no! This catch block isn't catching anything", err);
}

An async function that throws is the equivalent of a Promise.reject, and when written that way, it's a bit clearer what's going on:

try {
  return Promise.reject(new Error("alas! an error"));
} catch (err) {
  console.error("It's clearer that this `catch` can't catch that `Promise.reject`. This is equivalent to the earlier code");
}

Personally, I'm starting to wonder whether using try and catch blocks at all is a mistake when dealing with async code. They take up space and don't offer the same pattern matching that a library like Bluebirdjs can add to catch when you only want to catch some specific known errors: await tryThing().catch(NotFoundErrorClass, handleErrPattern) feels substantially cleaner to me than the equivalent try/catch block.

Array.filter(async () => false)

In recent years, JavaScript has added lots of useful Array methods like filter, map, forEach, and flatMap, and JavaScript programmers often use libraries like lodash to write functional code rather than writing for loops. Sadly, none of those Array methods or lodash helpers work with red async functions and are a common source of coding errors.

const things = [true, false, 1, 0, "", new Date("not a date") - 0];
const filteredThings = things.filter(async (thing) => thing);

How many things do we end up with in filteredThings? Surprisingly, the answer has little to do with JavaScript type coercion: filteredThings will be the same size as things. An async function returns a Promise and even a Promise that resolves to false is still a truthy value: Boolean(Promise.resolve(false)) === true. If we want to do any sort of filtering using an async function, we need to switch out of blue sync mode and into red async mode.

(async function () {
  // You should use a library like Bluebird rather than filtering like this! this is only for illustration
  const things = [true, false, 1, 0, "", new Date("not a date") - 0];
  const predicateValues = await Promise.all(things.map(async (thing) => thing));
  const filteredThings = things.filter((_thing, i) => predicateValues[i]);
})();

When you see Array.filter(async (thing) => thing) written out like that, the mistake is pretty clear. It can be harder to notice when you see code like const goodThings = things.filter(isGoodThing); you need to check whether isGoodThing is red or blue.

Array.forEach(async...

We see a similar problem when people use Array.forEach with an async function:

const fruitStatus = {};
["apple", "tomato", "potato"].forEach(async (food) => {
  fruitStatus[food] = await isFruit(food);
});
return fruitStatus;

In some ways, this is a more dangerous pattern. Depending on when you check, fruitStatus may have some, none, or all of the correct isFruit values. If isFruit is normally fast, problems and bugs might not manifest until isFruit slows down. A bug that only shows up some of the time is much harder to debug than one that's always there.

Await off my shoulders

Despite how easy it is to make mistakes with async/await, I still love it – it feels easier to work with than Promises or callbacks. Dealing with asynchronous code is still one of the harder parts of programming in JavaScript, but tools like bluebird, the TypesScript no-unnecessary-condition rule, and the eslint promise plugin can help surface these easy-to-make red/blue function mistakes early. Hopefully, seeing the mistakes we often make will help you avoid some frustrating minutes debugging.

    Since the dawn of the internet, software engineers have found ways to effectively collaborate remotely, and over the years we've become quite good at it. If you need convincing, take a look at any large-scale open source project, and you'll find that it was likely created by people from different countries, time zones, and languages, all collaborating on a single project and working toward a shared vision.

    The fact that developers can collaborate successfully no matter where they’re located shows that coders don’t need to be tied to a specific office or time zone. Unfortunately, until recently, when the pandemic forced their hands, many companies (especially large companies) tended to frown on remote work. Full-time remote positions did exist, but they were far less common than on-site positions. And even when you found these positions, they were typically only for contract work.

    Enlightened employers now believe that people should be free to work where they’re most productive. For many of us, that's from home, but it could just as easily be from a coffee shop down the street, on a beach, or even in an RV, nestled among trees deep in the forest.

    I’ve always dreamed of being a nomad, of sailing the ocean and seeing the world, but I always found myself stuck in a particular place. Family, school, work, etc. — all were rooted to a specific geographic location that I didn't venture far from. Once I purchased a home, that became my base of operations. Trips away from home had to be planned in advance, and were usually expensive (flight, hotel, rental car, etc.). I ended up filling my home with stuff, as one does, and I found that the more stuff I kept there, the more tied down I felt ... but I yearned to be free.

    After a lot of research, I convinced myself that a software engineer like me could live and work nomadically while raising my three young children as a single parent and not go crazy (or go crazier?). In the span of a few months, I sold my house and all of the stuff that was previously tying me down, purchased an RV, moved in, and hit the road.

    I’ve been living and working full-time in an RV for a year now. Along the way, I upgraded to a larger RV, purchased a tow vehicle, and have learned many things about life on the road and being a digital nomad.

    My home is a Thor Challenger, which is a large motor coach. I tow a Jeep Wrangler, which I use as my daily driver and for going places where the coach can't. I no longer feel tied down, and I can live and work wherever the road takes me — and let me tell you, it can lead to some pretty amazing places.

    My life now is so different from what it was before, it's hard to compare things directly. Inevitably, my lifestyle has presented unique challenges, but they say each challenge is an opportunity in disguise, and the fact that I'm finally living the adventure I always dreamt of makes each challenge seem like a small bump along the journey.

    Staying connected on the road

    One of the first challenges I faced was figuring out how to have reliable internet while on the road and at campsites, regardless of where I was staying. To stay connected, I first need to know where I plan to be during the work week. I usually don't travel much Monday through Friday, which makes planning easier.

    Before I travel anywhere, I check the cell phone signal strength for an area by looking at coverage maps so that I know what internet options will be available to me once I'm there. OpenSignal.com is the best website I have found so far for checking cellular signal strength in different areas.

    If I'm staying somewhere with a strong cell signal, I use my phone hotspot as my primary internet. It's not terribly fast, but it's reliable and unlimited thanks to my plan through Visible. While this plan is affordable (only $50/month), unfortunately it’s for mobile devices only, and can't be used for dedicated hotspot devices.

    If I'm outside of town or I need a fast connection, I use a Winegard ConnecT 2.0 4G2 system as my primary internet connection. The Winegard system sits on top of my RV and pulls in even the faintest 3G or 4G signal thanks to its high-gain antenna. Once a signal is captured, it's rebroadcast via three Wi-Fi routers spread throughout the coach. A hotspot-specific data plan is required to use this device, and many hotspot plans have 100GB data caps, so I use the Winegard system sparingly.

    If I'm staying at a place with Wi-Fi, the Winegard can use that signal as its source instead of cellular. Since Wi-Fi at RV parks and campgrounds is notoriously spotty, having a device that can rebroadcast a weak signal is essential for staying connected.

    Advice from an experienced nomad

    Nomadic life is increasingly popular. Younger generations are drawn to it as an alternative to the expensive housing and rental markets, and older generations are drawn to it as a way to have a life of adventure while living comfortably within their means during retirement. New products and services are becoming available to support digital nomads. For example, satellite internet (Starlink in particular) is becoming more viable as a mobile internet solution, and may replace cellular-based hotspot devices. For more information on the latest in mobile internet, RVMobileInternet.com is the best resource I have found.

    If you’re thinking about life as a digital nomad, I hope that hearing about my solutions to staying connected will help you to do the same, no matter where the road takes you. And as always, feel free to reach out to me if you have any questions. Happy travels!

      Newer posts
      Older posts