Engineering

Scalable, Replicated VPN - with Pritunl

VPCs are a pretty ubiquitous piece of cloud infrastructure for most tech stack these days. It is an important to get the IP address layout of your Virtual Private Cloud (VPC) right from the start. With far-reaching implications for scaling, fault-tolerance and security, a VPC is a great tool to separate out the wild-west of the web and your precious servers. However, once you have a VPC, you’ll need the ability to access your VPC protected servers. This is where VPNs come in.

Our VPN journey

ClassDojo used the OpenVPN offering for quite a few years but we are now running using Pritnul as our VPN implementation. Our OpenVPN setup had a pretty standard bastion instance setup - a box that sits on the edge of the private and public networks. Bastion instances have a network interface with an IP address in your private VPC and one exposed to the public internet. This lets clients of the VPN connect via the public IP address and then tunnel their way into the private network space - with the proper auth of course.

More recently, we’ve moved to Pritunl. Pritunl itself builds upon the OpenVPN protocol but offers a better UI for managing servers and users. Pritnl uses Mongo to store the user registration and configuration details. Our try run on Pritunl was pretty successful.

Here's what we learnt from our trial run

  • Installing and maintaining the Pritunl instance was pretty straightforward
  • Their client support is pretty good
  • Their support for Google auth was a big plus
  • This might be a small thing but being able to point users to the Pritunl public website to self-serve download client profiles made onboarding easier for us

Scalability

Pritunl features scalability out of the box via Mongo. Once we decided to move out of the trial phase, we wanted to set up multiple instances so that in case an instance goes down, our employees can continue having a seamless connectivity experience. Pritunl communicates with replica instances via MongoDB.

Our recipe for setting up a scalable replica for Pritunl:

  • Create a MongoDB instance using MongoAtlas
    • PS: We tried using the AWS managed Mongo but it doesn’t implement some critical features like tailable cursors or capped collections which are required for Pritunl app functionality.
    • Setup VPC peering between MongoAtlas and your local VPC if it doesn’t exist
  • If you are migrating from an existing Pritunl setup like us and don’t want your employees to recreate client accounts with Pritunl, you can use mongo-dump and mongo-restore to move the existing Pritunl data into the new Mongo Instance. Make sure you pause the Pritunl instance during this time to avoid data changes while you’re migrating to the new Mongo instance.
  • Point your Pritunl instance to the new Mongo Atlas endpoint and restart it.
  • Once Pritunl is up and running, verify that you’re able to connect and use the VPN functionality as before.

Now, if you want to scale up your Pritunl cluster, you can simply start a new server instance, install Pritunl and point it to the same Mongo endpoint. From within the Pritunl interface, remember to create a new host for the new instance and connect it to the same virtual Pritunl server instance

If you want to look under the hood and verify that everything is working correctly, you can inspect the detailed configuration from within your Pritunl client and check that multiple public IP addresses show up - one each for your Pritunl instance. It'll look something like this:


remote <Public IP address 1> <port> udp
remote <Public IP address 2> <port> udp
remote-random

That’s it - this creates a scalable, replicated VPN solution based on Pritunl.

  • Engineering
  • Programming
  • Networking

ClassDojo has a fleet of services that work together to serve user requests as well as complete other data operation tasks. Most of these services are set to auto-scale to better utilize our resources. Part of doing this involves synchronizing a lot of metadata like tags and service ports to keep our configurations up to date. We started thinking about how we could improve distributed runtime changes to our infrastructure-wide config and service discovery as servers join and leave the fleet.

Continue reading

About four months ago, our API tests took about 12-15 minutes to run on a decently powered linux machine. Test runtimes on Macs were slightly slower even with comparable hardware. Today it takes us 1 minute and 55 seconds to run the entire test suite including setup & teardown saving us ~50 hours of development time every week.

The number of tests we run for API has more than doubled (from ~2000) since we last talked about it on our engineering blog

Continue reading