Manipulating Text & Files solves problems

A decent number of programming tasks boil down to manipulating text and files:

• automating code migrations
• exporting & transforming data
• exploring logs
• looking for patterns in a codebase

Knowing the basics of shell tools that are designed to manipulate text & files can make these tasks simple, and the interactive pipe-based programming environment of the shell makes figuring out the right command fun. It makes sense that tons of engineers evangelize learning tools like sed, awk, ag/rg, find, cut, head, tail, xargs, jq, and all of the other amazing shell tools that are designed for dealing with streams of text and files.

That said, while shell tools are useful, it is always possible to accomplish the same tasks with whatever programming language you regularly use! As long as you have the ability to easily manipulate text and files in a script, you'll find that it can make entire classes of tasks easy to accomplish. If you aren't comfortable manipulating text and files like this, you won't even notice the potential opportunities you have to make your own job easier, and to provide important business value.

At ClassDojo, we have people who use Bash, Go, and NodeJS for these scripts. All of these languages work well! Some of the high-value text & file manipulation scripts we've written have done things like:

• automatically rewrite require statements in our tests to allow converting our tests to Typescript (Bash)
• do a large code migration to allow targeted detection of changes to fixtures & indexes (NodeJS)
• set up a MongoDB export job (NodeJS)
• set up product-areas for all of our HTTP routes to allow better team ownership over errors (Bash)
• set up fast local database restores (NodeJS and Bash)
• parallelize tests (Go)
• parallelize & cache lint steps (Bash)

Shell scripting can be difficult to get into, but it doesn't mean that you can't start getting the same value out of whatever language you regularly use. Getting comfortable with text manipulation, file manipulation, and regular expressions is incredibly useful no matter what tool you choose to get there.