I was listening to a recording of a Twitter Space by Colby Fayock on developer education while looking for resources to help me get better at my new job in the field. It was comprised of a group of other highly skilled developers who also use their skills to help educate others in the field. Since the field of software development is also one of never-ending learning, one of the topics was how to learn new things. It got me thinking about my approach to learning new technologies.

It’s worth noting that this system is NOT for brand new developers, but for those who have at least a basic understanding of programming and can build a project of their own. That said, once you get the basics of development down and understand how to build something, you can easily make this work.

Set up a ‘common’ project

Pick something to build and something you can build over and over again. I call it a ‘common’ project not because it's something everyone has done, but because it is something you can continue to work on over and over again. It helps if it's about a topic you love since you’ll be more inclined to work on it and have a closer personal connection to it.

That said, it doesn't have to be complicated. Even the simplest note-taking app or to-do app when built properly has a lot of moving components, and you can make them as simple or complex as you want. Running with the concept of a note-taking app, here is how potentially complicated you can make it:

  • Build an API to store data in a relational database.
  • Use OAuth to enable multi-user functionality.
  • Build a front end with React and deploy to Netlify.
  • Build a native mobile app for Android or iOS.
  • Set up real-time UI updates with WebSockets or gRPC.
  • Set up asynchronous automations with queueing and serverless functions.

It doesn't matter what the app does, as long as you understand what it should do. Then you get to fill in the blanks with whatever framework/language/etc you want.

Understand where ‘the thing’ fits

I use ‘the thing’ to refer to any new piece of tech you wish to explore. They all belong to one of four general categories:

  • Frontend
  • Backend
  • Storage
  • Conduit

I like to think of the general system flow from left to right like so.

Let’s break each of these down. I’ll be using the term “data” to represent information as it traverses the system. It could be a text file, an image, or a JSON structure. It doesn’t matter what it is, but it moves through the system and uses various services.

Frontend

This is the pretty stuff that shows up on screen and is the part users interact with most. This includes web frameworks, mobile apps, etc.

Backend

This is where the data goes to be processed in some capacity before it is stored in the storage (or data) layer, or retrieved from the storage layer and returned to the frontend system. This is why it sits between storage & front end. It’s the piece that connects them and makes them talk. This layer also often handles checking requests from the front end to make sure they are valid (ie; user authentication).

Storage

This is where data is stored. This could be a relational database, a NoSQL data store, or a file storage service. There isn’t much more to say about this layer, but that doesn’t make it simple. Each type of storage service has its own conventions and ways to store and retrieve data.

Conduit

This is a tricky one to place. I consider it to be in the Conduit category if it involves protocols that are used for services to communicate with one another. It could be low-level protocols like TCP and HTTP, but I also group things like GraphQL and queuing services here.

A real-world example

If you’ve followed me at all over the last two years, you’ll know I built an open-source web service for Destiny 2 players called GuardianForge. I built this service using mostly AWS services but intermingled third-party services like Algoliaand Stripe. Here is a diagram of the architecture of the site, categorized based on the definitions above, and described in detail below.

  1. The front end is comprised of 3 apps. They are built with React and hosted AWS S3 with CloudFront serving up the files to the user.
  2. When an API request is made (a user creates a build), the front end sends the request through AWS API Gatewayto an AWS Lambda function that processes the request.
  3. Lambda will create a summary record in DynamoDB.
  4. Lambda will also create a raw JSON file for the front end to render and reduce the load on the database.
  5. A message is sent to AWS SQS to further process the build, this results in a faster response to the user since the post-build logic doesn't necessarily need to be done immediately.
  6. The post-build Lambda function picks up the message and creates an OpenGraph social image to be shared on Twitter and sends a message to the GuardianForge Discord server (not shown).
  7. The Lambda from 6 will also send a message to Algolia, which powers the search on GuardianForge.
  8. Using Algolia libraries, the front end can interact with Algolia to search for builds.
  9. When a user signs up to support GuardianForge, the Lambda API interacts with Stripe to set up the subscription and flag the user in Dynamo that they are a premium user.
  10. Finally, the front end can also interact directly with the JSON files from 4.

How I used this process

When I joined PlanetScale, I was tasked with building something on the service. Since I already had GuardianForge setup, and I had a separate environment to play with new stuff that doesn't affect production, all I had to do was update the Lambdas that wrote to DynamoDB to now write my PlanetScale database. I essentially swapped out one of the boxes in my diagram.

I did all of this in a separate branch in the repo, which is still public to this day. I’ve also redone the frontend several times (it originally was a Vue project) and because I use this approach, I only had to replace the green parts in my diagram and everything else stayed as is.

Closing thoughts

If you want to make this work for you, set up a project of your own that you can continue to use. Start as simple as you need, or over-engineer the hell out of it to fit any category or service you want. If you can, make it on a topic you are passionate about. Then, when the next shiny thing comes out that you HAVE to try, figure out which box it fits in and swap out that component in your stack.