AWS Dynamic DNS

Battlesnake Programming Competition 2018

Book Notes: DevOps Handbook, The

Dark Mode React Toggle

Deno Tests & GitHub Actions CI

Deploy a Next.js Static Site to GitHub Pages

Migrating Frontend Assets During Grails 2 → 3 Upgrade

Lightshift Game

Building a Mechanical Keyboard from Scratch

My Introduction to Game Development

Next.js Tag Pages

Node CI/CD Pipeline using GitHub Actions & AWS Elastic Beanstalk

Resources I Found Valuable in 2022

Small Form Factor PC Build Part 1

Small Form Factor PC Part 2 - Water Cooling

Static Site Search Part 3 - Optimizing the Index

Static Site Search Part 1 - Preprocessing Articles

Static Site Search Part 2 - Search Component

How I use my Synology NAS as my personal cloud

What is a Synology NAS?

What I'm Learning in 2022

What is DevOps?


In my role as a software developer at my day job, I’ve been getting more interested in DevOps as a career trajectory. But what does that even mean? Do I even know?
  
This is a brief exploration into what DevOps is and how I see it relating to a small development team.

![DevOps Cycle from Atlassian](development-cycle.jpg)

## Index

* [Tl;dr](#tldr)
* [Initial Assumptions](#initial-assumptions)
* [Operations & Facilitators](#operations--facilitators)
* [Agile Development](#agile-development)
	* [Quick Iteration](#quick-iteration)
	* [Feedback & Data](#feedback--data)
* [Automation](#automation)
* [Not Only Software Development](#not-only-software-development)
* [Conclusion](#conclusion)
* [Resources](#resources)

## Tl;dr
Here are a few key ideas that I think define what DevOps is and what someone in a DevOps position should focus on:

* Automate things like dev environment setup and employ tools to make writing code simple. Make it easy and painless to write code.
* Automate testing so developers and teams are confident in the code being produced. Developers shouldn't have to "remember" to run tests.
* Automate infrastructure provisioning and blue/green deployments. Code that's reviewed and ready to go should be able to reach customers with a single click (with proper authentication). Make deploying new code trivial.
* Automate data collection and illumination for the dev team primarily but also for the business side depending on the size of your company.
* And I think in general, a DevOps role should facilitate software development by increasing confidence in work being done by both developers and product managers.

## Initial Assumptions

Earlier on in my career and in university, DevOps was a buzz word that was thrown around all the time. When I heard the term DevOps I associated that with “cloud infrastructure”. I was connecting the idea of “building things on cloud infrastructure” to the role of DevOps.
  
While using cloud infrastructure might be one aspect of the tools and responsibilities of a DevOps role, it doesn’t get to the root of what the role is about.

## Operations & Facilitators

Within a software company, big or small, there are various so-called “Operations” roles. Business Operations, Customer Operations, Insert Anything Operations. My perspective on these roles in general is that they are facilitator roles rather than managerial roles. “Managers” may often have an Officer title, while “facilitators” may have an Operations title.

What do I mean by Facilitators? These are roles that essentially amplify the work in a given area or team. Their job is to ensure process and procedure are in place so that work can get done effectively, impactfully, and efficiently. It’s not necessarily their job to oversee the work being done, but to boost the impact and productivity of the work being done.

DevOps is short for Development Operations. And in my context as a software developer, Development is actually short for Software Development (Development can mean many different things in different fields).
  
So in more plain language, the role of DevOps is related to the operations of the development team and the development lifecycle. Ok, so that sounds more appropriate than just “cloud infrastructure”, but what is involved in the operations of those things?

## Agile Development

While I’m no expert on Agile development practices, I’ve been working with variations of it for my entire software development career thus far.

In essence, Agile (with a capital A) is intended to make the development process more iterative and quick to react to user feedback. Rather than blocking out big periods of time for large projects that overhaul huge portions of the customer experience, we attempt to release work quicker and in smaller pieces to get feedback on particular features sooner.

This allows us to not waste time perfecting features that our customers don’t want or won’t use. Instead we can focus more directly on things that are impactful for our customers and our bottom line. Therefore, one aspect of a DevOps role is facilitating the Agile development process.

### Quick Iteration

Getting code written, reviewed, and deployed quickly helps you then iterate on that code sooner, leading to better experiences for the customer over time. Building and improving processes around code testing, review, and deployment is a key facilitator opportunity for a DevOps role. This can cover many different things, from automating testing and deployments to improving code review process.

### Feedback & Data

Timely feedback is also vital to the Agile development process. Both user experience feedback and systems operation feedback (through statistics and error reporting). This feedback informs subsequent development cycles and how features, tweaks, and bug fixes are prioritized against each other for the greatest impact. Developing systems to effectively and concisely collect and relay feedback to the appropriate developers and project leads allows for more informed decision making, and thus is another key responsibility of a DevOps role.

## Automation

There are many pieces of manual work and human intervention in the software development cycle, and the level of automation in place can vary wildly between different teams and companies. I entered my software development career from essentially a cloud-native perspective, having little to no experience deploying code in a different paradigm.

But today, with things like IaC (infrastructure as code) and remote dev environments being the norm, every aspect of the developer experience can be automated in some way. Including writing code believe it or not.

* __Dev environment setup can be automated with old tools like Docker, or emerging tools like [JetBrains Space](https://www.jetbrains.com/space/).__ Space is essentially a remote dev environment that helps teams standardize their development environments across a team. Space is in closed beta as of the writing of this article. GitHub & Microsoft are working on similar tools with GitHub Codespaces and VS Code.
* __Writing code can be automated with new tools like [GitHub Copilot](https://copilot.github.com/).__ Copilot is a machine learning tool that has learned how to write code in many popular languages. While it can't directly replace a human developer yet, it can greatly speed up the process of writing code from scratch.
* __Reviewing, unit-testing, and merging code can be automated with tools like GitHub and [GitHub Actions](https://github.com/features/actions).__ There is still a bit of human touch needed in the review process to ensure features are truly meeting the business needs of the brief, but reviewing security and dependencies for example can be greatly sped up with the help of automation tools. Running tests can be automated with a variety of CI tools. Merging code and resolving merge conflicts has become downright trivial with tools like git and GitHub.
* __Deploying code to infrastructure can be automated with a variety of CD tools.__ I've been mainly interested in GitHub Actions and automating deployments to AWS. But I'm now also interested in learning more about AWS CDK (cloud development kit). [This blog post from GitHub](https://github.blog/2021-01-25-improving-how-we-deploy-github/) has been quite influential in my current thinking about automated deployments.
* __App stats and error monitoring can be automated with tools like Sentry, Datadog, and NewRelic.__ Those stats can be viewed through there respective dashboards or made more accessible through things like Slack bots.

## Not Only Software Development

In a small company or on a small team, often certain roles wear many hats. In our case, business operations bleed into the development team all the time. People working with clients, company finances, product managers, or even the CEO, will often have inquiries for data about various aspects of our business operations.

Without adequate access to that data, requests can sometimes fall on developers who are familiar with the system and the data. Ad-hoc SQL queries to our database can sometimes be the norm, but this isn’t an efficient use of anyone's time.

Building systems and interfaces to our company’s data can be integral to operations and development. Depending on the size of the company and the other roles there, this idea of data illumination could fall on a Business Operations role, but often it can require integration and understanding of the technical stack that is foundational to your application. Therefore that responsibility can fall on a DevOps role, to build digestible interfaces and secure pipelines to get the relevant data to the team members that need it.

## Conclusion

In summary, I see DevOps as primarily a facilitator and multiplier role. Depending on the size of the team they could be simultaneously doing development themselves, but the focus is on how to make the development team more productive.

Facilitating the development team to be more productive means a few things:

* Make code easy and painless to write. Automate environment setup.
* Make code simple to test. Automate tests.
* Make code trivial to deploy when ready. Automate infrastructure and blue/green deployments.
* Make data about development, testing, and deployment visible to the team. Automate data collection and illumination with tools like Datadog.

My summary and this article represents my understanding and opinions right now (Spring 2022). I am constantly learning and improving on my career journey and my knowledge and opinions will undoubtedly change over time.

If you have any comments or suggestions please feel free to reach out to me on Twitter [@tyreldelaney](https://twitter.com/tyreldelaney)! 👋🏻

## Resources

* [The Phoenix Project](https://itrevolution.com/the-phoenix-project/) - Gene Kim, Kevin Behr, and George Spafford
* [Improving how we deploy GitHub](https://github.blog/2021-01-25-improving-how-we-deploy-github/) - GitHub Blog
* [DevOps principles](https://www.atlassian.com/devops/what-is-devops) - Atlassian


Here are some things that I'd like to learn more about in 2022. This is kindof a reference for myself. So I can look back at this next year and reflect on what I learned and what I wanted to learn. [Let me know on twitter](https://twitter.com/tyreldelaney) if you are learning these same things or you have any tips or resources to share!

![Logo montage of all the technologies outlined in this article](what-im-learning-in-2022.jpg)

## Index

[Learning from scratch](#learning-from-scratch)

* [Docker](#docker)
* [Vim/NeoVim](#vimneovim)

[Learning more of](#learning-more-of)

* [Github Actions](#github-actions)
* [AWS DynamoDB](#aws-dynamodb)
* [AWS Client VPN, VPNs in general, and SSO](#aws-client-vpn-vpns-in-general-and-sso)
* [SSH](#ssh)
* [HTTPS/SSL](#httpsssl)
* [Raspberry Pi](#raspberry-pi)
* [Home Network](#home-network)

[Things I'm interested in but might not get to this year](#things-im-interested-in-but-might-not-get-to-this-year)

* [A better static site generator / CMS](#a-better-static-site-generator--cms)
* [Webhooks](#webhooks)

## Learning from scratch

### Docker
Docker really feels like one of those technologies that I've been conveniently ignoring for a little too long. I think what I'll do to start is simply try to containerize some smaller projects I'm working on this year to dip my toes in.

I acquired a handful of second-hand Raspberry Pis not long ago and I've got a few ideas for some small home automation & monitoring projects that might be perfect simple projects to dockerize.

A stretch goal would be to work up to containerizing our monolithic app at my day job.

### Vim/NeoVim
I've always just barely snuck by with my fluency in the terminal. The more I get into networking and Dev Ops the more this deficiency is showing. In general I'd just like to work in the terminal a bit more this year.

Specifically I'm going to try to develop a couple Node and Python projects I'm cooking up remotely on the Raspberry Pis I hope to run them on.

Really two main goals I'd like to accomplish with this practice:
1. Get proficient and comfortable using vim.
2. Learn more about making a portable and lightweight dev environment. Something that I can easily spin up on a Raspberry Pi or AWS EC2 instance over SSH and feel comfortable doing real work in.

## Learning more of

### Github Actions
Automation is a big focus for me this year. And really just because I rely on Github both personally and professionally, Github Actions is a natural choice to skill up in.

We're now looking into automating and outsourcing our test suite at work and since we use Github for source control and code reviews/pull requests it makes total sense to use Github Actions.

I've automated a few things using Github Actions already, namely [tests and deployment of my](https://github.com/tyrelh/battlesnake-typescript-node) [Battlesnake](https://play.battlesnake.com/) to AWS Elastic Beanstalk. Automating even just tests for our app at work will be a bit more of a challenge since it's a much older stack and has many more moving parts.

### AWS DynamoDB
I wasn't actually planning on learning DynamoDB at the start of the year. But a project idea came to mind and the possibility of using Dynamo to store data rather than something like S3 sounded intriguing.

Essentially what I'm planning is building out a few Raspberry Pi temperature sensors for my house. Maybe 4 or 5 in total monitoring a few rooms in the house as well as the outdoor temperature.

The idea would be to collect data from all these sensors using Raspberry Pis maybe every 5 mins or so, and ship that data off to Dynamo. Then building an infrastructure to rotate through tables. Basically make a new table each day with a high write provision, and rotate down the previous days tables read and write provisions. Maybe after some period archive the data in S3.

Might be over engineering for the task, but it's for learning.

[Best Practices for Handling Time Series Data in DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-time-series.html)

### AWS Client VPN, VPNs in general, and SSO
With my day job going fully remote this year, setting up secure remote tools has been high on our priority list. Part of that will be learning about AWS Client VPN which we may use to grant our devs access to some of our AWS infrastructure.

Part of this work is also moving to more zero trust architecture, and leaning into SSO for authentication.

Also I recently acquired a Synology NAS for my home (which we actually use to use in our office before we gave that space up). I'm looking to centralize alot of my personal cloud use to the Synology and move away from services like Google Drive (office and documents), iCloud (photos), and Notion (notes). Part of that will be to setup a VPN at home so that I can use those services hosted in my home from anywhere. As well as accessing some of my Raspberry Pis for remote development.

### SSH
Learning more about SSH and tunneling goes hand-in-hand with a few other learning priorities I have. Learning best practices for connecting securely and crafting secure infrastructure. Also using SSH to practice remote development and learning terminal based tools.

### HTTPS/SSL
Same as above. Learning more about best practices around secure remote communication. Getting more confident understanding certs and how they work.

### Raspberry Pi
I've owned a Raspberry Pi 3 for a handful of years now, but I've never really had a project for it. I experimented a bit with retro game emulation on it, but those projects are pretty trivial to setup.

Now I have a handful of Raspberry Pi 2s and 3s and I have some plans for some smaller single purpose projects for the house that will be perfect for them.

### Home network
This kinda just happened organically, but I have a lot of computers in my house now! What I'd like to do is centralize a bunch of things into a proper rack. I have a few computers that are mining cryptocurrency that I'd like to transfer to rack-mountable server cases. Along with my new Synology NAS and a few Raspberry Pis.

Wiring my house with Cat-6 probably won't happen any time soon, but centralizing a bunch of things in a rack will let me hard-wire them all to a small switch without needing to run cables through all my walls.

Like I mentioned above, also setting up network storage and remote access for my home network.

## Things I'm interested in but might not get to

### A better static site generator / CMS
My website currently is hand-built in Javascript and React with a few plugins that let me render my articles from markdown. There is still a fair amount of manual process to get each markdown file wired up to display as a page and route, but at least I don't need to reformat the article. I can just paste it in from my notes app and then wire it up.

Ideally I'd like to learn a better static site generator to simplify this process even more. I've been thinking of learning [Gatsby](https://www.gatsbyjs.com/) for a while now, but I've just found it a bit cumbersome and hard to get into.

Some alternatives I'm considering learning instead of Gatsby are [Jeckyll](https://jekyllrb.com/) or [Hexo](https://hexo.io/).

### Webhooks
Just something I'm interested in learning more about.

## End

[@ me on twitter](https://twitter.com/tyreldelaney) if you are learning these same things or you have any tips or resources to share!


During a recent project to convert my JavaScript & Node [Battlesnake](https://play.battlesnake.com/) over to TypeScript & [Deno](https://deno.land/), I decided I would also create a more robust suite of tests for it.

![Deno and Github logos with an addition symbol between them](DenoTestsAndGithubActionsCI.png)

If you are unfamiliar with [Battlesnake](https://play.battlesnake.com/), it is an online and in-person programming competition where you build a AI in the form of a REST API that can play a multiplayer version of the classic game [Snake](https://en.wikipedia.org/wiki/Snake_(video_game_genre). You can check out details on their [website](https://play.battlesnake.com/) to learn more, and even watch some past events on their [Twitch channel](https://www.twitch.tv/BattlesnakeOfficial).

## Tl;dr
* [Deno Tests](#deno-tests): Deno includes a very simple test runner framework.
* [Github Actions](#github-actions): Github actions allow you to automate tasks based on triggers within your Github repo.
* [Create the workflow](#create-the-workflow): A .yml file describes a series of actions you would like to perform based on a trigger.
* [Automate Deno tests](#automate-deno-tests): Just add a `run: deno test` command to your workflow .yml.

## Deno Tests
[Deno](https://deno.land/) includes a simple test runner for tests in js, ts, jsx, or tsx files. It can be run with:

```bash
deno test
```

To create tests, all you need to use is use the `Deno.test()` method included with Deno within files ending in *test* (`{*_,}test.{js,ts,jsx,tsx}`) anywhere in your project structure. I put mine in *tests/*.

Example of a basic test:

```typescript
import { assert } from "https://deno.land/std/testing/asserts.ts";
import { root } from "../app/main.ts";

Deno.test("Root response contains expected fields", () => {
  // when
  const result = root();
  // then
  assert(result?.apiversion);
  assert(result?.author);
  assert(result?.head);
  assert(result?.tail);
  assert(result?.color);
});
```

This test is simply asserting that values exist for these specified fields.

You can add many tests to single files by calling `Deno.test()` over and over again. You can also make as many test files as you like, Deno will automatically execute all the files it recognizes ending in *test* (`{*_,}test.{js,ts,jsx,tsx}`).

Visit the [Deno Testing docs](https://deno.land/manual/testing) for more info about creating tests.

Easy! So now I can create a robust set of unit tests for my Battlesnake AI.

Since the game Battlesnake works by [sending you the entire game state for every move](https://docs.battlesnake.com/references/api), I can also save the game state JSON from particularly tricky moves and use them within their own tests. The output can be compared against the single best move, or even a set of equally good moves if their are multiple. Creating tests like this tests the behaviour of the AI overall and ensures that changes to the behaviour aren't making the AI perform worse on moves it previously was able to do correctly.

So now that I have tests up and working, I wanted to automate them using Github Actions.

## Github Actions
Github Actions are still quite new to me and I hadn't touched them at all before this project. I just assumed it would be complicated to set up.

For what I wanted to do for this project it was actually trivial. Couldn't have been easier.

What I wanted was to run these tests automatically whenever pushing to my repository. I figured this Deno app is pretty simple all things considered, so I shouldn't have to mess around with Docker or anything like that.

I'll step you through what I did.

## Create the workflow
Github Actions are based around these things called workflows. They are essentially .yml scripts that describe the series of events you want to perform in your action.

If you click the Actions tab within the repo you want to create the Action you will be presented with a Get started with GitHub Actions page. This page has many example actions you get you started for so many different languages, frameworks, and platforms. It will even suggest actions to you based on the contents of your repo. I chose the *Skip this asn set up a workflow yourself* option.

![Get started with Github Actions](DenoAndGithubActions-get-started-with-github-actions.png)

This plops you into a web editor and gives you a basic example workflow .yml file to start with. It looks something like this:

```yaml
# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2

    # Runs a single command using the runners shell
    - name: Run a one-line script
      run: echo Hello, world!

    # Runs a set of commands using the runners shell
    - name: Run a multi-line script
      run: |
        echo Add other actions to build,
        echo test, and deploy your project.

```

With this you can actually click the *Start commit* button and that will guide you to commit this .yml file to your repo within the *.github/workflows/* directory.

You can see near the top of that file that there are triggers for `on push` and `on pull_request` attached to the master branch. You can try to push something to master and see it work!

If you visit the Actions tab of your repo now you will see your Actions and the history of all the times they have run. You can click into them for more details and to see the console outputs. Also by default it will email your Github account email whenever an action fails. I imagine this can be changed somewhere if you don't like that.

![Github Actions](DenoAndGithubActions-github-actions.png)

## Automate Deno tests
Now I wanted to try to run my tests using Deno. I knew this would involve installing Deno somehow so I searched around.

I found [this repo](https://github.com/denolib/setup-deno) from [denolib](https://github.com/denolib) with exactly what I wanted.

Really all you need to do is add the `uses` line pointing to this repo to your workflow .yml file under the `steps` section. You can also include the Deno version you prefer. Then within the `run` section you now have access to `deno` and all of its functions.

So now my workflow .yml file looks like this:

```yaml
name: CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@master
    - uses: denolib/setup-deno@master
      with:
        deno-version: v1.x
    - run: deno run https://deno.land/std/examples/welcome.ts
```

One really cool feature of Deno is that you can run scripts from the web right in the terminal, and you can also do this right within Github Actions. That is what the URL is doing in

```yaml
- run: deno run https://deno.land/std/examples/welcome.ts
```

You can run this and then look within your Github Actions for the console output from this example *welcome.ts* file.

So now to run my tests, all I needed to do was change that `deno run` command to `deno test`. My final .yml workflow looks something like this:

```yaml
name: CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  run-tests:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@master
    - uses: denolib/setup-deno@master
      with:
        deno-version: v1.x
    - run: deno test
```

Now every time I push to my master branch all my tests will be run, including any new tests I just committed, and I will be notified both on the Github UI and in my email if any tests fail. I can then log into Github and look at the output to see what failed.

I hope this small guide helped you out and showed you just how simple both Deno and Github Actions are!

#github-actions

What is DevOps?

April 10, 2022 7 Minute Read

What I'm Learning in 2022

March 23, 2022 5 Minute Read

Node CI/CD Pipeline using GitHub Actions & AWS Elastic Beanstalk

May 30, 2020 24 Minute Read

Deno Tests & GitHub Actions CI

May 23, 2020 6 Minute Read

Get in touch