AWS Dynamic DNS

AWS Solutions Architect Associate Certification

Battlesnake Programming Competition 2018

Book Notes: DevOps Handbook, The

Dark Mode React Toggle

Deno Tests & GitHub Actions CI

Deploy a Next.js Static Site to GitHub Pages

Migrating Frontend Assets During Grails 2 → 3 Upgrade

Lightshift Game

Markdown based Code Block Syntax Highlighting

Building a Mechanical Keyboard from Scratch

My Introduction to Game Development

Next.js Tag Pages

Node CI/CD Pipeline using GitHub Actions & AWS Elastic Beanstalk

Resources I Found Valuable in 2022

Small Form Factor PC Build Part 1

Small Form Factor PC Part 2 - Water Cooling

Shipping ECS Fargate logs to Datadog with FireLens (and what broke)

Static Site Search Part 3 - Optimizing the Index

Static Site Search Part 1 - Preprocessing Articles

Static Site Search Part 2 - Search Component

How I use my Synology NAS as my personal cloud

What is a Synology NAS?

What I'm Learning in 2022

What is DevOps?

This past June I set up log shipping from ECS Fargate to Datadog using FireLens. It worked on the first deploy. Then I spent a week chasing two problems with it, and months later I found out I'd been wrong about what fixed one of them.

![Shipping ECS Fargate logs to Datadog with FireLens heading on a dark background](shipping-ecs-fargate-logs-to-datadog-with-firelens-and-what-broke-ecs-fargate-logs-hero.png)

## Tl;dr

[The setup](#The%20setup): A new Go API on ECS Fargate, and three ways to get its logs into Datadog.
[The happy path](#The%20happy%20path): About twenty lines of Terraform for the log router and the app's log config.
[Breakage #1: our tags weren't applied (except they were)](#Breakage%20%231%3A%20our%20tags%20weren't%20applied%20(except%20they%20were)): I filed a bug on myself for mixing up tags and attributes.
[Breakage #2: every log is INFO](#Breakage%20%232%3A%20every%20log%20is%20INFO): We logged the HTTP status code as `status`, which is a reserved attribute. Datadog read `200` as a severity.
[What I'd do differently](#What%20I'd%20do%20differently): Grep for reserved attribute names, and prove each log level on day one.
[So what did I learn?](#So%20what%20did%20I%20learn%3F)
[Resources and links](#Resources%20and%20links)

## The setup

I was standing up the platform side of a new Go API on ECS Fargate. Small team, SOC2 coming up, and we needed to see what the thing was doing in production. We already used Datadog, so the only question was how to get the logs there:

* Run the Datadog agent as a sidecar in every task. Well documented, but it's a second container to size, patch, and pay for in every task definition.
* Ship logs straight from our custom logger to [Datadog's HTTP intake endpoint](https://docs.datadoghq.com/logs/log_collection/?tab=http#additional-configuration-options). Tempting, but then our code owns retries, batching, and backpressure to a third party, and logging is the thing you want working *most* when everything else is broken.
* Use [AWS FireLens](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_firelens.html), ECS's built-in log routing on Fluent Bit, with Datadog as the output plugin.

We went with FireLens. [AWS](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_firelens.html) and [Datadog](https://docs.datadoghq.com/integrations/ecs_fargate/?tab=webui#log-collection) both point you at it for Fargate, it lives entirely in the task definition (so it's Terraform, not application code), and the application keeps writing JSON to stdout.

Logs were flowing on a Wednesday. That Saturday I filed two bugs against my own setup, and both were closed the following Thursday, so about a week from "it works" to "it actually works". Understanding why took a few more months.

## The happy path

FireLens is a log *router*. You add an `aws-for-fluent-bit` container to your task definition, mark it as the FireLens container, and point your application container's log driver at it. Your app keeps writing to stdout, and Fluent Bit forwards it to Datadog's intake.

The log router container, in Terraform:

```hcl
{
  name              = "datadog_log_router"
  image             = var.aws_for_fluent_bit_image   # pinned to a specific tag
  essential         = true
  cpu               = floor(var.cpu * var.log_router_container_resource_ratio)
  memoryReservation = floor(var.memory * var.log_router_container_resource_ratio)
  firelensConfiguration = {
    type    = "fluentbit"
    options = { enable-ecs-log-metadata = "true" }
  }
  user = "0"
}
```

`enable-ecs-log-metadata` attaches the ECS context (cluster, task ARN, task definition family, container name) to every log line. It defaults to on, but I'd rather state that default than rediscover it when it's missing.

The resource ratio is the part I couldn't find in any of the docs, which happily show a FireLens container with no CPU or memory on it at all. We give the log router a fixed fraction of whatever the main container gets, so a beefy task gets a beefy log router and a small one doesn't waste reservation. It's not sophisticated, but we never had to think about it again.

The app container's log configuration is where the Datadog behaviour lives:

```hcl
logConfiguration = {
  logDriver = "awsfirelens"
  options = {
    "Name"           = "datadog"
    "Host"           = var.datadog_host
    "dd_service"     = var.dd_service
    "dd_source"      = "ecs-task"
    "dd_tags"        = "environment:${terraform.workspace},stack:platform"
    "dd_message_key" = "log"
    "TLS"            = "on"
    "provider"       = "ecs"
  }
  secretOptions = [
    {
      name      = "apikey"
      valueFrom = "${local.datadog_api_key_secret_arn}:api_key::"
    }
  ]
}
```

The API key lives in Secrets Manager and ECS injects it through `secretOptions`, so it never lands in the task definition JSON or in state. The `:api_key::` suffix pulls a single key out of a JSON secret: the format is `<arn>:<json-key>:<version-stage>:<version-id>`, and the trailing colons are the empty version fields. It's in the [AWS docs](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-tutorial.html), but it's easy to skim past, and getting it wrong gives you a container that won't start with a message about the secret, not the format.

Logs showed up in Datadog within minutes. It really was that easy, until Saturday.

## Breakage #1: our tags weren't applied (except they were)

`dd_tags` was setting `environment:<workspace>` and `stack:platform` on every log, and as far as I could tell in the Datadog UI, neither was there. We run two products in one Datadog org, and tags were how we'd separate them in dashboards, monitors, and eventually access. If tags didn't work, the whole logging story was blocked.

So I filed a bug. I was looking at the *Attributes* section of the log side panel, and tags are in the *Tags* section. They'd been applied the whole time. Five days later I closed it with a comment that amounts to "oh, this already works, I was looking at the wrong panel."

Datadog does make this easy to get wrong, because it splits log metadata into two things that look identical when you're new to it.

Tags are infrastructure-level metadata: `dd_tags`, the ECS metadata from `enable-ecs-log-metadata`, `service`, `source`, `env`. They share a namespace with your metrics and traces, which is what lets you pivot from a log to a dashboard. You query them bare, like `env:prod`.

Attributes are the parsed contents of your JSON log body, whatever your logger emitted. You query them with an `@` prefix, like `@env:prod`.

Ten minutes of poking at the log panel would have saved me the ticket.

## Breakage #2: every log is INFO

Every log line arrived in Datadog as INFO, warnings and errors included. The status facet showed exactly one value, no monitor on `status:error` could fire, and the error rate graph was a flat line at zero, which looks like good news right up until you find out what it means.

My hunch, written into the ticket, was that Datadog reads severity from an attribute called `status`, while Go's `slog.NewJSONHandler` emits `"level":"INFO"`. Different key, so everything defaults to INFO. We renamed the key, the severities came back, I closed the ticket, and that was the week.

That explanation is wrong, and I only found out months later while writing this up.

### What the docs actually say

Datadog [preprocesses JSON logs](https://docs.datadoghq.com/logs/log_configuration/pipelines/#preprocessing) before your pipelines see them. On status, it says:

> if a JSON formatted log file includes one of the following attributes, Datadog interprets its value as the log's official status: `status`, `severity`, `level`, `syslog.severity`

`level` is in that list, and it was before our incident. Our preprocessing config is stock, so Go's default `slog` output was always going to be read correctly. What actually made everything INFO is in two details of the docs that are easy to read past.

First, the list is ordered, and the order *is* the precedence. I'd read it as a set. Preprocessing takes the *first* of those attributes it finds, so with both `status` and `level` present, `level` is never consulted.

Second, unrecognized values don't fail, they become INFO. From the [Log Status Remapper docs](https://docs.datadoghq.com/logs/log_configuration/processors/log_status_remapper/), integers 0 through 7 map to Syslog severities, and "All others map to info (6)." A *missing* `status` sends the search on to `severity`, then `level`. A `status` that's present but unreadable ends the search right there, at INFO. Nothing errors, and nothing tells you Datadog didn't understand your value.

### The actual root cause

Our HTTP request logger did this:

```go
slog.Int("status", 200)
```

That's the HTTP status code, in the reserved attribute, on every request log. Preprocessing found `status`, read `200`, which isn't in 0-7, mapped it to INFO, and stopped looking. `level` was sitting right there in the same log line, correctly populated, and never got read.

It stayed quiet because an HTTP status code is a *plausible* integer, just never a valid Syslog severity. `200`, `404`, and `500` all map to INFO, so a 500 logged at the same severity as a 200, and a 500 is exactly what you'd want to alert on.

### The fix, and the half of it that did nothing

The PR shipped two changes:

```go
// The change we credited: rename slog's severity key to `status`.
stdoutHandler = slog.NewJSONHandler(os.Stdout, &slog.HandlerOptions{
	ReplaceAttr: func(_ []string, a slog.Attr) slog.Attr {
		if a.Key == slog.LevelKey {
			a.Key = "status"
		}
		return a
	},
})

// The change that actually fixed it: stop squatting on the reserved attribute.
slog.Int("status_code", 200)   // was: slog.Int("status", 200)
```

The second one is the fix. With the HTTP code moved to `status_code`, preprocessing found no `status`, fell through to `level`, and read `"INFO"` / `"WARN"` / `"ERROR"` exactly as documented.

The `ReplaceAttr` block does nothing, because `level` was already supported, and it makes our output less portable by emitting a non-standard severity key. It's eight lines that should be deleted.

Both changes shipped together, the symptom went away, and the rename got the credit for months because it was the more interesting-looking change. We had a working fix and a wrong mental model, and nothing in the system was ever going to correct us.

We also considered a Datadog Status Remapper processor pointed at `level`, or a Fluent Bit filter renaming the key before it ships. Both add configuration to work around a problem we created in the application, while fixing the squatted attribute is one line.

### Grep your logger for these

Datadog reserves these attribute names and aliases, and silently prefers them over whatever you meant:

| Reserved | Aliases it also reads |
| --- | --- |
| `status` | `severity`, `level`, `syslog.severity` |
| `message` | `msg`, `log` |
| `service` | `syslog.appname`, `dd.service` |
| `host` | `syslog.hostname`, `dd.host` |
| `timestamp` | `date`, `_timestamp`, `published_date` |
| `trace_id` | `dd.trace_id`, `contextMap.dd.trace_id` |

`status` is the one that bites, because it's the obvious name for an HTTP status code. But `message`, `host`, and `date` are all names a reasonable logger emits for reasonable reasons. Grep your logging code for every one of these before your first deploy. It takes two minutes, and the alternative is a monitor that silently never fires.

If you got here from the symptom, Datadog has a guide: [Logs show INFO status for warnings or errors](https://docs.datadoghq.com/logs/guide/logs-show-info-status-for-warnings-or-errors/). It states the default: "By default, when Datadog's Intake API receives a log, an `INFO` status generates and appends itself as the `status` attribute."

## What I'd do differently

* Grep for reserved attribute names before the first deploy. It's the highest-value thing on this list.
* Verify every log level on day one. Emit a DEBUG, INFO, WARN and ERROR as soon as logs flow, and confirm all four in the UI. I found this three days late because "logs are showing up" felt like done.
* Decide your retention before you need it. We dual-wrote to CloudWatch and Datadog at first, paying twice for the same lines, and landed on 30 days in Datadog. That window is also why I couldn't audit this later: by the time I checked whether the rename had done anything, the original logs were gone, and I had to reconstruct it from the diff and the docs.
* Know when direct HTTP is the better answer. FireLens is the right default for Fargate because it's declarative and lives outside your application. If you're somewhere ECS doesn't reach, or need something a Fluent Bit output plugin doesn't support, Datadog's HTTP intake is right there. Just know your application now owns log delivery.

## So what did I learn?

The integration was twenty lines of config. The hard part was the semantics: which metadata is a tag and which is an attribute, and who owns the word `status`. Datadog won't tell you it preferred a different attribute than the one you meant, and a fix that ships next to the real one will take the credit for months, because the symptom went away and nobody had a reason to look closer.

So distrust the pipeline until you've personally seen one WARN and one tag in the UI. And when the symptom clears, make sure you know *which* change cleared it.

## Resources and links

* [Datadog: ECS Fargate log collection](https://docs.datadoghq.com/integrations/ecs_fargate/?tab=webui#log-collection)
* [AWS: Custom log routing with FireLens](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/using_firelens.html)
* [AWS: Specifying sensitive data with Secrets Manager](https://docs.aws.amazon.com/AmazonECS/latest/developerguide/specifying-sensitive-data-tutorial.html)
* [Datadog: Preprocessing for JSON logs](https://docs.datadoghq.com/logs/log_configuration/pipelines/#preprocessing)
* [Datadog: Log Status Remapper](https://docs.datadoghq.com/logs/log_configuration/processors/log_status_remapper/)
* [Datadog: Logs show INFO status for warnings or errors](https://docs.datadoghq.com/logs/guide/logs-show-info-status-for-warnings-or-errors/)
* [Datadog: Log collection over HTTP](https://docs.datadoghq.com/logs/log_collection/?tab=http#additional-configuration-options)


![Book cover of Dev ops handbook](devopshandbook.png)

Here are some point-form takeaways from my notes from reading [The DevOps Handbook](https://app.thestorygraph.com/books/70e99c96-fc97-465a-bf1b-058072e18bc5) by John Willis, Gene Kim, and Patrick Debois.

## Index

[Core premise](#core-premise)
[First way of DevOps: Flow](#first-way-of-devops-flow)
- [Make work visible](#make-work-visible)
- [Limit WIP](#limit-wip)
- [Small batch sizes](#small-batch-sizes)
- [Elevate constraints in value streams](#elevate-constraints-in-value-streams)
- [Categories of waste and hardship](#categories-of-waste-and-hardship)
- [Developer productivity vs number of developers](#developer-productivity-vs-number-of-developers)
[Second way of DevOps: Feedback](#second-way-of-devops-feedback)
[Third way of DevOps: Continuous Learning and Experimentation](#third-way-of-devops-continuous-learning-and-experimentation)
[Practices to improve the three ways](#practices-to-improve-the-three-ways)
- [Immutable infrastructure](#immutable-infrastructure)
- [Infrastructure as code](#infrastructure-as-code)
- [Testing pyramid](#testing-pyramid)
- [Continuous integration](#continuous-integration)
- [No more pull requests](#no-more-pull-requests)
- [Git strategies](#git-strategies)
- [Continuous delivery](#continuous-delivery)
- [Metrics](#metrics)
- [A/B testing](#ab-testing)
- [Chatbots for automation](#chatbots-for-automation)
- [Single company-wide repository](#single-company-wide-repository)

## Core premise

At the core of this book are the three "ways of DevOps".

1. Left-to-right flow of work from development to operations to the customer.
	- Speeding up this channel gets work from idea to the customer faster.
	- Decreasing lead times through better process leads to higher value delivered to customers and higher developer satisfaction.
	- Resulting practices from improving this channel include continuous integration, continuous deployment, and infrastructure-as-code, among others.
2. Right-to-left constant feedback from customers to operations to developers.
	- Prevent problems from happening again.
	- Enable faster detection of and recovery from issues.
	- Resulting practices from improving this channel are easy to access and understand metrics, and making your systems more observable.
3. Cycles of learning. Creating a generative, high-trust culture.
	- Scientific approach to risk taking.
	- Organizational learning. Multiply the effects of new knowledge. Transform local discoveries into global improvements.
	- Amplifying positive feedback loops creating ever-safer systems.

## First way of DevOps: Flow

### Make work visible

Make work visible. Kanban boards allow you to easily see the state of a current projects and the team in general. Keep work from development and operations on the same kanban board so that both teams can see the in-progress work of the other.

### Limit WIP

Multi-tasking significantly degrades efficiency. Limit the number of WIP items to keep teams focused. You can do this by controlling the swim lane size in your kanban boards.

Reference: [Making Work Visible: Exposing Time Theft to Optimize Work & Flow](https://app.thestorygraph.com/books/6c5c24e2-6006-43b9-9e30-569400cd1581) by Dominica Degrandis

### Small batch sizes

Small batch sizes lead to faster lead times for individual projects, less WIP, faster detection of issues, and less rework.

### Elevate constraints in value streams

Focus efforts on optimizing bottlenecks in your value streams. The most value and effect can be had by remediating bottlenecks rather than steps before or after a bottleneck.

### Categories of waste and hardship

These categories of work are where inefficiencies are introduced and time and effort are wasted.

- Partially done work: WIP, work waiting in queues, etc. Partially done work loses value as time passes.
- Extra processes: Additional work that doesn't add value to the customer. Increases lead time.
- Extra features: Output that is not valued by the customer. Adds complexity and increases lead time.
- Task switching: Multitasking adds effort and time to the value stream.
- Waiting: Delays or queues increase lead time.
- Motion: Friction in handoffs adds ambiguity into the value stream and can increase lead time.
- Defects: Bugs, miscommunications, etc. add lead time and rework.

Reference: [Implementing Lean Software Development: From Concept to Cash](https://app.thestorygraph.com/books/a18e5dee-89e4-4c79-948e-ada45f61b476) by Mary Poppendieck and Tom Poppendieck

Added categories from Damon Edwards:

- Nonstandard or manual work: Any manual work that can be automated. Increases lead time and can lead to human error.
- Heroics: Individuals and teams should never be put in positions where they must perform unreasonable acts. Don't let heroics become the norm. When an team member needs to perform heroics, a breakdown of process has occurred. Should be remedied such that it's never required again.
### Developer productivity vs number of developers

When we increase the number of developers, individual developer productivity often significantly decreases due to communication, integration, and testing overhead

## Second way of DevOps: Feedback

Feedback and feedforward loops allow us to detect and remediate problems while they're smaller, cheaper, and easier to fix, to avert problems before they cause a catastrophe, and create organizational learning that we integrate into future work.

![Nesting circles showing the time difference of feedback from different components of the value stream](feedback-cycle-times.png)
^ Time it takes to get feedback from different stages of feedback. Taken from DevOps Handbook

## Third way of DevOps: Continuous Learning and Experimentation

Couple of quotes:

> When accidents and failures occur, instead of looking for human error, we look for how we can redesign the system to prevent the accident from happening again.

^ DevOps Handbook

> Even more important than daily work is the improvement of daily work.

^ Mike Orzen, author of Lean IT

You want to create an environment of trust where experimentation and learning are the norm. You want to have simple process in place to elevate local learnings to global company-wide knowledge.

## Practices to improve the three ways

### Immutable infrastructure

Never allow changes directly to infrastructure. Any changes must be checked into version control in our infrastructure-as-code, then deployed as new infrastructure that replaces the existing infrastructure, rather than tweaking it in-place.

You wouldn't let someone log into app servers and make changes directly to the code. So why let that happen for infrastructure?

### Infrastructure as code

- Version control for your infrastructure. Blame and change history.
- Single source of truth for the state of your infrastructure.
- Replicate environments in a quick, repeatable, documented way.
- Make environments easier to rebuild than repair, makes it simpler and quicker to resolve issues.

### Testing pyramid

Should be able to discover the majority of issues in the quickest running tests, to facilitate fast feedback and iteration.

![Pyramid shape showing the different levels of testing comparing their speed to receive feedback](TestingPyramid.jpg)

If an issue is found during integration or end-to-end testing, which usually take a long time to run, a unit test should be created that catches this issue. That way if the same issue happens again, it will be caught sooner and the developer can get faster feedback.

Not only should you improve your code coverage over time, you should aim to move tests down the pyramid so that issues continue to be found faster.

Manual tests can be run in parallel with automated tests (which also could be running in parallel with each other) to improve testing efficiency.

![Flow chart showing the stages of testing and which portions can be done in parallel](AutomatedandManualParallelTesting.jpg)

### Continuous integration

The purpose of creating and working in branches is so that multiple developers can work in parallel and minimize conflicts.

Developers then need to manually "integrate" their code with other work. This process can be error prone. Especially when you don't understand someone else's code that is causing a conflict. This issue is exasperated the more developers are working in parallel.

Continuous Integration solves this problem by getting everyone to integrate into the main branch daily as part of their normal work.

### No more pull requests

Pull requests were designed by GitHub to facilitate accepting contributions from unknown and untrusted developers on open source projects. They give maintainers an opportunity to confirm the code matches the style and ethos of the project.

Developers within an organization should already share the style and ethos of the team. They should be trusted to continue to promote that ethos in their code.

Pull requests sacrifice performance and quality. This is a worth while tradeoff for open source projects. They force the author to wait for review, causing their work to become stale. And they force reviewers to context switch to review code. Often they don't take the time necessary to actually understand the code because of time pressures, so errors slip though anyway even though all that time was spent!

Using pull request internally is like forcing your family through airport security just to enter your house. It's a costly solution to a different problem.

If your worry is having multiple pairs of eyes on the code before it's deployed, that should happen way earlier in the project, before hard to change decisions are solidified. For example, having multiple devs on a project, Pari programming, frequent code reviews, etc. These can replace pull requests and assist in automating the integration and deployment pipeline.

Related reference not from DevOps Handbook: [Pull Requests from Infrastructure as Code](https://infrastructure-as-code.com/book/2021/01/02/pull-requests.html) by Kief Morris

### Git strategies

**Optimizing for individual productivity**: individual branches. Quick progress in the beginning, but offload integration work to the end, or sporadically through the project. This is more like waterfall development.

**Optimizing for team productivity**: always integrating and working off of main. Main is always up to date with the latest work and in a deployable state. This is more like agile development.

When merging is difficult because of large batch sizes, developers become incentivized not to refactor code so that they keep their integration simple. This contributes to tech debt in the codebase growing over time.

### Continuous delivery

Any code that passes all tests, builds, and is deployable in a *production-like* environment will be automatically deployed to production.

Our code and environments should be designed and built in such a way that the release of new functionality to users does not require a change to our application code. We should decouple the ideas of releasing features to users and deploying code to an environment. They are two different acts serving two different stakeholders.

When we're deploying code all the time through continuous deployment and dark launching, releasing new functionality to users becomes simply a business and marketing decision.

Reference: [Continuous Delivery: Reliable Software Releases Through Build, Test, and Deployment Automation.](https://app.thestorygraph.com/books/77eb0975-4194-42e7-9db3-005d4250940f) by Jez Humble and David Farley

### Metrics

We should enable every developer to create metrics in their daily work and be able to display and analyze them.

Metrics should be visible and digestible for anyone in the company, and perhaps even publicly.

### A/B testing

If you don't have data about how whether a feature or change is performing compared to a baseline, then you don't know if the change has zero impact or even negative impact.

A study at Microsoft showed that through AB testing, 2/3 of proposed changed showed zero or negative impact.

Reference: [Release It!: Design and Deploy Production-Ready Software](https://app.thestorygraph.com/books/1bd756aa-7a60-4af1-a407-6311f9741421) by Michael T. Nygard

### Chatbots for automation

Example:
> @hubot deploy owl to production

The results of chatbot commands are returned to the chat.

- Everyone can see everything that is happening
- Engineers, on their first day of work, can see what daily work looked like and how it was performed
- People were more apt to ask for help when they saw others helping each other
- Rapid organizational learning was enabled and accumulated

### Single company-wide repository

Everyone has access to all the most up to date code easily.

Randy Shoup described the single repository as the most powerful mechanism for preventing failures at Google.

Tom Limoncelli described it as a competitive advantage for Google.


Let's say you've built an nice new static site using Next.js and you want to host it on GitHub Pages. Well there's a few steps to get it working, but it isn't too hard.

![next js logo and github logo](./NextJSGitHubPagesHero.png)

This article assumes you already have a website built locally and you're just looking to host it on [GitHub Pages](https://docs.github.com/en/pages/getting-started-with-github-pages). In the future I may write some how-tos for creating a blog using markdown and Next.js.

In this example I'm going to outline how I deploy to my main GitHub Pages repo for [superflux.dev](https://superflux.dev). It's hosted in my GitHub account's [main Pages repository](https://github.com/tyrelh/tyrelh.github.io). Everyone has access to this special repo. Just create a new repo with the title _your-github-username.github.io_. GitHub will serve the static content there on your own subdomain `https://your-github-username.github.io`. You can even connect a custom domain like I did, I'll talk about that optionally at the end.

You can also use this tutorial to push repo-specific pages. Google for that if your unfamiliar.

## Tl;dr

* [Static content](#static-content): A quick overview of how to build static assets for a Next.js site.
* [Install gh-pages package](#install-gh-pages-package): There's a lovely NPM package that makes it very simple to deploy static assets to GitHub Pages.
* [Branch setup](#branch-setup): I use two repositories on GitHub, one for my source code and one that is the Pages repository that gets hosted under your username.
* [Deploy script](#deploy-script): The deploy script uses the *gh-pages* package.
	* [Static content directory](#static-content-directory): Set the *out* directory in the deploy script that Next.js outputs static assets to when building and exporting.
	* [Remote & branch](#remote--branch): Set the remote and the branch that you want *gh-pages* to push your static assets to.
	* [Force push to remote](#force-push-to-remote): Allow the deploy script to force-push changes to your GitHub Pages repo.
	* [\_next hidden folder](#_next-hidden-folder): Some tweaks to make GitHub Pages play nicely with Next.js
* [\[Optional\] Configure a custom domain](#optional-configure-a-custom-domain): Allow a GitHub Pages site to be served over a custom domain name.
* [Resources](#resources)
* [Conclusion](#conclusion)

## Static content

GitHub Pages can only serve static content. You need to be able to export your Next.js project to static code. You can do this by running

```shell
next build && next export
```

`next export` will export the static code for your project to the _out/_ directory. This is the code we're going to want to host on GitHub Pages.

To simplify and make it easy to remember I added this as a `build` script in my *package.json*. Add/edit the `build` script:

```json
"build": "next build && next export"
```

## Install gh-pages package

I've used this package in the past and it works well for GitHub Pages websites. *[gh-pages](https://github.com/tschaub/gh-pages)* is a simple little tool that makes it easy to push projects to GitHub Pages repos or branches.

```shell
npm install gh-pages --save-dev
```

## Branch setup

I store my website source in a separate repository, in my case a repo called [personal-site-nextjs](https://github.com/tyrelh/personal-site-nextjs). By default *gh-pages* will deploy your static site to the `origin` remote. But we can change that. I like to keep the `origin` for my source repo pointing to the source remote.

* `personal-site-nextjs`: repo containing the source code for my website. I do development here
* `tyrelh.github.io`: my main GitHub Pages repo. This is were the compiled static assets are served from

You can also just use separate branches in the same GitHub Pages repository if you prefer. One for the source and one for the public assets. That feels messy to me so I prefer to keep them separate.

## Deploy script

Next we'll setup a deploy script in our _package.json_ to deploy our static content!

In your _package.json_ create a new script. Remember it should be on the same level as the other scripts, in the `"scripts"` block:

```json
"deploy": "gh-pages"
```

Now this by itself wont work yet, but I'm going to step you through each additional step to get the `gh-pages` command working. If you want to [skip to the end and just see the final script](#conclusion), go ahead!

### Static content directory

First you want to tell it what directory that your static assets are in. Remember this is the directory that `next export` in your `build` script outputs to. By default this is the _out/_ directory in your project. Make sure you run your build script first so that your static files are here.

Use the `-d` flag to set the directory.

```json
"deploy": "gh-pages -d out
```

### Remote & branch

We want to make sure `gh-pages` knows where it should push our static code to for hosting. It's good to set this up first so you don't accidentally push code somewhere you don't intend to.

For the `gh-pages` command, you can use the `-b` flag to tell it the remote branch you want to push to. Whatever the default branch of your _username.github.io_ repo is will work. `main` or `master` for example.

```json
"deploy": "gh-pages -d out -b main"
```

Next you'll want to set the remote. I like to hard code this in the script itself. Since it's going to be static for this project I don't see any reason not to. You can use the `-r` flag to set a remote URL. Copy the remote URL from your _username.github.io_ repo.

```json
"deploy": "gh-pages -d out -b main -r https://github.com/tyrelh/tyrelh.github.io.git"
```

### Force Push to Remote

When deploying new static content to your GitHub Pages repo, generally you're going to want to overwrite what is there. At least this is the simplest way to do it. Before setting this just double check that your branch and remote are set correctly. You don't want to overwrite anything you don't intend to!

Use the `-f` flag to force the push to the remote, overwriting files and changes that may exist on the remote.

```json
"deploy": "gh-pages -d out -b main -r https://github.com/tyrelh/tyrelh.github.io.git -f"
```

Ok! We're almost there! If this was a Jekyll project, or maybe a standard React project this would probably be all you need. But since we're using Next.js there is one more piece of config we need to do.

### \_next Hidden Folder

In your _out/_ directory of static assets, you should see a folder called _\_next/_. This is where Next.js puts your compiled JavaScript and CSS.

By default, GitHub pages actually ignores folders beginning with a \_. This is because GitHub Pages is setup to use Jekyll. Jekyll considers folders beginning with \_ to be hidden folders, so GitHub Pages just doesn't serve them.

You can deploy the code as is, and it will look like it worked since you can see the _\_next/_ directory in you _username.github.io_ repo. But any requests to those assets when you visit your website will result in a 404 not found error.

To fix this you need to add a file named _.nojekyll_ to the root of your _username.github.io_ repo. You can do this right on the GitHub website to test it out. Once that file is there and you give it a couple minutes to take effect, your website should work without 404 errors!

But now if you deploy your static code again that file will get erased, breaking your website. We need to incorporate this file into our deploy or build scripts. I put it at the beginning of my deploy script, but you could also put it at the end of your build script too if that makes more sense to you.

We want to create that `.nojekyll` file in our _out/_ directory after our code is built, but before we deploy it using `gh-pages`.

At the beginning of your deploy script, add an `echo` command to create the file. The file can contain anything, I just echo the string `true`. Use the `&&` between the `echo` and `gh-pages` commands so that they will run sequentially.

```json
"deploy": "echo 'true' > ./out/.nojekyll && gh-pages -d out -b main -r https://github.com/tyrelh/tyrelh.github.io.git -f"
```

This wont quite work, because by default `gh-pages` ignores hidden dotfiles! To tell `gh-pages` you want to include dotfiles (that's files beginning with a .), use the `-t` flag on the `gh-pages` command.

```json
"deploy": "echo 'true' > ./out/.nojekyll && gh-pages -d out -b main -r https://github.com/tyrelh/tyrelh.github.io.git -f -t"
```

And that's basically it!

## Configure your repo for GitHub Pages

You'll need to make sure your repo is configured for Pages. Go to your repo, click on the repo _Settings_, and click _Pages_ on the settings side bar. See the [GitHub Pages docs](https://docs.github.com/en/pages/getting-started-with-github-pages) for more general info about hosting static content.

## \[Optional\] Configure a custom domain

After you follow the [instructions here](https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site) to setup your custom domain with your GitHub Pages website, you'll notice that you need a _CNAME_ file in the root of your repo for the domain to map correctly.

Since our deploy process overwrites all the files in our GitHub Pages repo on each deploy, we need to add this file to our deploy/build process so it will be included each time. This is similar to what we needed to do for the _.nojekyll_ file.

At the beginning of your deploy script, add another `echo` command. You want to output the root domain for your website to a file called _CNAME_. In my case my root domain is `superflux.dev`.

```json
"deploy": "echo 'superflux.dev' > ./out/CNAME && echo 'true' > ./out/.nojekyll && gh-pages -d out -b main -r https://github.com/tyrelh/tyrelh.github.io.git -f -t"
```

Make sure to include the `&&` between each command so they run sequentially!

## Resources

* Allow the _\_next_ folder to get picked up in repo: https://github.blog/2009-12-29-bypassing-jekyll-on-github-pages/
* `gh-pages` package to easily deploy to GitHub Pages repositories: https://github.com/tschaub/gh-pages
* GitHub Pages docs: https://docs.github.com/en/pages/getting-started-with-github-pages
* GitHub Pages custom domain docs: https://docs.github.com/en/pages/configuring-a-custom-domain-for-your-github-pages-site/managing-a-custom-domain-for-your-github-pages-site

## Conclusion

And now that's it! To summarize we built a deploy script in our _package.json_ that looks like this

```json
"deploy": "echo 'true' > ./out/.nojekyll && gh-pages -d out -b main -r https://github.com/tyrelh/tyrelh.github.io.git -f -t"
```

You'll want to make sure your `-b` branch is pointing to the branch being served on your GitHub Pages repo, and that the `-r` remote is the URL of your GitHub Pages repo.

After building your website locally just run `npm run deploy` to push your static code to your GitHub Pages site! Remember the `gh-pages` package uses _git_ to push code to the remotes, so you'll need to configure your username, email, and credentials to authenticate the push to your GitHub Pages repo.

Reach out to me on Twitter [@tyreldelaney](https://twitter.com/tyreldelaney) if you have any questions, comments, or suggestions!

*Note: this article was written in April 2022. It sat for a while in my queue of articles as we were gearing up to have our first child. Hopefully nothing has changed drastically with Next.js and GitHub Pages that renders this info unusable.*


In my role as a software developer at my day job, I’ve been getting more interested in DevOps as a career trajectory. But what does that even mean? Do I even know?
  
This is a brief exploration into what DevOps is and how I see it relating to a small development team.

![DevOps Cycle from Atlassian](development-cycle.jpg)

## Index

* [Tl;dr](#tldr)
* [Initial Assumptions](#initial-assumptions)
* [Operations & Facilitators](#operations--facilitators)
* [Agile Development](#agile-development)
	* [Quick Iteration](#quick-iteration)
	* [Feedback & Data](#feedback--data)
* [Automation](#automation)
* [Not Only Software Development](#not-only-software-development)
* [Conclusion](#conclusion)
* [Resources](#resources)

## Tl;dr
Here are a few key ideas that I think define what DevOps is and what someone in a DevOps position should focus on:

* Automate things like dev environment setup and employ tools to make writing code simple. Make it easy and painless to write code.
* Automate testing so developers and teams are confident in the code being produced. Developers shouldn't have to "remember" to run tests.
* Automate infrastructure provisioning and blue/green deployments. Code that's reviewed and ready to go should be able to reach customers with a single click (with proper authentication). Make deploying new code trivial.
* Automate data collection and illumination for the dev team primarily but also for the business side depending on the size of your company.
* And I think in general, a DevOps role should facilitate software development by increasing confidence in work being done by both developers and product managers.

## Initial Assumptions

Earlier on in my career and in university, DevOps was a buzz word that was thrown around all the time. When I heard the term DevOps I associated that with “cloud infrastructure”. I was connecting the idea of “building things on cloud infrastructure” to the role of DevOps.
  
While using cloud infrastructure might be one aspect of the tools and responsibilities of a DevOps role, it doesn’t get to the root of what the role is about.

## Operations & Facilitators

Within a software company, big or small, there are various so-called “Operations” roles. Business Operations, Customer Operations, Insert Anything Operations. My perspective on these roles in general is that they are facilitator roles rather than managerial roles. “Managers” may often have an Officer title, while “facilitators” may have an Operations title.

What do I mean by Facilitators? These are roles that essentially amplify the work in a given area or team. Their job is to ensure process and procedure are in place so that work can get done effectively, impactfully, and efficiently. It’s not necessarily their job to oversee the work being done, but to boost the impact and productivity of the work being done.

DevOps is short for Development Operations. And in my context as a software developer, Development is actually short for Software Development (Development can mean many different things in different fields).
  
So in more plain language, the role of DevOps is related to the operations of the development team and the development lifecycle. Ok, so that sounds more appropriate than just “cloud infrastructure”, but what is involved in the operations of those things?

## Agile Development

While I’m no expert on Agile development practices, I’ve been working with variations of it for my entire software development career thus far.

In essence, Agile (with a capital A) is intended to make the development process more iterative and quick to react to user feedback. Rather than blocking out big periods of time for large projects that overhaul huge portions of the customer experience, we attempt to release work quicker and in smaller pieces to get feedback on particular features sooner.

This allows us to not waste time perfecting features that our customers don’t want or won’t use. Instead we can focus more directly on things that are impactful for our customers and our bottom line. Therefore, one aspect of a DevOps role is facilitating the Agile development process.

### Quick Iteration

Getting code written, reviewed, and deployed quickly helps you then iterate on that code sooner, leading to better experiences for the customer over time. Building and improving processes around code testing, review, and deployment is a key facilitator opportunity for a DevOps role. This can cover many different things, from automating testing and deployments to improving code review process.

### Feedback & Data

Timely feedback is also vital to the Agile development process. Both user experience feedback and systems operation feedback (through statistics and error reporting). This feedback informs subsequent development cycles and how features, tweaks, and bug fixes are prioritized against each other for the greatest impact. Developing systems to effectively and concisely collect and relay feedback to the appropriate developers and project leads allows for more informed decision making, and thus is another key responsibility of a DevOps role.

## Automation

There are many pieces of manual work and human intervention in the software development cycle, and the level of automation in place can vary wildly between different teams and companies. I entered my software development career from essentially a cloud-native perspective, having little to no experience deploying code in a different paradigm.

But today, with things like IaC (infrastructure as code) and remote dev environments being the norm, every aspect of the developer experience can be automated in some way. Including writing code believe it or not.

* __Dev environment setup can be automated with old tools like Docker, or emerging tools like [JetBrains Space](https://www.jetbrains.com/space/).__ Space is essentially a remote dev environment that helps teams standardize their development environments across a team. Space is in closed beta as of the writing of this article. GitHub & Microsoft are working on similar tools with GitHub Codespaces and VS Code.
* __Writing code can be automated with new tools like [GitHub Copilot](https://copilot.github.com/).__ Copilot is a machine learning tool that has learned how to write code in many popular languages. While it can't directly replace a human developer yet, it can greatly speed up the process of writing code from scratch.
* __Reviewing, unit-testing, and merging code can be automated with tools like GitHub and [GitHub Actions](https://github.com/features/actions).__ There is still a bit of human touch needed in the review process to ensure features are truly meeting the business needs of the brief, but reviewing security and dependencies for example can be greatly sped up with the help of automation tools. Running tests can be automated with a variety of CI tools. Merging code and resolving merge conflicts has become downright trivial with tools like git and GitHub.
* __Deploying code to infrastructure can be automated with a variety of CD tools.__ I've been mainly interested in GitHub Actions and automating deployments to AWS. But I'm now also interested in learning more about AWS CDK (cloud development kit). [This blog post from GitHub](https://github.blog/2021-01-25-improving-how-we-deploy-github/) has been quite influential in my current thinking about automated deployments.
* __App stats and error monitoring can be automated with tools like Sentry, Datadog, and NewRelic.__ Those stats can be viewed through there respective dashboards or made more accessible through things like Slack bots.

## Not Only Software Development

In a small company or on a small team, often certain roles wear many hats. In our case, business operations bleed into the development team all the time. People working with clients, company finances, product managers, or even the CEO, will often have inquiries for data about various aspects of our business operations.

Without adequate access to that data, requests can sometimes fall on developers who are familiar with the system and the data. Ad-hoc SQL queries to our database can sometimes be the norm, but this isn’t an efficient use of anyone's time.

Building systems and interfaces to our company’s data can be integral to operations and development. Depending on the size of the company and the other roles there, this idea of data illumination could fall on a Business Operations role, but often it can require integration and understanding of the technical stack that is foundational to your application. Therefore that responsibility can fall on a DevOps role, to build digestible interfaces and secure pipelines to get the relevant data to the team members that need it.

## Conclusion

In summary, I see DevOps as primarily a facilitator and multiplier role. Depending on the size of the team they could be simultaneously doing development themselves, but the focus is on how to make the development team more productive.

Facilitating the development team to be more productive means a few things:

* Make code easy and painless to write. Automate environment setup.
* Make code simple to test. Automate tests.
* Make code trivial to deploy when ready. Automate infrastructure and blue/green deployments.
* Make data about development, testing, and deployment visible to the team. Automate data collection and illumination with tools like Datadog.

My summary and this article represents my understanding and opinions right now (Spring 2022). I am constantly learning and improving on my career journey and my knowledge and opinions will undoubtedly change over time.

If you have any comments or suggestions please feel free to reach out to me on Twitter [@tyreldelaney](https://twitter.com/tyreldelaney)! 👋🏻

## Resources

* [The Phoenix Project](https://itrevolution.com/the-phoenix-project/) - Gene Kim, Kevin Behr, and George Spafford
* [Improving how we deploy GitHub](https://github.blog/2021-01-25-improving-how-we-deploy-github/) - GitHub Blog
* [DevOps principles](https://www.atlassian.com/devops/what-is-devops) - Atlassian


Here are some things that I'd like to learn more about in 2022. This is kindof a reference for myself. So I can look back at this next year and reflect on what I learned and what I wanted to learn. [Let me know on twitter](https://twitter.com/tyreldelaney) if you are learning these same things or you have any tips or resources to share!

![Logo montage of all the technologies outlined in this article](what-im-learning-in-2022.jpg)

## Index

[Learning from scratch](#learning-from-scratch)

* [Docker](#docker)
* [Vim/NeoVim](#vimneovim)

[Learning more of](#learning-more-of)

* [Github Actions](#github-actions)
* [AWS DynamoDB](#aws-dynamodb)
* [AWS Client VPN, VPNs in general, and SSO](#aws-client-vpn-vpns-in-general-and-sso)
* [SSH](#ssh)
* [HTTPS/SSL](#httpsssl)
* [Raspberry Pi](#raspberry-pi)
* [Home Network](#home-network)

[Things I'm interested in but might not get to this year](#things-im-interested-in-but-might-not-get-to-this-year)

* [A better static site generator / CMS](#a-better-static-site-generator--cms)
* [Webhooks](#webhooks)

## Learning from scratch

### Docker
Docker really feels like one of those technologies that I've been conveniently ignoring for a little too long. I think what I'll do to start is simply try to containerize some smaller projects I'm working on this year to dip my toes in.

I acquired a handful of second-hand Raspberry Pis not long ago and I've got a few ideas for some small home automation & monitoring projects that might be perfect simple projects to dockerize.

A stretch goal would be to work up to containerizing our monolithic app at my day job.

### Vim/NeoVim
I've always just barely snuck by with my fluency in the terminal. The more I get into networking and Dev Ops the more this deficiency is showing. In general I'd just like to work in the terminal a bit more this year.

Specifically I'm going to try to develop a couple Node and Python projects I'm cooking up remotely on the Raspberry Pis I hope to run them on.

Really two main goals I'd like to accomplish with this practice:
1. Get proficient and comfortable using vim.
2. Learn more about making a portable and lightweight dev environment. Something that I can easily spin up on a Raspberry Pi or AWS EC2 instance over SSH and feel comfortable doing real work in.

## Learning more of

### Github Actions
Automation is a big focus for me this year. And really just because I rely on Github both personally and professionally, Github Actions is a natural choice to skill up in.

We're now looking into automating and outsourcing our test suite at work and since we use Github for source control and code reviews/pull requests it makes total sense to use Github Actions.

I've automated a few things using Github Actions already, namely [tests and deployment of my](https://github.com/tyrelh/battlesnake-typescript-node) [Battlesnake](https://play.battlesnake.com/) to AWS Elastic Beanstalk. Automating even just tests for our app at work will be a bit more of a challenge since it's a much older stack and has many more moving parts.

### AWS DynamoDB
I wasn't actually planning on learning DynamoDB at the start of the year. But a project idea came to mind and the possibility of using Dynamo to store data rather than something like S3 sounded intriguing.

Essentially what I'm planning is building out a few Raspberry Pi temperature sensors for my house. Maybe 4 or 5 in total monitoring a few rooms in the house as well as the outdoor temperature.

The idea would be to collect data from all these sensors using Raspberry Pis maybe every 5 mins or so, and ship that data off to Dynamo. Then building an infrastructure to rotate through tables. Basically make a new table each day with a high write provision, and rotate down the previous days tables read and write provisions. Maybe after some period archive the data in S3.

Might be over engineering for the task, but it's for learning.

[Best Practices for Handling Time Series Data in DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/bp-time-series.html)

### AWS Client VPN, VPNs in general, and SSO
With my day job going fully remote this year, setting up secure remote tools has been high on our priority list. Part of that will be learning about AWS Client VPN which we may use to grant our devs access to some of our AWS infrastructure.

Part of this work is also moving to more zero trust architecture, and leaning into SSO for authentication.

Also I recently acquired a Synology NAS for my home (which we actually use to use in our office before we gave that space up). I'm looking to centralize alot of my personal cloud use to the Synology and move away from services like Google Drive (office and documents), iCloud (photos), and Notion (notes). Part of that will be to setup a VPN at home so that I can use those services hosted in my home from anywhere. As well as accessing some of my Raspberry Pis for remote development.

### SSH
Learning more about SSH and tunneling goes hand-in-hand with a few other learning priorities I have. Learning best practices for connecting securely and crafting secure infrastructure. Also using SSH to practice remote development and learning terminal based tools.

### HTTPS/SSL
Same as above. Learning more about best practices around secure remote communication. Getting more confident understanding certs and how they work.

### Raspberry Pi
I've owned a Raspberry Pi 3 for a handful of years now, but I've never really had a project for it. I experimented a bit with retro game emulation on it, but those projects are pretty trivial to setup.

Now I have a handful of Raspberry Pi 2s and 3s and I have some plans for some smaller single purpose projects for the house that will be perfect for them.

### Home network
This kinda just happened organically, but I have a lot of computers in my house now! What I'd like to do is centralize a bunch of things into a proper rack. I have a few computers that are mining cryptocurrency that I'd like to transfer to rack-mountable server cases. Along with my new Synology NAS and a few Raspberry Pis.

Wiring my house with Cat-6 probably won't happen any time soon, but centralizing a bunch of things in a rack will let me hard-wire them all to a small switch without needing to run cables through all my walls.

Like I mentioned above, also setting up network storage and remote access for my home network.

## Things I'm interested in but might not get to

### A better static site generator / CMS
My website currently is hand-built in Javascript and React with a few plugins that let me render my articles from markdown. There is still a fair amount of manual process to get each markdown file wired up to display as a page and route, but at least I don't need to reformat the article. I can just paste it in from my notes app and then wire it up.

Ideally I'd like to learn a better static site generator to simplify this process even more. I've been thinking of learning [Gatsby](https://www.gatsbyjs.com/) for a while now, but I've just found it a bit cumbersome and hard to get into.

Some alternatives I'm considering learning instead of Gatsby are [Jeckyll](https://jekyllrb.com/) or [Hexo](https://hexo.io/).

### Webhooks
Just something I'm interested in learning more about.

## End

[@ me on twitter](https://twitter.com/tyreldelaney) if you are learning these same things or you have any tips or resources to share!


During a recent project to convert my JavaScript & Node [Battlesnake](https://play.battlesnake.com/) over to TypeScript & [Deno](https://deno.land/), I decided I would also create a more robust suite of tests for it.

![Deno and Github logos with an addition symbol between them](DenoTestsAndGithubActionsCI.png)

If you are unfamiliar with [Battlesnake](https://play.battlesnake.com/), it is an online and in-person programming competition where you build a AI in the form of a REST API that can play a multiplayer version of the classic game [Snake](https://en.wikipedia.org/wiki/Snake_(video_game_genre). You can check out details on their [website](https://play.battlesnake.com/) to learn more, and even watch some past events on their [Twitch channel](https://www.twitch.tv/BattlesnakeOfficial).

## Tl;dr
* [Deno Tests](#deno-tests): Deno includes a very simple test runner framework.
* [Github Actions](#github-actions): Github actions allow you to automate tasks based on triggers within your Github repo.
* [Create the workflow](#create-the-workflow): A .yml file describes a series of actions you would like to perform based on a trigger.
* [Automate Deno tests](#automate-deno-tests): Just add a `run: deno test` command to your workflow .yml.

## Deno Tests
[Deno](https://deno.land/) includes a simple test runner for tests in js, ts, jsx, or tsx files. It can be run with:

```bash
deno test
```

To create tests, all you need to use is use the `Deno.test()` method included with Deno within files ending in *test* (`{*_,}test.{js,ts,jsx,tsx}`) anywhere in your project structure. I put mine in *tests/*.

Example of a basic test:

```typescript
import { assert } from "https://deno.land/std/testing/asserts.ts";
import { root } from "../app/main.ts";

Deno.test("Root response contains expected fields", () => {
  // when
  const result = root();
  // then
  assert(result?.apiversion);
  assert(result?.author);
  assert(result?.head);
  assert(result?.tail);
  assert(result?.color);
});
```

This test is simply asserting that values exist for these specified fields.

You can add many tests to single files by calling `Deno.test()` over and over again. You can also make as many test files as you like, Deno will automatically execute all the files it recognizes ending in *test* (`{*_,}test.{js,ts,jsx,tsx}`).

Visit the [Deno Testing docs](https://deno.land/manual/testing) for more info about creating tests.

Easy! So now I can create a robust set of unit tests for my Battlesnake AI.

Since the game Battlesnake works by [sending you the entire game state for every move](https://docs.battlesnake.com/references/api), I can also save the game state JSON from particularly tricky moves and use them within their own tests. The output can be compared against the single best move, or even a set of equally good moves if their are multiple. Creating tests like this tests the behaviour of the AI overall and ensures that changes to the behaviour aren't making the AI perform worse on moves it previously was able to do correctly.

So now that I have tests up and working, I wanted to automate them using Github Actions.

## Github Actions
Github Actions are still quite new to me and I hadn't touched them at all before this project. I just assumed it would be complicated to set up.

For what I wanted to do for this project it was actually trivial. Couldn't have been easier.

What I wanted was to run these tests automatically whenever pushing to my repository. I figured this Deno app is pretty simple all things considered, so I shouldn't have to mess around with Docker or anything like that.

I'll step you through what I did.

## Create the workflow
Github Actions are based around these things called workflows. They are essentially .yml scripts that describe the series of events you want to perform in your action.

If you click the Actions tab within the repo you want to create the Action you will be presented with a Get started with GitHub Actions page. This page has many example actions you get you started for so many different languages, frameworks, and platforms. It will even suggest actions to you based on the contents of your repo. I chose the *Skip this asn set up a workflow yourself* option.

![Get started with Github Actions](DenoAndGithubActions-get-started-with-github-actions.png)

This plops you into a web editor and gives you a basic example workflow .yml file to start with. It looks something like this:

```yaml
# This is a basic workflow to help you get started with Actions

name: CI

# Controls when the action will run. Triggers the workflow on push or pull request
# events but only for the master branch
on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
  # This workflow contains a single job called "build"
  build:
    # The type of runner that the job will run on
    runs-on: ubuntu-latest

    # Steps represent a sequence of tasks that will be executed as part of the job
    steps:
    # Checks-out your repository under $GITHUB_WORKSPACE, so your job can access it
    - uses: actions/checkout@v2

    # Runs a single command using the runners shell
    - name: Run a one-line script
      run: echo Hello, world!

    # Runs a set of commands using the runners shell
    - name: Run a multi-line script
      run: |
        echo Add other actions to build,
        echo test, and deploy your project.

```

With this you can actually click the *Start commit* button and that will guide you to commit this .yml file to your repo within the *.github/workflows/* directory.

You can see near the top of that file that there are triggers for `on push` and `on pull_request` attached to the master branch. You can try to push something to master and see it work!

If you visit the Actions tab of your repo now you will see your Actions and the history of all the times they have run. You can click into them for more details and to see the console outputs. Also by default it will email your Github account email whenever an action fails. I imagine this can be changed somewhere if you don't like that.

![Github Actions](DenoAndGithubActions-github-actions.png)

## Automate Deno tests
Now I wanted to try to run my tests using Deno. I knew this would involve installing Deno somehow so I searched around.

I found [this repo](https://github.com/denolib/setup-deno) from [denolib](https://github.com/denolib) with exactly what I wanted.

Really all you need to do is add the `uses` line pointing to this repo to your workflow .yml file under the `steps` section. You can also include the Deno version you prefer. Then within the `run` section you now have access to `deno` and all of its functions.

So now my workflow .yml file looks like this:

```yaml
name: CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@master
    - uses: denolib/setup-deno@master
      with:
        deno-version: v1.x
    - run: deno run https://deno.land/std/examples/welcome.ts
```

One really cool feature of Deno is that you can run scripts from the web right in the terminal, and you can also do this right within Github Actions. That is what the URL is doing in

```yaml
- run: deno run https://deno.land/std/examples/welcome.ts
```

You can run this and then look within your Github Actions for the console output from this example *welcome.ts* file.

So now to run my tests, all I needed to do was change that `deno run` command to `deno test`. My final .yml workflow looks something like this:

```yaml
name: CI

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  run-tests:
    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@master
    - uses: denolib/setup-deno@master
      with:
        deno-version: v1.x
    - run: deno test
```

Now every time I push to my master branch all my tests will be run, including any new tests I just committed, and I will be notified both on the Github UI and in my email if any tests fail. I can then log into Github and look at the output to see what failed.

I hope this small guide helped you out and showed you just how simple both Deno and Github Actions are!


#platform

Shipping ECS Fargate logs to Datadog with FireLens (and what broke)

October 20, 2025 10 Minute Read

Book Notes: DevOps Handbook, The

August 27, 2023 8 Minute Read

Deploy a Next.js Static Site to GitHub Pages

July 23, 2023 8 Minute Read

What is DevOps?

April 10, 2022 7 Minute Read

What I'm Learning in 2022

March 23, 2022 5 Minute Read

Node CI/CD Pipeline using GitHub Actions & AWS Elastic Beanstalk

May 30, 2020 24 Minute Read

Deno Tests & GitHub Actions CI

May 23, 2020 6 Minute Read

Get in touch