Preface

What is this book?

This book is a guide about:

• Software delivery: how to run & maintain software in production?

• DevOps: a methodology to make software delivery more efficient

Instead of focus on culture & theory, this book focuses on hand-ons guide:

• Step-by-step examples about how to run real system & real code.
• Core concepts & best practices of modern DevOps and software delivery.

Why this book exists?

• There is no hands-on guide that teach software delivery end-to-end.

• Software delivery is current learned in the hard way - through trial and error - which can be very costly (outages, data lose, security breaches…)

[!NOTE] The author learned from his experience when he worked at LinkedIn in 2011:

• LinkedIn’d just IPO, share price was up 100%, revenue was growing 100% by year, 100M members, growing fast.
• From the inside, the company was in turmoil because of software delivery - a $10 billion company could not deploy code:
  • They deployed once every 2 weeks through a painful, tedious, slow, error-prone way
  • A deployment went so bad, that it could not be completed; new changes, some fixes, more issues…Team worked overnight several days, then everything was roll-backed.
• They kicked of Project Inversion:
  • new features development was freezed for several months
  • the entire engineering, product, design team reworked all the infrastructure, tooling, technique
• Months later, they could deploy dozens of times per day:
  • with fewer issues, outages
  • allowing the whole company move much faster

[!NOTE] How did they do that?

• They didn’t know what they didn’t know

• They learn about best practices from the industry:

  • Trunk-based development (from one company)
  • Canary deployment (from another)
  • Feature toggles (from another)
  • …

• Most developers don’t know what they don’t know:

  • About software delivery and DevOps
  • Best practices that top tech companies had figured out

• This book helps you learn from the experience of others so you can build software faster, more reliably and more securely.

[!WARNING] The results from adopting DevOps can be wonderful, but the experience along the way may be not.

Watch out for snakes

• “DevOps” is used to describe a lot of unrelated concepts. ⛓️‍💥🌕🌑

  e.g. A TLS certificate (& the cryptography behind it), a deployment pipeline, and backing up data from a database.

[!NOTE] What makes DevOps hard? (It’s not the complexity of the concepts)

• It’s the number of concepts to master (DevOps is an incredibly broad surface area)
• It’s how to get everything connected together correctly (or nothing works at all)

• “DevOps” is a box of cables. 🧰⛓️

  You pull out a cable but end up with a giant mess where everything is tangled together

[!TIP] This book try to untangle this mess of cables:

• Each cable in that mess is in fact a separate cable.
• In isolation, each concept in DevOps (a cable in that mess) is within your grasp.

• Sometimes, DevOps even feels like a box of snakes. 🧰🐍🐍

  You pull of a cable but end up getting bitten.

[!CAUTION] DevOps is current a giant mess:

• A new industry
• Tools, techniques aren’t mature
• It often feels like everything is broken, frustrating & hopelessly tangled

• In DevOps, each time you learn a new buzzword (a new concept):

  • it comes with 10 more unfamiliar buzzwords (it’s a mess of cables)
  • or it might try to by you (a cable or a snake)

  but stick with it & watch for the snake

Who should read this book?

Anyone responsible for deploying & managing apps in production:

• Individual contributors in operations roles: SREs, DevOps Engineers, SysAdmins…, who wants to level up about software delivery & DevOps.

• Individual contributors in development roles: Software Engineers/Developers.., who wants to learn about the operations side.

• Managers: Engineering Managers, CTOs…, who want to adopt DevOps & software delivery best practices in their organizations.

What is in this book?

What isn’t in this book?

Code examples

• This book includes many examples to work through, which is available at GitHub repository: https://github.com/brikis98/devops-book

• The code samples are organized

  • by chapter (e.g. ch1, ch2),
    • and within each chapter, by tool (e.g. ansible, kubernetes, tofu)

[!TIP] The examples show what the code looks like at the end of a chapter.

To maximum the the learning:

• write the code yourself
• check the “official” solutions at the end

Opinionated Code Examples

The code examples represents just one opinionated way to implement this book core concepts - IaC, CI/CD…

[!IMPORTANT] In real world, there is no single “best” way that applies to all situations:

• All technology choices has a trade-off.
• Some solution maybe a better fit in some situations that others.

Always use your judgment to pick the right tool for the job.

[!NOTE] The core concepts in this book only change & evolve over a long time span (5-10 years). But the code examples that implement these core concepts may change more frequently. e.g. Kubernetes has a release cycle of 4-month¹.

You Have to Get Your Hands Dirty

This book will teach you principles, techniques, tools about DevOps & software delivery.

But you can only achieve serious results if you learn by doing:

• re-create the example code yourself

  • writing code
  • running code
  • make the code work

• do the extra get your hands dirty section & tweak the examples

  • customize to your needs
  • break things
  • figure out how to fix them
  • …

Using Code Examples

The code examples in this book may be used

• in your programs and documentation (but not too much)
• but not for selling & distribution

Chapter 1: An Introduction to DevOps and Software Delivery

I wrote an app. Now what?.

Delivery it to users! But how?

• AWS, Azure or Google Cloud (Heroku, Vercel, Netlify)?
• One server or multiple servers?
• Docker? Kubernetes?
• VPC? VPN, SSH?
• Domain name? DNS, TLS?
• Backup database?
• Why the app crashed? How to debug it?

A Primer on DevOps

Why DevOps matters?

• The gap between companies with world-class software delivery and everyone else is 10x, 100x or even more.

• Dora’s software delivery metrics:

  • What is it?

    … of software changes	Metrics	Description
    Throughput…	🚀⏱️ Lead time	How long it takes a change to go from code committed to deployed in production?
    	🚀🔢 Deploy frequency	How often does your organization deploy code to production?
    Stability…	🆘🚨 Change fail percentage	How often deployments cause failures that need immediate remediation (e.g. hotfix, rollback)
    	🚧⏱️ Recovery time	How long it takes to recover from a failed deployment?

  • Performance according to 2023 State of DevOps Report

    Metrics		Low performers	Elite performers	World-class performers	Elite vs low performers
    🚀⏱️ Lead time	Deployment processes takes…	36 hours	5 mins	In minutes (100% automated)	10-200x more often
    🚀🔢 Deploy frequency	Deploying …	Once/month	Many/day	Anytime (Thousands/day)	10-200x faster
    🆘🚨 Change fail percentage	The rate of deployment causing problems…	2/3	1/20	Detect in seconds (before user-visible impact)	13x lower
    🚧⏱️ Recovery time	Outages last	24 hours	2 mins	In minutes (sometimes automated)	700-4000x faster

• It’s possible to achieve the performance of the elite (or even the world-class):

  • Each of these performers may do it a little differently
  • But in common, most of these performers share a lot of best practices.

[!IMPORTANT] The DevOps movement is an attempt to capture some of the best practices from the world-class performers in DORA software delivery metrics.

Where DevOps Came From

Before DevOps

• Building a software company …

  • write the software

    • which is handled by the Developers - Dev team

  … also means manage a lot of hardware:

  • setup cabinets, racks -> load with servers -> install wiring, cooling -> build redundancy power systems…

    • which is handled by the Operations - Ops team

• An application would be:

  • built by the Dev team, then
  • “tossed over the wall” to the Ops team

  The Ops team had to figured out the software delivery:

  • how to deploy, run, maintain… it.

  • most was done manually:

    • manage the hardware
    • install the app & dependencies

• The company eventually run into problems:

  • release are manual, slow, error-prone
  • frequent outages, downtime

  The Ops team

  • reduce the release cycle (because they can handle all these manually things)
  • but each release is bigger, causing more problems

  Teams begin blaming each other, silos form…

After Devops

• Instead of managing their own hardware (or data-centers)

  • many companies take advantage of cloud providers (e.g. AWS, Azure, Google Cloud)
  • many Ops teams spend their time working on software - e.g. Terraform, Ansible, Docker, Kubernetes - to manage the hardware.

• Both Dev & Ops teams spend most of their time working on software:

  • The distinction between the two team is blurring.

  • There may still a separation of responsibility …

    • The Dev teams are responsible for the application code
    • The Ops team are responsible for the operation code

  • …but both teams need to work more closely together…

• There come the DevOps movement with the goal of

  • making software delivery vastly more efficient
  • (building better software faster)

  by moving to the cloud & shifting to DevOps mindset:

  	Before	After	After Example
  👥 Teams	Devs write code, “toss it over the wall” to Ops	Devs & Ops work together on cross-functional teams
  🧮 Servers	Dedicated physical servers	Elastic virtual servers	AWS’s EC2
  🌐 Connectivity	Static IPs	Dynamic IPs, service discovery
  🛡️ Security	Physical, strong perimeter, high trust interior	Virtual, end-to-end, zero trust
  ⚡ Infrastructure provisioning	Manual	Infrastructure as Code (IaC) tools	Terraform
  🔧 Server configuration	Manual	Configuration management tools	Ansible
  ✅ Testing	Manual	Automated testing	CI
  🚀 Deployments	Manual	Automated	CD
  💱 Change process	Change request tickets 🎫	Self-service 🏧
  🔢🔄 Deploy cadence (Deploy frequency)	Weeks or months	Many times per day
  🔢🔁 Change cadence (Lead time)	Weeks or months	Minutes

• DevOps movement has transformed a lot of companies:

  • Nordstrom:
    • number of features delivered by month increased 100%
    • defects reduced 50%
    • lead time reduced 60%
    • number of production accidents reduced 60 - 90%
  • HP’s LaserJet Firmware:
    • the amount spent on developing features went from 5% to 40%
    • development cost reduced 40%
  • Etsy:
    • From infrequent deployments to 25-50 deployments/day

The Evolution of DevOps Software Architecture & Software Delivery Process

The architecture & software delivery process evolution can be broken down into:

• 3 high-level stages
  • each stages consists of 3 steps

Stage 1

Stage 1 applies to most software projects start: new startups, new initiatives (at existing company), side projects.

• Step 1:

  • Single server: everything runs on a single server
  • ClickOps (Process): manage infrastructure & deployment manually

  User -> SERVER
  

• Step 2:

  • Standalone database: database become a bottleneck -> break it to a separate server
  • Version Control (Process): team grows -> collaborate & track changes
  • Continuous Integration (Process): reduce bugs/outages -> automated tests

       User -> Server -> DATABASE
  
  Developer ->  VERSION + CONTINUOS
                CONTROL   INTEGRATION
  

• Step 3:

  • Multiple servers: a single server is not enough
  • Load Balancing: distributed traffic across servers
  • Networking: protect servers -> a private networks
  • Data Management: scheduled backups, data migration
  • Monitoring (Process): get better visibility of system

                ---- VPC ----------------------------
               |                             BACKUPS |
               |                  SERVER        ↑    |
       User -> | LOAD BALANCER -> SERVER -> Database |
               |                  SERVER             |
               |                   ...               |
                --------------------------------------
  
  Developer ->  Version + Continuos   + MONITORING
                Control   Integration
  

State 1 is

• simple
• fast to learn, easy to set up
• fun to work with

Most software projects never need to make it past stage 1.

[!NOTE] If your application is so good and the number of users keep going - in other words, you have scaling problems - you may have to move on to the subsequent stages.

[!CAUTION] Only move to the subsequent stages, if you’re facing problems that require more complex architecture & processes to solve.

• These complexity has a considerable cost.
• If you’re not facing these problems, then you can and should avoid that cost.

Stage 2

Stage 2 applies to larger, more established companies software that has larger user bases and more complexities.

• Step 4:

  • Caching for data stores: database is still a bottleneck -> add read replicas & caches
  • Caching for static content: traffic grows -> add CDN for content that doesn’t change often

                ---- VPC -----------------------------------------
               |                                         Backups |
               |                                            ↑    |
       User -> | CDN     -> Load balancer -> Servers -> Database |
               | (CACHE)                                    ↓    |
               |                                          CACHE  |
                --------------------------------------------------
  
  Developer ->  Version + Continuos   + Monitoring
                Control   Integration
  

• Step 5: team size become a problem, deployment is slow, unreliable

  • Multiple environments: to help teams do better testing. Each env is a full copy of infrastructure, e.g. dev, stage, prod
  • Continuous delivery (Process): fast/reliable deployment -> deployment pipeline
  • Authentication & secrets (Process): a little security

                ---- VPC ------------------------------------- _
               |                                      Backup  | |_
               |                                        ↑     | | |
       User -> | CDN -> Load balancer -> Servers --> Database | | |
               |                                  ↓           | | |
               |PROD                            Cache         | | |
                ----------------------------------------------  | |
                 |STAGE                                         | |
                  ----------------------------------------------  |
                   |DEV                                           |
                    ----------------------------------------------
  
  Developer ->  Version + Continuos   + CONTINUOS + Monitoring + AUTH,
                Control   Integration   DELIVERY                 SECRETS
  

• Step 6: teams keep growing to keep moving quick

  • Microservices: allow teams work independently, each microservice comes with its own data store & caches.
  • Infrastructure as Code (Process): infrastructure for all microservices is a too much to be managed manually.

                ---- VPC ---------------------------------------------------- _
               |                                              Cache  Backups | |_
               |                                                ↑      ↑     | | |
               |                 ------> SERVICES <-> SERVICES --> Database  | | |
               |                |           ↕      ↕      ↕                  | | |
       User -> | CDN -> Load balancer -> SERVICES <-> SERVICES --> Database  | | |
               |                                                ↓      ↓     | | |
               |prod                                          Cache  Backups | | |
                -------------------------------------------------------------  | |
                 |stage                                                        | |
                  -------------------------------------------------------------  |
                   |dev                                                          |
                    -------------------------------------------------------------
  
  Developer ->  Version + Continuos   + Continuos + Monitoring + Auth,   + INFRASTRUCTURE
                Control   Integration   Delivery                 Secrets   AS CODE
  

Stage 2 represent a significant step up in terms of complexity:

• The architecture has more moving parts
• The processes are more complicated
• The need of a dedicated infrastructure team to manage all of this.

Stage 3

Stage 3 applies to large enterprises with massive user bases.

• Step 7: massive user bases

  • Observability: More visibility <- Tracing + observability
  • Service discovery: So many microservices, how to communicate with each other?
  • Server & networking hardening -> Compliance standard, e.g. PCI, NIST, CIS
  • Service mesh: Unified solution for manage microservices -> all items about + load balancing + traffic control, error handling

                ---- VPC ---------------------------------------------------------------- _
               |                                                          Cache  Backups | |_
               |                     ----------------------------------     ↑      ↑     | | |
               |                    |      Services <--> Services-----|------> Database  | | |
               |                    |                                 |                  | | |
               |                    |           OBSERVABILITY         |                  | | |
               |                    |                                 |                  | | |
               |                    |                                 |                  | | |
       User -> | CDN -> Load     -> |       ↕     SERVICE      ↕      |                  | | |
               |        balancer    |            DISCOVERY            |                  | | |
               |                    |                                 |                  | | |
               |                    |                                 |                  | | |
               |                    |            HARDENING            |                  | | |
               |                    |                                 |                  | | |
               |                    |      Services <--> Services-----|------> Database  | | |
               |                     ---------------------------------      ↓      ↓     | | |
               |prod                            SERVICE MESH              Cache  Backups | | |
                -------------------------------------------------------------------------  | |
                 |stage                                                                    | |
                  -------------------------------------------------------------------------  |
                   |dev                                                                      |
                    -------------------------------------------------------------------------
  
  Developer ->  Version + Continuos   + Continuos + Monitoring + Auth,   + Infrastructure
                Control   Integration   Delivery                 Secrets   as Code
  

• Step 8: a lot of data from users

  • Analytics tools: process & analyze data <- data warehouse/lake, machine learning platforms…
  • Event bus: more microservices, more data -> event bus -> event-driven architecture
  • Feature toggles & canary deployment (Process): deploy faster, more reliable <- advanced deployment strategies

                ---- VPC -------------------------------------------------------------------------- _
               |                                                          Cache  Backups           | |_
               |                     ----------------------------------     ↑      ↑               | | |
               |                    |      Services <--> Services-----|------> Database ----       | | |
               |                    |                                 |                     |      | | |
               |                    |           Observability         |                     |      | | |
               |                    |                                 |                     |      | | |
               |                    |                                 |                     ↓      | | |
       User -> | CDN -> Load     -> |       ↕     Service      ↕      |                   DATA     | | |
               |        balancer    |            Discovery            |                 WAREHOUSE  | | |
               |                    |                                 |                     ↑ |    | | |
               |                    |                                 |                     | |    | | |
               |                    |            Hardening            |                     | |    | | |
               |                    |                                 |                     | |    | | |
               |                    |      Services <--> Services-----|------> Database ----  |    | | |
               |                     ---------------------------------      ↓    |  ↓         |    | | |
               |                             |   Service Mesh   |          Cache | Backups    |    | | |
               |                             ↓                  ↓           ↓    ↓            ↓    | | |
               |                EVENT BUS =======================================================  | | |
               |prod                                                                               | | |
                -----------------------------------------------------------------------------------  | |
                 |stage                                                                              | |
                  -----------------------------------------------------------------------------------  |
                   |dev                                                                                |
                    -----------------------------------------------------------------------------------
  
  Developer ->  Version + Continuos   + Continuos + Monitoring + Auth,   + Infrastructure + FEATURE + CANARY
                Control   Integration   Delivery                 Secrets   as Code          TOGGLE    DEPLOYMENT
  

• Step 9:

  • Multiple data centers: -> global user base
  • Multiple accounts: larger employee base -> isolate teams/products
  • Advanced networking: connect data centers, accounts
  • Internal developer platform (Process): boost developer productivity; ensure all accounts are secure <- account baseline/factory

   ---->   DATA   (With all the infrastructure as in data center 1)
  |      CENTER 2
  |          |
  |          |     ---- VPC -------------------------------------------------------------------------- _
  |          |    |                                                          Cache  Backups           | |_
  |     ADVANCED  |                     ----------------------------------     ↑      ↑               | | |
  |     NETWORKING|                    |      Services <--> Services-----|------> Database ----       | | |
  |          |    |                    |                                 |                     |      | | |
  |          |    |                    |           Observability         |                     |      | | |
  |          |    |                    |                                 |                     |      | | |
  |          |    |                    |                                 |                     ↓      | | |
  User ->  DATA   | CDN -> Load     -> |       ↕     Service      ↕      |                   Data     | | |
          CENTER 1|        balancer    |            Discovery            |                 Warehouse  | | |
                  |                    |                                 |                     ↑ |    | | |
                  |                    |                                 |                     | |    | | |
                  |                    |            Hardening            |                     | |    | | |
                  |                    |                                 |                     | |    | | |
                  |                    |      Services <--> Services-----|------> Database ----  |    | | |
                  |                     ---------------------------------      ↓    |  ↓         |    | | |
                  |                             |   Service Mesh   |          Cache | Backups    |    | | |
                  |                             ↓                  ↓           ↓    ↓            ↓    | | |
                  |                Event Bus =======================================================  | | |
                  |prod                                                                               | | |
                   -----------------------------------------------------------------------------------  | |
                    |stage                                                                              | |
                     -----------------------------------------------------------------------------------  |
                      |dev                                                                                |
                       -----------------------------------------------------------------------------------
  
  Developer ->     Version + Continuos   + Continuos + Monitoring + Auth,   + Infrastructure + Feature + Canary     + Developer
                   Control   Integration   Delivery                 Secrets   as Code          Toggle    Deployment   Platform
  

Stage 3 applies for company with the toughest problems that deal with the more complexity: global deployments, thousands of developers, millions of users.

[!NOTE] The architecture in stage 3 is still a simplification to what the top 0.1% of the companies face.

Adopting DevOps Practices

Which DevOps practices to adopt?

[!IMPORTANT] KEY TAKEAWAY #1.1 You should adopt the architecture & software delivery processes that are appropriate for the stage of your company

[!CAUTION] Don’t immediately jump to the end and use the architecture & processes of the largest, most elite companies:

• You don’t have the same scale
• You don’t have the same problems to solve

Their solutions may not be a good fit for you.

How to adopt DevOps practices?

The key to a success of adopting DevOps (or any migration project) is to do it incrementally:

• Split up the work in a way that every step brings its own value, even if the later steps never happen

• Don’t fall into false incrementalism where all steps need to be completed before any step can bring value.

  There is a big changes that the projects gets:

  • modified
  • paused or even cancelled

[!IMPORTANT] KEY TAKEAWAY #1.2 Adopt DevOps incrementally, as a series of small steps, where each step is valuable by itself.

[!CAUTION] Avoid “big bang” migration (all or nothing).

[!TIP] Focus on solving small, concrete problem one at a time.

e.g.

• Migrate to cloud:
  • Instead of migrating all teams at the same time
  • Identifying one small, specific app/team -> Migrate just that app/team
• Adopt DevOps:
  • Instead of applying all processes
  • Identifying one small problem, e.g. outages during deployment -> Automate the deployment steps

Even if the larger migration doesn’t work, at least

• one team is more successful
• one process works better

An Introduction to Deploying Apps

Run an App Locally

Example: Run the Sample App Locally

• A Node.js “Hello, World” - a web server

  // app.js
  const { createServer } = require("node:http");
  
  const server = createServer((req, res) => {
    res.writeHead(200, { "Content-Type": "text/plain" });
    res.end("Hello, World!");
  });
  
  const port = 8080;
  server.listen(port, () => {
    console.log(`Server listening on port ${port}`);
  });
  

• Install Node.js (if you haven’t installed)

• Run the app

  node app.js
  

• Open link http://localhost:8080 in browser

[!NOTE] By default, when you run a web server on your computer:

• It’s only available on localhost.
• In other words, the web server can only be accessed from your computer.

[!TIP] The localhost is a hostname - configured on every computer - points back to the loopback network interface (which is typically 127.0.0.1)

The problem with expose an app run on your personal computer

• Security

  Your personal computer (PC) is not hardened:

  • There’s a lot of app installed. The more apps running, the more likely an app has an CVE that could be exploited by attacker.
  • There is your personal data (documents, photos, videos, passwords…)

• Availability

  Your PC might:

  • be accidentally shutdown.
  • not be designed to run 24/7.

• Performance

  If you’re using your PC,

  • that might take away system resources from your app,
    • which might cause performance issues for your users.

• Collaboration

  If your app has a bug, or needs to be updated:

  • someone (coworkers, collaborators…) needs to access to your PC,
  • should you give them access to your personal data? No!

[!IMPORTANT] KEY TAKEAWAY #1.3 You should never expose apps running on a personal computer to the outside world.

When to expose an app that runs on your PC

You can deploy an app locally, and expose that app only when:

• You’re exposing it to a trusted 3rd-party, (e.g. a coworker)…
• … to get feedback

[!TIP] You can use tunnelling tools, e.g. localtunnel, ngrok, btunnel, localhost.run

• to get a temporary URL of your app

Then give someone you trust that URL to access your app.

Why many businesses still expose their critical apps from a PC

Maybe because of:

1. The company has resource constrained, e.g. a tiny startup
2. The person running the app doesn’t know any better
3. The software delivery process is so slow, cumbersome; sneaking the app in a personal computer is the quickly way to get it running.

The solutions:

• For 1, it’s the cloud.

• For 2 & 3, it’s reading this book:

  • You know better (2)
  • You know how to create a software delivery process that allow your team to quickly & easily run their apps the right way: on a server. (3)

Deploying an App on a Server

There are 2 ways to get access to servers:

1. On prem: Buying & setting up your own servers, e.g. Dell R7625 Rack Server[^1]¹
2. In the cloud: You rent servers from others, e.g. AWS EC2

Deploying On Prem Versus in the Cloud

on-prem : Abbreviated for on-premises software : Software you run: : - on your own servers : - in a physical location you own: e.g. your garage/office/data center

in the cloud : Software you run: : - on servers in a cloud computing platform, e.g. AWS, Azure : In other words, you rent servers from a cloud platform via a software interface, and use these rented servers to run your software.

When to Go with the Cloud

Using the cloud should be the default chose because of the following advantages:

[!IMPORTANT] KEY TAKEAWAY #1.4 Using the cloud should be your default choice for most new deployments these days.

When to Go with On-Prem

When to Go with Hybrid

hybrid : a mixture of cloud & on-prem

The most uses cases of hybrid cloud:

Two types of cloud: PaaS and IaaS

There are 2 types of cloud:

• IaaS - Infrastructure as a Service

  IaaS gives you access directly to the low-level primitives computing resources, e.g. servers, so

  • you can create your own software delivery process.

  e.g. Amazon AWS, Microsoft Azure, Google Cloud

• PaaS - Platform as a Service

  PaaS gives you a full, opinionated software delivery process.

  e.g. Heroku, Netlify, Fly.io, Vercel, Firebase, Render, Railway, Platform.sh

[!TIP] One of the first service from AWS (the first cloud that came out in 2006) is Elastic Compute Cloud (EC2), which allow you to rent servers from AWS.

This is the first Infrastructure as a Service (IaaS) in the market.

EC2 gives you access directly to the (low-level) primitive computing resources - the server.

[!TIP] A year later, in 2007, Heroku came out with one of the first Platform as a Service (PaaS) offerings, which focus on high-level primitive.

In additional to the infrastructure, e.g. server, Heroku also provides a full, opinionated software delivery process:

• application packaging
• deployment pipelines
• database management
• …

Deploying An App Using PaaS

[!NOTE] The examples in this chapter use Fly.io as the PaaS

[!TIP] Why Fly.io?

• Provides $5 free credits -> the example can be running without cost anything.
• Support automatically packaging code for deployment via Buildpacks -> code can be deployed without any build system, Docker image…
• Has a CLI tool flyctl -> deploy code straight from your computer.

Example: Deploying an app using Fly.io

• Step 1: Install flyctl

• Step 2: Sign up & sign in

  fly auth signup
  
  fly auth login
  

• Step 3: Configure the build

  # examples/ch1/sample-app/fly.toml
  [build]
    builder = "paketobuildpacks/builder:base"
    buildpacks = ["gcr.io/paketo-buildpacks/nodejs"]
  
  [http_service]
    internal_port = 8080
    force_https = true
    auto_stop_machines = true
    auto_start_machines = true
    min_machines_running = 0
  

[!TIP] For real-world applications, flyclt can recognize many popular app frameworks automatically and you wouldn’t this config file.

• Step 4: Launch the app

  fly launch --generate-name --copy-config --yes
  

Get your hands dirty with Fly.io

• Check the app status with fly status
• See the app logs with fly logs, or https://fly-metrics.net
• Scale the numbers of servers up & down with fly scale
• (Make a change then) Deploy a new version of the app with fly deploy

[!NOTE] When working with the cloud, make a habit of undeploy any things you don’t need anymore.

• For fly.io, it’s by using fly apps destroy <NAME>

How PaaS stacks up

A Paas provides:

• not just the low-level primitives, e.g. the servers “🖥️”
• but also the high-level primitives - powerful functionalyity out-of-the-box, such as:
  • ⬆️⬇️ Scaling servers
  • 🌐 Domain names
  • 🔒 TLS certificates & termination
  • 📊 Monitoring
  • 🤖 Automated deployment

These high-level primitives is what make PaaS magic - it just works.

In a matter of minutes, a good PaaS take care of so many software delivery concern for you.

[!WARNING] The magic of PaaS is also the greatest weakness of PaaS.

• Everything is happenning behind the scenes. If something doesn’t work, it can be hard to debug/fix it.
• There is a lot of limitation:
  • What you can deploy
  • What types of apps you can run
  • What sort of access you can have to the underlying hardware
  • What sort of hardware is available
  • …

[!NOTE] Many projects start on PaaS, then

• migrate to IaaS if they grow big enough and require more control.

Deploying an App Using IaaS

• There are 3 types of IaaS: VPS, CDN, cloud providers:

  IaaS type	Description	Example
  VPS Providers	- Provide access to the Virtual Private Servers (VPSs) as cheap as possible	Hetzner, DigitaOcean, Vultr…
  	- aka VPS Hosting Providers, might offer other features, e.g. networking, storage…
  CDN Providers	- Provide access to Content Delivery Network - CDN servers2	CloudFlare, Akamai, Fastly
  	- Might also offer: DDoS protection…
  Cloud Providers	- Very large companies provides general-purpose cloud solutions for everything: VPS, CDN, serverless, edge computing, data/file storages…	Amazon Web Services (AWS), Microsoft Azure, Google Cloud Platform (GCP)
  		Alibaba Cloud, IBM Cloud

• In general, VPS and CDN providers are

  • specialists in their respective area,

    • so they will beat a general cloud in term of features & pricing in those areas.

      e.g. A VPS from Hetzner is usually much faster & cheaper than from AWS.

  • if you only need the features in their area, better off going with them.

• If you are

  • building the infrastructure for the entire company,
    • especially one that is in later stages of its DevOps evolution,
  • your architecture usually needs many types of infrastructure
  • the general-purpose cloud providers will typical a better fit.

Example: Deploying an app using AWS

• Step 1: Sign up for AWS

  After you signed up,

  • you initially sign in as the root user, which has full permissions to do anything in the AWS account.
  • you can create IAM user - which is more-limited user account within your AWS account.

  [!WARNING] Never use your AWS root user for daily tasks.

• Step 2: Create an IAM user.

  Use the Identity and Access Management (IAM) service to:

  • create an IAM user
  • manage IAM users
    • add permissions to that IAM user via IAM policy, which can be attached
      • directly to the IAM user
      • or via IAM group

  After you create an IAM user, AWS will show you the security credentials for that users: 1. Sign-in URL, 2. Username, 3. Console password.

  [!TIP] The password is called console password because it’s used for signing in to the AWS Management Console - the web application that manage your AWS account.

  [!TIP] Keep both the root user’s password and IAM user’s password in a password manager, e.g. 1Password, BitWarden

• Step 3: Login as the IAM user.

  Go the the sign-in URL and sign in with the IAM user credential.

  [!TIP] The sign-in URL is unique for each AWS account.

  In other words, each AWS account has it own authenticated & authorization system.

• Step 4: Deploy an EC2 instance.

  Use the AWS Elastic Compute Computing (EC2) Service to deploy an EC2 instance:

  • Click Launch instance

  • Fill in name of the instance

  • Choose the Application & OS Images (Amazon Machine Image - AMI)

    • Use the default - Amazon Linux

  • Choose the Instance type, which specifies the type of server: CPU, memory, hard drive…

    • Use the default - t2.micro or t3.micro (Small instance with 1 CPU, 1GB of memory that including in AWS free tier)

  • Choose Proceed without a key-pair because you’re not going to use SSH for this example

  • Configure Network settings:

    • Use the default settings:

      • Network: Default VPC
      • Subnet: No preference - Default subnet

    • Firewall (Security group): Choose Create security group with the rules:

      • Disable Allow SSH traffic from
      • Enable Allow HTTP traffic from the internet <- This allows inbound TCP traffic on port 80 so the example app can receive requests and response with “Hello, World!”

      [!NOTE] By default, EC2 instances have firewalls, called security groups that don’t allow any network traffic in or out.

  • Configure User data:

    [!NOTE] User data is a script that will be executed by the EC2 instance the very first time it boots up

    Fill in a Bash script that:

    • Install node.js
    • Get the code for example server (a simple Node server in a file)
    • Run the server (and ignore hangup signals by using nohup)

[!CAUTION] Watch out for snakes: These examples have several problems

Problem	What the example app does	What you should do instead
Root user	The app is running from user data, which runs as root user.	Run apps using a separate OS user with limited permissions.
Port 80	The app is listening on port 80, which required root user permissions.	Run apps on ephemeral ports - port greater than 1024.
User data’s limit	The app put all its code & dependencies in user data, which is limited to 16KB.	User configuration management tool or server templating tools.
No process supervision	The app is started by user data script, which only run on the first boot.	Use process supervisors to restart that app if it crashes, or after server reboots.
Node.js specifics	The app is run in development mode, which only a have minimum of logging and doesn’t have optimized performance	Run Node.js in production mode3.

Get your hands dirty with AWS

• Restart your EC2 instance: Does the app still work? Why (not)?
• Create a custom security group opens up port 8080.
• Find logs/metrics about the EC2 instance, compare with monitoring from fly.io.

How IaaS stacks up

Comparing PaaS and IaaS

When to Go with PaaS

[!TIP] Your customers don’t care what kind of CI/CD pipeline you have:

• Whether you’re running a fancy Kubernetes cluster
• Whether you’re on the newest NoSQL database
• …

All they matters is you can create a product that meets your customers’ needs.

[!IMPORTANT] KEY TAKEAWAY #1.5 You should spend as little time on software delivery as you possibly can, while still meeting your company’s requirements.

• If you can find a PaaS meets your requirements, you should:
  • use it & stick with it as long as you can.
  • avoid re-creating all those software delivery pieces until you absolutely have to.

The following use cases is a good fit for PaaS:

• 🛝 Side projects

  Focus all your time on the side project itself, instead of wasting any time to the software delivery process.

• 🚀 Startup & small companies

  A startup lives or dies based on its product - something the market wants.

  • Invest all the time/resources to the product.
  • Only when you’re facing the scaling problem, which means you’ve found your product/market, start thinking of moving of PaaS.

• 🧪 New & experimental projects (at established companies)

  Established companies might have invested in IaaS but still have a slow & inefficient software delivery process:

  • by using PaaS, you can quickly launch something & iterate on it.

When to Go with IaaS

Only move to IaaS when a PaaS can no longer meet your requirements, which means you’re facing the following problems:

• 🪨 Load & scaling:

  When you are dealing with a huge a mount traffic:

  • In other words, you’re facing the scaling problem (and have found your product)
  • PaaS might no longer meet your requirements:
    • The pricing of PaaS might become prohibitively.
    • The supported architectures by PaaS is limited

  a migrate to IaaS is require to handling that load & scaling.

• 🍕 Company size

  For companies with dozens of teams with hundreds or thousands of developers, PaaS offers for governance & access controls might be not enough.

  e.g.

  • Allow some teams to make changes, but not the others

• 🅱️ Availability

  Your business might have a higher level than what PaaS offers for uptime guarantees (SLOs, SLAs)

  PaaS offerings are limited in term of visibility & connectivity options, e.g.

  • Many PaaS don’t let you SSH to the server, when there is an outage/bug you can’t know what really happening.

  [!NOTE] Heroku - the leading PaaS - only supports SSH into a running server after a decade.

• 🛡️ Security & compliance

  If your business needs to meet some strict security, compliance requirements - e.g. PCI, GCPR, HIPPA - IaaS might be the only option.

[!IMPORTANT] KEY TAKEAWAY #1.6 Go with PaaS whenever you can; go with IaaS when you have to.

Conclusion

• Adopt the architecture & software delivery processes that are appropriate for your stage of company
• Adopt DevOps incrementally, as a series of small steps, where each step is valuable by itself
• You should never expose apps running on a PC to the outside world
• Using the cloud should be your default choice for most new deployments these days
• You should spend as little time on software delivery as you possibly can, while still meeting your company’s requirements
• Go with PaaS whenever you can; go with IaaS when you have to

Chapter 2: How to Manage Your Infrastructure as Code

ClickOps and IaC

ClickOps

ClickOps : clicking through an web UI of a cloud provider’s website to configure computing infrastructure

The problems of ClickOps:

• Deployments are slow & tedious → You can’t deploy more often
• Deployments are error-prone → Bugs, outages…
• Only one person knows how to deploy → If that person is overloaded, everything takes ages; there is also bus factor

Infrastructure as Code

Infrastructure as Code (IaC) : You write & execute code to define, deploy, update, destroy your infrastructure : This marks a shift in mindset in which : - all aspects of operations are treated as software : - even those represent hardware, e.g. setting up a server

• With modern DevOps, you can manage almost everything as code:

  Task	How to manage as code	Example	Chapter
  Provision servers	Provisioning tools	Use OpenTofu to deploy a server	This chapter (2)
  Configure servers	Configuration management & templating tools	Use Packer to create an image of a server	This chapter (2)
  Configure apps	Configuration files & services	Read configuration from a JSON file during boot
  Configure networking	Provisioning tools, service discovery	Use Kubernetes’s service discovery
  Build apps	Build systems, continuous integration	Build your app with npm
  Test apps	Automated tests, continuous integration	Write automated tests using Jest
  Deploy apps	Automated deployment, continuous delivery	Do arolling deployment with Kubernetes	Chapter 3
  Scale apps	Auto scaling	Set upauto scaling policies in AWS	Chapter 3
  Recover from outages	Auto healing	Set upliveness probes in Kubernetes	Chapter 3
  Manage databases	Schema migrations	Use Flyway to update your database schema
  Test for compliance	Automated tests, policy as code	Check compliance using Open Policy Agent (OPA)

• For infrastructure, there are 4 type of IaC tools:

  IaC tool	Example
  Ad-hoc scripts	Use a Bash script to deploy a server.
  Configuration management tools	Use Ansible to deploy a server.
  Server templating tools	Use Packer to create an image of a server.
  Provision tools	Use OpenTofu to deploy a server.

The Benefits of IaC

When your infrastructure is defined as code:

• the entire deployment process can be automated
• you can apply software engineering practices (to your software delivery processes)

which bring a lot of benefits:

Ad Hoc Scripts

What is Ad Hoc Script

ad hoc (ad-hoc) : (adj) arranged or happening when necessary and not planned in advance

ad hoc script : code written in a scripting language - e.g. Bash/Ruby/Python - to automate a task you were doing manually

Example: Deploy an EC2 Instance Using a Bash Script

In this example, you will automate all the manual steps, in example in chap 1 that deploy an app using AWS.

• Migrate the user data

  cd examples
  mkdir -p ch2/bash
  

  cp examples
  cp ch1/ec2-user-data-script/user-data.sh ch2/bash/
  

• Write the Bash script to deploy an app using AWS

  • Create security group
  • Create rule for that security group
  • Run the instance
  • Get the public ip of the instance
  • Print: instance id, security group id, public ip

  # examples/ch2/bash/deploy-ec2-instance.sh
  # TODO
  

[!CAUTION] Watch out for snakes: these are simplified examples for learning, not for production

Get your hands dirty: Ad hoc scripts

1. What happens if you run the Bash script a second time?

   1. Do you get a error?
   2. If so, why?

2. How would you have to tweak the script if you wanted to run multiple EC2 instances?

1.i. If the script is run the second time, there will be an error. 1.i. Because in a VPC - the default VPC in this case - the security group’s name need to be unique.

2. To have multiple EC2, you can duplicate the whole script an change the name of the security-group.

[!WARNING] When you’re done experimenting with the script, you should manually un-deployed the EC2 instance by using the EC2 Console

How Ad Hoc Scripts Stack Up

[!IMPORTANT] Key takeaway #2.1: Ad hoc scripts are

• great for small, one-off tasks,
  • but not for managing all your infrastructure as code.

Configuration Management Tools

What is Configuration Management Tools

Configuration Management Tools : e.g. Chef, Puppet, Ansible : Appear before cloud computing → Designed with the assumption that: : - someone else had set up the hardware, e.g. Ops team racked the servers in data center. : - primary purpose is to handle the software - configure the servers: OS, dependencies, your app (deploy, update).

[!NOTE] The configuration management tools can also deploy & manage servers or other infrastructure.

How Configuration Management Tools work

• Most configuration tools makes changes directly on a set of server you specify, which is called mutable infrastructure paradigm:

  • The same long-running servers will be mutate over & over again, over many years.

• To be able to make changes on these servers, you need 2 things: something to drive the changes & a way to connect to the server.

  	Chef, Puppet	Ansible
  Something to drive the changes	You run amaster server(s)	You use an CLI
  	➕ Can have areconciliation loop: check & match the desired configuration	➕ Can run anywhere (dev PC, build server…)
  A way to connect to the server	Viaagent software that installed on each server	Via SSH
  	➖ Need to install the agent	➖ Need to open extra port

  [!WARNING] Chicken-and-egg 🐥🥚 problem You have a tool that configure your servers:

  • before you can use that tool
    • you need to configure your servers.

Example: Deploy an EC2 Instance Using Ansible

[!NOTE] This example use Ansible to deploy an EC2 instance so you can have a server to use the configuration management tool - Ansible.

[!WARNING] Although configuration tools can also deploy & manage servers:

• they’re not originally designed to that.

For this example, spinning up a single server for learning & testing, Ansible is good enough.

[!TIP] Before start this example, you can read the docs about the basic concepts in Ansible.

See:

• https://docs.ansible.com/ansible/latest/getting_started/index.html
• https://docs.ansible.com/ansible/latest/getting_started/basic_concepts.html

To deploy an EC2 instance using Ansible, you need to:

• Define an Ansible playbook³

  • in Ansible’s domain specific language (DSL), which is based on YAML.
  • to tell Ansible to do what you want:
    • create a security group
    • create an EC2 key-pair (& save it)
    • create the EC2 instance (& tag it)

  # examples/ch2/ansible/create_ec2_instance_playbook.yml
  # TODO
  

Example: Configure a Server Using Ansible

1. To let Ansible know which servers it needs configure, you provide an inventory⁴ that:

• Specify a list of static IP addresses of the servers (in group).

  e.g.

  webservers: # A group of servers named webservers
    hosts:
      10.16.10.1:
  dbservers: # A group of servers named dbservers
    hosts:
      10.16.20.1:
      10.16.20.2:
  

  • Now, you can use Ansible playbook to target the servers in those 2 groups: webservers, dbservers

• Use an inventory plugin to dynamically discover your servers with IP addresses that change frequently.

  • e.g.

    • Use the aws_ec2 inventory plugin to discovered EC2 instance on AWS

      # examples/ch2/ansible/inventory.aws_ec2.yml
      plugin: amazon.aws.aws_ec2
      regions:
        - us-east-2
      keyed_groups:
        - key: tags.Ansible # 1️⃣
      leading_separator: "" # 2️⃣
      

    • 1️⃣: Ansible will create groups bases on the value of the tag Ansible

    • 2️⃣: By default, Ansible adds a leading underscore to the group names. This disables it so the group name matches the tag value.

  • For each group (of servers) in the inventory, you can specify group variables⁵ to configure how to connect to the servers in that group.

    # examples/ch2/ansible/group_vars/ch2_instances.yml
    ansible_user: ec2-user # The user Ansible ‘logs in’ as.
    ansible_ssh_private_key_file: ansible-ch2.key
    ansible_host_key_checking: false # Turn off host key checking so Ansible don't prompt you
    

2. To let Ansible know what to do (with the servers), you provides a playbook (that specifies the roles⁶ of these server).

• The playbook

  # examples/ch2/ansible/configure_sample_app_playbook.yml
  - name: Configure the EC2 instance to run a sample app
    hosts: ch2_instances # Target the servers in group ch2_instances - the one created in previous example, grouped by the inventory plugin
    gather_facts: true
    become: true
    roles:
      - sample-app # Configure the server using an Ansible role called sample-app
  

• The role:

  • Tasks

    # ch2/ansible/roles/sample-app/tasks/main.yml
    - name: Add Node packages to yum
      shell: curl -fsSL https://rpm.nodesource.com/setup_21.x | bash - # 1️⃣
    
    - name: Install Node.js
      yum:
        name: nodejs # 2️⃣
    
    - name: Copy sample app
      copy: #          3️⃣
        src: app.js #  Relative path to the role's files directory
        dest: app.js # Relative path on the server
    
    - name: Start sample app
      shell: nohup node app.js &
    

    • 1️⃣: Use the shell module to install yum
    • 2️⃣: Use the yum module to install nodejs
    • 3️⃣: Use the copy module to copy app.js to the server.

  • Files

    Copy app.js from chapter 1 to examples/roles/sample-app/files/app.js

3. The final structure of the example

.
├── configure_sample_app_playbook.yml
├── group_vars
│   └── ch2_instances.yml
├── inventory.aws_ec2.yml
└── roles
    └── sample-app
        ├── files
        │   └── app.js
        └── tasks
            └── main.yml

4. Run the playbook

[!TIP] Don’t forget to authenticate to AWS on the command line.

ansible-playbook -v -i inventory.aws_ec2.yml configure_sample_app_playbook.yml

PLAY RECAP
xxx.us-east-2.compute.amazonaws.com : ok=5    changed=4    failed=0

Get your hands dirty with Ansible

1. What happens if you run the Ansible playbook a second time? How does this compare to running the Bash script a second time?
2. How would you have to tweak the playbook if you wanted to run multiple EC2 instances?
3. Figure out how to use the SSH key created by Ansible (ansible.key) to manually SSH to your EC2 instance and make changes locally.

[!WARNING] When you’re done experimenting with Ansible, you should manually un-deployed the EC2 instance by using the EC2 Console

How Configuration Management Tools Stack Up

Drawbacks of configuration management tools

• Setup cost
• Configuration drift due to mutable infrastructure paradigm: each long-running server can be a little different from the others.

Immutable infrastructure paradigm

With immutable infrastructure paradigm:

• Instead of long-running physical servers,
  • you use short-lived virtual servers (that will be replaced every time you do an update).
• Once you’ve deployed a server, you’ve never make changes to it again.
  • If you need to update something, even it’s just a new version of your application
    • you deploy a new server.

[!TIP] Cattle vs pets

[!NOTE] Immutable infrastructure paradigm is inspired by:

• Function programming:
  • Variables are immutable
    • After you set a variable to a value, you can’t change that variable again.
    • If you need to update something, you create a new variable.
  • It’s a lot easier to reason about your code.

[!IMPORTANT] Key takeaway #2.2

Configuration management tools are

• great for managing the configuration of servers,
  • but not for deploying the servers themselves, or other infrastructure.

Server Templating Tools

What is Server Templating Tools

Server Templating Tools : e.g. Docker, Packer, Vagrant : instead of: : 1. launching servers : 2. configure them (by running the same code on each) : you: : 1. create an image of a server that captures a fully self-contained “snapshot” of the operating system (OS), the software, the files, and all other relevant details. : 2. use some other IaC to install that image on all of your servers.

Two types of image tools - Virtual machine and container

Virtual machine

virtual machine (VM) : a VM emulates an entire computer system, including the hardware (and of course the software)

VM image : the blueprint for a VM : defined with tools: Packer, Vagrant

hypervisor : aka virtualizer : a type of computer software/firmware/hardware that creates & runs virtual machines.

• You run a hypervisor⁷ with the VM image to create a VM that virtualize/emulate

  • the underlying hardware: CPU, memory, hard driver, networking…
  • the software: OS, dependencies, apps…

• Pros and cons of VM:

  	VM
  Pros	- Each VM is fully isolated from the host machine & other VM.	<- Can run any 3rd-party code without worry of malicious actions
  	- All VMs from the same VM image will run exactly the same way in all environments.	e.g. Your PC, a QA server, a production server.
  Cons	- Overhead of CPU/memory usage.	<- For each VM, the hypervisor needs to virtual all hardware & running a guest OS …
  	- Overhead of startup time.	<- … that whole OS needs to start.

Container

container : a container emulates the user space⁸ of an OS

container image : the blueprint for a container

container engine : a Container Engine takes a Container Image : - (simulates an user space with memory, mount points & networking) : - turns it into a Container (aka running processes) : e.g. Docker, cri-o, Podman

Example: Create a VM Image Using Packer

In this example, you will use Packer to create a VM image for AWS (called an Amazon Machine Image - AMI)

• Create a Packer template

  # examples/ch2/packer/sample-app.pkr.hcl
  packer {
    required_plugins { #                                                  0️⃣
      amazon = {
        version = ">= 1.3.1"
        source  = "github.com/hashicorp/amazon"
      }
    }
  }
  
  source "amazon-ebs" "amazon_linux" { #                                  1️⃣
    ami_name        = "sample-app-packer-${uuidv4()}"
    ami_description = "Amazon Linux 2023 AMI with a Node.js sample app."
    instance_type   = "t2.micro"
    region          = "us-east-2"
    source_ami      = "ami-0900fe555666598a2"
    ssh_username    = "ec2-user"
  }
  
  build { #                                                               2️⃣
    sources = ["source.amazon-ebs.amazon_linux"]
  
    provisioner "file" { #                                                3️⃣
      source      = "app.js"
      destination = "/home/ec2-user/app.js"
    }
  
    provisioner "shell" { #                                               4️⃣
      inline = [
        "curl -fsSL https://rpm.nodesource.com/setup_21.x | sudo bash -",
        "sudo yum install -y nodejs"
      ]
      pause_before = "30s"
    }
  }
  

  • 0️⃣ - Plugin: Use the Amazon plugin⁹ to build Amazon Machine Image (AMI)
  • 1️⃣ - Builder: Use the amazon-ebs builder to create EBS-backed AMIs by
    • (launching a source AMI)
    • (re-packaging it into a new AMI after provisioning¹⁰)
  • 2️⃣ - Build steps:
    • After provision the EC2 instance, Packer connects to the server and runs the build steps in the order specified in the Packer template.
    • (When all the builds steps have finished, Packer will take a snapshot of the servers and use it to create an AMI)
  • 3️⃣ - File provisioner: Copy the files to the server.
  • 4️⃣ - Shell provisioner: Execute shell commands on the server.

  [!NOTE] The Packer template is nearly identical to the Bash script & Ansible playbook,

  • except it doesn’t actually run the app.

• Install Packer

• Install Packer plugins (used in the Packer template)

  packer init sample-app.pkr.hcl
  

  [!NOTE] Packer can create images for many cloud providers, e.g. AWS, Azure, GCP. The code for each providers is

  • not in the Packer binary itself
  • but in a separate plugin (that the packer init command can install)

• Build image from Packer template

  packer build sample-app.pkr.hcl
  

  Output

  ==> Builds finished. The artifacts of successful builds are:
    --> amazon-ebs.amazon_linux: AMIs were created:
    us-east-2: ami-XXXXXXXXXXXXXXXXX
  

  • The ami-XXX value is the ID of the AMI that was created from the Packer template.

  [!NOTE] The result of running Packer is not a server running your app, but the image of the server.

  • This image will be used by another IaC tolls to launch one or more servers (running the image)
  • The app will be run when the image is deployed (or the server is launched).

Get your hands dirty with Packer

1. What happens if you run packer build on this template a second time? Why?

2. Figure out how to update the Packer template so it builds images that

   • not only can run on AWS,
   • but also can run on other clouds (e.g., Azure or GCP)
     • or on your own computer (e.g., VirtualBox or Docker).

How Server Templating Tools Stack Up

[!WARNING] Server templating tools cannot be used in isolated (because it only supports create).

• If you use a server templating tool, you need another tool to support all CRUD operations, e.g. a provisioning tool

[!NOTE] All server templating tools will create images but for slightly different purposes:

• Packer: create VM images run on production servers, e.g. AMI
• Vagrant: create VM images run on development computers, e.g. VirtualBox image
• Docker: create container images of individual applications, which can be run any where as long as that computer has installed an container engine.

[!IMPORTANT] Key takeaway #2.3 Server templating tools are

• great for managing the configuration of servers with immutable infrastructure practices.
  • (but needs to be used with another provisioning tools)

Provisioning Tools

What is Provisioning Tools

provisioning tool : e.g. OpenTofu/Terraform, CloudFormation, OpenStack Heat, Pulumi… : a provisioning tool is responsible for : - deploying : - managing (all CRUD operations) : the servers & other infrastructure in the clouds: : - (servers), databases, caches, load balances, queues, monitoring : - subnet configurations, firewall settings, routing rules, TLS certificates : - …

[!NOTE] What are the different between ad-hoc script, configuration management tools, server templating tools & provisioning tools?

• Configuration management tools: manage configurations of servers
• Server templating tools: manage configurations of servers with immutable infrastructure practices
• Provisioning tools: deploy & manage the servers (& other infrastructure)

How Provisioning Tools work

Under the hood, provisioning tools work by

• translating the code you write
  • into API calls to the cloud providers you’re using

e.g. If you write OpenTofu/Terraform code to create a server in AWS, when you run OpenTofu, it will:

• Parse your code
• (Based on the the configuration you specified,) make a number of APIs calls to AWS
  • to create an EC2 instance

[!NOTE] By making APIs to cloud providers, provisioning tools bring in many advantages:

• You don’t need to setup master servers.
• You don’t need to setup connection to the servers ← Take advantages of the authentication mechanism of cloud providers.

Example: Deploy an EC2 Instance Using OpenTofu

[!TIP] Terraform vs OpenTofu

• Terraform is a popular provisioning tool that HashiCorp open sourced in 2014 under Mozilla Public Licenses (MPL) 2.0.
  • In 2024, HashiCorp switched Terraform to non-open source Business Source License (BSL).
• As a result, the community fork Terraform under the named OpenTofu, which remains open source under the MPL 2.0 license.

To deploy an EC2 Instance using OpenTofu, you

• write an OpenTofu module

  • in HCL¹²,
  • in configuration files with a .tf extension (instead of .pkr.hcl for Packer template)

  [!NOTE] An OpenTofu module is a folder with all .tf files in that folder:

  • No matter are the name of these .tf files.
  • But there are some conventions, e.g.
    • main.tf: Main resources
    • variables.tf: Input variables
    • outputs.tf: Output variables

• use that OpenTofu module (run OpenTofu code) to deploy the EC2 instance.

For this example, the OpenTofu module for an EC2 instance looks like this:

1. main.tf: Main resources

   # examples/ch2/tofu/ec2-instance/main.tf
   provider "aws" {                                               # 1️⃣
     region = "us-east-2"
   }
   
   resource "aws_security_group" "sample_app" {                   # 2️⃣
     name        = "sample-app-tofu"
     description = "Allow HTTP traffic into the sample app"
   }
   
   resource "aws_security_group_rule" "allow_http_inbound" {      # 3️⃣
     type              = "ingress"
     protocol          = "tcp"
     from_port         = 8080
     to_port           = 8080
     security_group_id = aws_security_group.sample_app.id
     cidr_blocks       = ["0.0.0.0/0"]
   }
   
   resource "aws_instance" "sample_app" {                         # 4️⃣
     ami                    = var.ami_id                          # 4️⃣1️⃣
     instance_type          = "t2.micro"
     vpc_security_group_ids = [aws_security_group.sample_app.id]
     user_data              = file("${path.module}/user-data.sh") # 4️⃣2️⃣
   
     tags = {
       Name = "sample-app-tofu"
     }
   }
   

   What the OpenTofu code do?

   • 1️⃣ - Use AWS provider: to work with AWS cloud provider.

     [!NOTE] OpenTofu can works with many providers, e.g. AWS, Azure, GCP…

     • An OpenTofu provider is like a Packer plugin.

     [!TIP] AWS has data centers all over the world, grouped into regions.

     • An AWS region is a separate geographic area, e.g. us-east-1 (Virginia), us-east-2 (Ohio), eu-west-1 (Ireland), ap-southeast-1 (Singapore)
       • Within each region, there are multiple isolated data centers, called Availability Zones (AZs)

   • 2️⃣ - Create a security group: to control the network traffic go in & out the EC2 instance

     [!NOTE] For each type of provider, there are

     • several kinds of resources that you can create
       • e.g. servers, databases, load balancers, firewall settings…

     ---

     The syntax for creating a resource (of a provider) in OpenTofu is as follows:

     • resource "<PROVIDER>_<TYPE>" "<NAME>" {
         [CONFIG ...]
       }
       

       with:

       • PROVIDER: name of the provider, e.g. aws
       • TYPE: type of the resource (of that provider) to create, e.g. instance (an AWS EC2 instance)
       • NAME: an identifier you can use in OpenTofu code to refer to this resource, e.g. my_instance
       • CONFIG: one or more arguments that specific to that resource.

   • 3️⃣ - Create a rule for the security group: to allow inbound HTTP request on port 8080.

   • 4️⃣ - Create an EC2 instance: that uses the previous security group, and have a Name tag of sample-app-tofu.

      - 4️⃣1️⃣ - **Set the AMI**: to `var.ami_id`, which is a reference to an `input variable` named `ami_id` in `variables.tf`.
      - 4️⃣2️⃣ - **Set the user data**: to a file named `user-data.sh`, which is in the OpenTofu module's directory, next to other `.tf` files.
     

2. variables.tf: Input variables

   # examples/ch2/tofu/ec2-instance/variables.tf
   variable "ami_id" {
     description = "The ID of the AMI to run."
     type        = string
   }
   

   [!NOTE] The input variables allow an OpenTofu module

   • to be customized when that module is used to provision resources.

   Example explain

   • The input variable ami_id allow you to pass in the ID of an AMI that will be used to run the EC2 instance.
     • You will pass in ID of the AMI you build Packer template in previous section.

3. outputs.tf: Output variables

   # examples/ch2/tofu/ec2-instance/outputs.tf
   output "instance_id" {
     description = "The ID of the EC2 instance"
     value       = aws_instance.sample_app.id
   }
   
   output "security_group_id" {
     description = "The ID of the security group"
     value       = aws_security_group.sample_app.id
   }
   
   output "public_ip" {
     description = "The public IP of the EC2 instance"
     value       = aws_instance.sample_app.public_ip
   }
   

   [!NOTE] The output variables can be used to log & share values betweens OpenTofu modules.

4. (Not about OpenTofu) The application & the user data

   • The application: is already included in the AMI (built from the Packer template in previous section).

   • The EC2 instance user data (to start the app)

     # examples/ch2/tofu/ec2-instance/user-data.sh
     #!/usr/bin/env bash
     nohup node /home/ec2-user/app.js &
     

After writing the OpenTofu module code, you need to run that module code to deploy the EC2 instance:

1. Install OpenTofu

2. Install any providers used in OpenTofu code

   tofu init
   

3. Apply the OpenTofu code to deploy the EC2 instance

   • Run the apply command

     tofu apply
     

   • The tofu apply command will prompt you for the ami_id value and you paste in the value via the CLI

     var.ami_id
       The ID of the AMI to run.
     
       Enter a value:
     

     Alternative to provide the values via the CLI prompt, you can do it via -var flag, environment variables, or variable definitions file.

     • -var flag:

       tofu apply -var ami_id=<YOUR_AMI_ID>
       

     • Environment variable TF_VAR_<var_name>

       export TF_VAR_ami_id=<YOUR_AMI_ID>
       tofu apply
       

     • Variable definition file (a file named terraform.tfvars)

       • Define terraform.tfvars

         # ch2/tofu/ec2-instance/terraform.tfvars
         ami_id = "<YOUR_AMI_ID>"
         

       • Run tofu apply and OpenTofu will automatically find the ami_id value.

   • The tofu apply command will then

     • show you the execution plan (plan for short)…

       OpenTofu will perform the following actions:
       

       …Details of the actions…

         # aws_instance.sample_app will be created
         + resource "aws_instance" "sample_app" {
             + ami                                  = "ami-0ee5157dd67ca79fc"
             + instance_type                        = "t2.micro"
             ... (truncated) ...
           }
       
         # aws_security_group.sample_app will be created
         + resource "aws_security_group" "sample_app" {
             + description            = "Allow HTTP traffic into the sample app"
             + name                   = "sample-app-tofu"
             ... (truncated) ...
           }
       
         # aws_security_group_rule.allow_http_inbound will be created
         + resource "aws_security_group_rule" "allow_http_inbound" {
             + from_port                = 8080
             + protocol                 = "tcp"
             + to_port                  = 8080
             + type                     = "ingress"
             ... (truncated) ...
           }
       

       Plan: 3 to add, 0 to change, 0 to destroy.
       
       Changes to Outputs:
         + instance_id       = (known after apply)
         + public_ip         = (known after apply)
         + security_group_id = (known after apply)
       

       [!NOTE] The plan output is similar to the output of the diff command of Linux and git diff:

       Anything with:

       • a plus sign (+) will be created
       • a minus sign (–) will be deleted
       • a tilde sign (~) will be modified in place

       [!TIP] The plan output can also be generated by running tofu plan.

     • …prompt you for confirmation

       Do you want to perform these actions?
         OpenTofu will perform the actions described above.
         Only 'yes' will be accepted to approve.
       
         Enter a value:
       

     • If you type yes and hit Enter, OpenTofu will proceed:

         Enter a value: yes
       

       Output

       aws_security_group.sample_app: Creating...
       aws_security_group.sample_app: Creation complete after 2s
       aws_security_group_rule.allow_http_inbound: Creating...
       aws_security_group_rule.allow_http_inbound: Creation complete after 0s
       aws_instance.sample_app: Creating...
       aws_instance.sample_app: Still creating... [10s elapsed]
       aws_instance.sample_app: Still creating... [20s elapsed]
       aws_instance.sample_app: Creation complete after 22s
       
       Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
       
       Outputs:
       
       instance_id = "i-0a4c593f4c9e645f8"
       public_ip = "3.138.110.216"
       security_group_id = "sg-087227914c9b3aa1e"
       

       • The 3 output variables from outputs.tf is shown at the end.

Example: Update Infrastructure Using OpenTofu

• Make a change to the configuration - add a Test tag with the value of "update"

  resource "aws_instance" "sample_app" {
  
    # ... (other params omitted) ...
  
    tags = {
      Name = "sample-app-tofu"
      Test = "update"
    }
  }
  

• Run tofu apply command again

  tofu apply
  

  Output

  aws_security_group.sample_app: Refreshing state...
  aws_security_group_rule.allow_http_inbound: Refreshing state...
  aws_instance.sample_app: Refreshing state...
  
  OpenTofu used the selected providers to generate the following execution plan.
  Resource actions are indicated with the following symbols:
    ~ update in-place
  
  OpenTofu will perform the following actions:
  
    # aws_instance.sample_app will be updated in-place
    ~ resource "aws_instance" "sample_app" {
        id = "i-0738de27643533e98"
      ~ tags = {
            "Name" = "sample-app-tofu"
          + "Test" = "update"
        }
        # (31 unchanged attributes hidden)
  
        # (8 unchanged blocks hidden)
      }
  
  

  Plan: 0 to add, 1 to change, 0 to destroy.
  
  Do you want to perform these actions?
  OpenTofu will perform the actions described above.
  Only 'yes' will be accepted to approve.
  
  Enter a value:
  

• OpenTofu will update the EC2 instance after you type yes and press Enter

[!NOTE] How OpenTofu know which infrastructure to update?

• Every time you run OpenTofu, it records information about the infrastructure it created/updated?
  • in an OpenTofu state file.

[!NOTE] How OpenTofu manages the information about the infrastructure it has created/updated?

• OpenTofu manages state using backends:
  • The default backend is local backend:
    • State is stored locally in a terraform.tfstate file (in the same folder as the OpenTofu module)

• For the previous example and this example:
  • When you run apply the first on the tofu module:
    • OpenTofu records in the files the IDs of the EC2 instance, security group, security group rules, and any other resources it created
  • When you run apply again:
    • OpenTofu updates it view of the world (Refreshing state...):
      • OpenTofu performs a diff of
        • the current state (in state file)
        • the desired state (in your OpenTofu code)
      • OpenTofu then show its execution plan: the actions it will perform (to transform the current state to the desired state).

Example: Destroy Infrastructure Using OpenTofu

• To destroy everything you’ve deployed with an OpenTofu module, you use destroy command

  tofu destroy
  

  Detail of the actions

  
  OpenTofu will perform the following actions:
  
    # aws_instance.sample_app will be destroyed
    - resource "aws_instance" "sample_app" {
        - ami                                  = "ami-0ee5157dd67ca79fc" -> null
        - associate_public_ip_address          = true -> null
        - id                                   = "i-0738de27643533e98" -> null
        ... (truncated) ...
      }
  
    # aws_security_group.sample_app will be destroyed
    - resource "aws_security_group" "sample_app" {
        - id                     = "sg-066de0b621838841a" -> null
        ... (truncated) ...
      }
  
    # aws_security_group_rule.allow_http_inbound will be destroyed
    - resource "aws_security_group_rule" "allow_http_inbound" {
        - from_port              = 8080 -> null
        - protocol               = "tcp" -> null
        - to_port                = 8080 -> null
        ... (truncated) ...
      }
  

  Plan: 0 to add, 0 to change, 3 to destroy.
  
  Changes to Outputs:
  
  - instance_id = "i-0738de27643533e98" -> null
  - public_ip = "18.188.174.48" -> null
  - security_group_id = "sg-066de0b621838841a" -> null
  
  

  Do you really want to destroy all resources?
    OpenTofu will destroy all your managed infrastructure, as shown above.
    There is no undo. Only 'yes' will be accepted to confirm.
  
    Enter a value:
  

• Type yes and hit Enter to confirm that you want OpenTofu to execute its destroy plan.

[!CAUTION] Be careful when you run destroy in production.

• It’s a one way door 🚪. There’s no "undo".

Get your hands dirty with OpenTofu - Part 1

1. How would you have to tweak the OpenTofu code if you wanted to run multiple EC2 instances?
2. Figure out how to configure the EC2 instance with an EC2 key pair so you can connect to it over SSH.

Example: Deploy an EC2 Instance Using an OpenTofu “Reusable Module”

[!NOTE] OpenTofu modules are containers for multiple resources that are used together.

There are 2 types modules in OpenTofu:

• root module: any module on which you run apply directly.
• reusable module: a module meant to be included in others modules (root modules, reusable modules).

So far, you’ve only used the root module - the ec2-instance module.

In this example, you will transform the ec2-instance as a root module into a reusable module.

• Create 3 folders: modules, live, sample-app:

  mkdir -p examples/ch2/tofu/modules         # For reusable modules
  mkdir -p examples/ch2/tofu/live            # For root modules
  mkdir -p examples/ch2/tofu/live/sample-app # The sample-app (root module) that use the ec2-instance reusable module
  

• Move the ec2-instance module into the modules folder:

  mkdir -p example/ch2/tofu/modules
  mv ch2/tofu/ec2-instance ch2/tofu/modules/ec2-instance
  

• In the sample-app folder, create main.tf for the main resources of the sample app:

  # examples/ch2/tofu/live/sample-app/main.tf
  module "sample_app_1" {                 # 1️⃣
    source = "../../modules/ec2-instance" # 2️⃣
  
    # TODO: fill in with your own AMI ID!
    ami_id = "ami-09a9ad4735def0515"      # 3️⃣
  }
  

  What does the code do?

  • 1️⃣ - module block: calls a reusable module from a parent module.
  • 2️⃣ - source parameter: path to a local directory containing the child module’s configuration files, e.g. ../../modules/ec2-instance
  • 3️⃣ - other parameters that will be passed to the module as input variables, e.g. ami_id

  If you run apply on sample-app module, OpenTofu will use the ec2-instance module to to create an EC2 instance (, security group and security group rules)

  [!NOTE] Modules are the main way to package & reuse resource configurations with OpenTofu.

  e.g.

  • Create multiple resources that meant to be used together (module ~ package)
  • Create same type of resource multiple times (module ~ function)

  [!TIP] What happen if you run a root module multiple times?

  • It will create/update the resources in that root module.

  [!TIP] So how do you reuse a module to create a group of resources multiple times?

  • You can’t re-apply a root module to do that.

  • You need to apply a root module that call another reusable module multiple times.

    e.g.

    module "sample_app_1" {
      source = "../../modules/ec2-instance"
    
      ami_id = "ami-XXXXXXXXXXXXXXXXX"
    }
    
    module "sample_app_2" {
      source = "../../modules/ec2-instance"
    
      ami_id = "ami-XXXXXXXXXXXXXXXXX"
    }
    

• Namespace all the resources created by the ec2-instance module.

  • Introduce a name input variable to use as the base name for resources of the ec2-instance module

    # examples/ch2/tofu/modules/ec2-instance/variables.tf
    variable "name" {
      description = "The base name for the instance and all other resources"
      type        = string
    }
    

  • Update the ec2-instance module to use the name input variable everywhere that was hard-coded:

    resource "aws_security_group" "sample_app" {
      name        = var.name
      description = "Allow HTTP traffic into ${var.name}"
    }
    
    resource "aws_instance" "sample_app" {
    
      # ... (other params omitted) ...
    
      tags = {
        Name = var.name
      }
    }
    

  • Back to sample-app/main.tf, set the name input to different values in each module block

    # examples/ch2/tofu/live/sample-app/main.tf
    module "sample_app_1" {
      source = "../../modules/ec2-instance"
    
      ami_id = "ami-XXXXXXXXXXXXXXXXX"
    
      name = "sample-app-tofu-1"
    }
    
    module "sample_app_2" {
      source = "../../modules/ec2-instance"
    
      ami_id = "ami-XXXXXXXXXXXXXXXXX"
    
      name = "sample-app-tofu-2"
    }
    

• Move the provider block (from the ec2-instance module) to the sample-app root module:

  # examples/ch2/tofu/live/sample-app/main.tf
  provider "aws" {
    region = "us-east-2"
  }
  
  module "sample_app_1" {
    # ...
  }
  module "sample_app_2" {
    # ...
  }
  

  [!NOTE] Typically, reusable module

  • do not declare provider blocks,
  • but inherit from root module. ← Any user of this reusable module can configure the provider in different ways for different usages.

• Finally, proxy the output variables from the ec2-instance module

  output "sample_app_1_public_ip" {
    value = module.sample_app_1.public_ip
  }
  
  output "sample_app_2_public_ip" {
    value = module.sample_app_2.public_ip
  }
  
  output "sample_app_1_instance_id" {
    value = module.sample_app_1.instance_id
  }
  
  output "sample_app_2_instance_id" {
    value = module.sample_app_2.instance_id
  }
  

The reusable module ec2-instance is ready, let’s init & apply the example-app

tofu init
tofu apply

Example: Deploy an EC2 Instance Using an OpenTofu “Reusable Module” from GitHub

[!NOTE] The OpenTofu module’s source parameter can be set a lot of different source types¹³.

• a local path
• Terraform Registry
• GitHub/Git repositories
• HTTP URLs
• S3 buckets, GCP buckets.
• …

In this example, you will set the sample-app module source to a GitHub repository (github.com/brikis98/devops-book), with the same source code for the ec2-instance module at the path ch2/tofu/modules/ec2-instance.

• Modify the source parameter

   module "sample_app_1" {
     source = "github.com/brikis98/devops-book//ch2/tofu/modules/ec2-instance"
  
     # ... (other params omitted) ...
   }
  

  • The double lash (//) is used to separate the Github repo & the path of module (in that repo)

• Run init:

  tofu init
  

  Initializing the backend...
  Initializing modules...
  Downloading git::https://github.com/brikis98/devops-book.git...
  Downloading git::https://github.com/brikis98/devops-book.git...
  
  Initializing provider plugins...
  

  • The init command will download the module code (from GitHub) & the provider code.

• Run apply and you will have the exact same two EC2 instance as the previous example.

[!WARNING] When you’re done experimenting, don’t forget to run destroy to clean everything up.

[!IMPORTANT] A common pattern at many company is:

• The Ops team define & manage a library of well-tested, reusable OpenTofu modules:
  • Module for deploying server
  • Module for deploying database
  • Module for configuring networking
  • …
• The Dev teams use these modules as a self-service way to deploy & manage the infrastructure they need for their apps

Get your hands dirty with OpenTofu - Part 2

1. Make the ec2-instance module more configurable:

   e.g. add input variables to configure

   • the instance type it uses,
   • the port it opens up for HTTP requests, and so on.

2. Instead of having to provide the AMI ID manually, make OpenTofu find the ID of your AMI automatically (Tip: Use data sources)

How Provisioning Tools Stack Up

[!IMPORTANT] Key takeaway #2.4 Provisioning tools are

• great for deploying & managing servers or infrastructure.

[!TIP] Many provisioning tools support:

• not only manage traditional infrastructure, e.g. servers
• but also many aspects of software delivery e.g. OpenTofu can manage
  • Version control system, e.g. GitHub
  • Metrics & dashboard, e.g. Grafana
  • On-call rotation, e.g. PagerDuty

Using Multiple IaC Tools Together

[!IMPORTANT] Key takeaway #2.5 You usually need to use multiple IaC tools together to manage your infrastructure.

Provisioning + Configuration Management

Example: OpenTofu + Ansible

• OpenTofu: Deploy all infrastructure:

  • networking, e.g. VPCs, subnets, route tables
  • load balancers
  • data stores, e.g. MySQL, Redis
  • servers

• Ansible: Deploy apps on top of these servers

 App      +  App      +  App      +  App      +  App      + ... | ← ANSIBLE
(Deps...)   (Deps...)   (Deps...)   (Deps...)   (Deps...)       |


 Server   + Server    + Server    + Server    + Server    + ... |
                                                                | ← OPENTOFU
 Networking, load balancers, data stores, users...              |

Provisioning + Server Templating

Example: OpenTofu + Packer ← Immutable infrastructure approach

• Packer: Package app as VM images
• OpenTofu: Deploy
  • networking, load balancers, data stores…
  • servers from VM images

 Server        +  Server        +  Server        +  Server         + ... | ← 3. OPENTOFU

 VM            +  VM            +  VM            +  VM             + ... | ← 2. PACKER
(App, Deps...)   (App, Deps...)   (App, Deps...)   (App, Deps...)        |

 Networking, load balancers, data stores, users...                       | ← 1. OPENTOFU

Provisioning + Server Templating + Orchestration

[!TIP] Orchestration tools - Kubernetes, Nomad, OpenShift - help you deploy & manages apps on top of your infrastructure.

Example: OpenTofu + Packer + Docker & Kubernetes

• Packer: Create a VM image that has Docker & Kubernetes agents installed.
• OpenTofu: Deploy
  • networking, load balancers, data stores…
  • a cluster of servers, each with the built VM image ← forms a Kubernetes cluster

The Kubernetes cluster is used to you run & manage your Dockerized applications.

                  Container                                              |
 Container        Container                         Container            | ← 4. KUBERNETES + DOCKER
 Container        Container        Container        Container            |

 VM            +  VM            +  VM            +  VM             + ... | ← 2. PACKER
(Docker, K8s)    (Docker, K8s)    (Docker, K8s)    (Docker, K8s)         |

 Server        +  Server        +  Server        +  Server         + ... | ← 3. OPENTOFU
                                                                         |
 Networking, load balancers, data stores, users...                       | ← 1. OPENTOFU

This approach

• has many advantages:

  • Docker images built quickly → Can run & test on your PC.
  • Kubernetes builtin functionality: auto healing/scaling, various deployment strategies…

• but also has the drawbacks in added complexity:

  • extra infrastructure to run (K8s clusters are difficult¹⁴ & expensive to deploy, manage)
  • several extra layers of abstraction - K8s, Docker, Packer - to learn, manage & debug.

Conclusion

• Instead of ClickOps (clicking out a web UI, which is tedious & error-prone), you can use IaC tools to:

  • automate the process
  • make it faster & more reliable

• With IaC, you can reuse code written by others:

  • Open source code, e.g. Ansible Galaxy, Docker Hub, Terraform Registry
  • Commercial code, e.g. Gruntwork IaC Library

• Pick the right IaC tool for the job:

  IaC tool	Great for	Not for
  Ad-hoc scripts	Small, one-off tasks	Managing IaC
  Configuration management tools	Managing configuration of servers	Deploying servers/infrastructure.
  Server templating tools	Managing configuration of servers with immutable infrastructure practices
  Provision tools	Deploying & managing servers/infrastructure

• You usually needs to use multiple IaC tools together to manage your infrastructure.

  e.g.

  • Provisioning + configuration management
  • Provisioning + server templating
  • Provisioning + server templating + orchestration

Chapter 3: How to Deploy Many Apps: Orchestration, VMs, Containers, and Serverless

• Chapter 3: How to Deploy Many Apps: Orchestration, VMs, Containers, and Serverless
  • An Introduction to Orchestration
    • Why use an orchestration?
    • What is an orchestration?
    • Four types of orchestration tools
  • Server Orchestration
    • What is Server Orchestration
    • Example: Deploy Multiple Servers in AWS Using Ansible
    • Example: Deploy an App Securely and Reliably Using Ansible
    • Example: Deploy a Load Balancer Using Ansible and Nginx
      • Introduction to Load Balancer
        • What is load balancer?
        • How load balancer works?
      • The example
    • Example: Roll Out Updates to Servers with Ansible
    • Get your hands dirty with Ansible and server orchestration
  • VM Orchestration
    • What is VM Orchestration
    • Example: Build a More Secure, Reliable VM Image Using Packer
    • Example: Deploy a VM Image in an Auto Scaling Group Using OpenTofu
    • Example: Deploy an Application Load Balancer Using OpenTofu
      • The problem with deployed your own load balancer using Nginx
      • Using cloud providers managed services for load balancing
      • The example code
    • Example: Roll Out Updates with OpenTofu and Auto Scaling Groups
    • Get your hands dirty with OpenTofu and VM orchestration
  • Container Orchestration
    • What is Container Orchestration
    • The advantages of container orchestration
    • Containers and container orchestration tools
    • A Crash Course on Docker
      • Install Docker
      • Basic Docker commands
        • docker run
        • docker ps
        • docker start
    • Example: Create a Docker Image for a Node.js app
    • A Crash Course on Kubernetes
      • What is Kubernetes?
      • Why Kubernetes?
      • Run Kubernetes on personal computer
      • How to use Kubernetes?
    • Example: Deploy a Dockerized App with Kubernetes
    • Example: Deploy a Load Balancer with Kubernetes
    • Example: Roll Out Updates with Kubernetes
    • Get your hands dirty with Kubernetes and YAML template tools
    • A Crash Course on AWS Elastic Kubernetes Service (EKS)
      • Why use a managed Kubernetes service
      • What is EKS
    • Example: Deploy a Kubernetes Cluster in AWS Using EKS
      • The eks-cluster OpenTofu module
      • Using the OpenTofu module to deploy an Kubernetes cluster using EKS
    • Example: Push a Docker Image to AWS Elastic Container Registry (ECR)
      • Container registry and ECR
      • Using ecr-repo OpenTofu module to create an ECR repo
    • Example: Deploy a Dockerized App into an EKS Cluster (With Load Balancer)
    • Get your hands dirty with Kubernetes and container orchestration
  • Serverless Orchestration
    • What is Serverless?
    • How Serverless works?
    • Serverless pros and cons
    • Type of serverless
    • Example: Deploy a Serverless Function with AWS Lambda
      • The lambda OpenTofu module
      • Using the lambda OpenTofu module to deploy a AWS Lambda function
    • A Crash course on AWS Lambda triggers
    • Example: Deploy an API Gateway in Front of AWS Lambda
      • The api-gateway OpenTofu module
      • Using api-gateway OpenTofu module to deploy an API Gateway in Front of AWS Lambda
    • Example: Roll Out Updates with AWS Lambda
    • Get your hands dirty with serverless web-apps and Serverless Orchestration
  • Comparing Orchestration Options
  • Conclusion

An Introduction to Orchestration

Why use an orchestration?

• The problem with a single server 🎵 - single point of failure:

  • Your single server cannot run all the time:
    • There will be a lot of outages 🛑 due to:
      • hardware issues
      • software issues
      • load: 🪨
      • deployments

• To remove this single point of failure, typically you needs

  • multiple copies, called replicas, of your app.
  • a way to
    • manages those replicas 👈 Who gonna be the manager 🧠🎼?
    • solve all the problems (of each server) 👈 Multiple failures ← Multiple servers 🎵🎵🎶
    • …

• The tools that done all of the previous things is called orchestration tools:

  • Capistrano, Ansible (👈 Server orchestration)
  • AWS Auto Scaling Group, EC2 (👈 VM orchestration)
  • Kubernetes, Nomad… & managed services: EKS, GKE, AKS, OpenShift, ECS (👈 Container orchestration)
  • AWS Lambda, Google Cloud Functions, Azure Serverless (👈 Serverless orchestration)

What is an orchestration?

orchestration tool : tool responsible for orchestration: : - manage the cluster (where the applications runs) : - coordinate individual apps to start/stop (how each application runs) : - increase/decrease hardware resources available to each app (which is available to each applications) : - increase/decrease the number of replicas (how many copies of each application) : - …

• An orchestration tool solves the following problems:

  The problem	What exactly is the problem?	Notes
  🚀 Deployment	How to initially deploy one/more replicas of your app onto your servers?
  🎢 Deployments strategies	How to roll out updates to all replicas? Without downtime1?
  🔙 Rollback	How to roll back a broken updates? Without downtime?
  🆔 Scheduling	How to decide which apps should run on which servers? With enough computing resources2?	Scheduling can be done: - manually - automatically by a scheduler3.
  ⬆️⬇️ Auto scaling	How to auto-scale your app up/down in response to the load?	There are 2 types of scaling: - vertical scaling (a “bigger” machine) - horizontal scaling (more small machines).
  🩹 Auto healing	How to know if an app/a server is not healthy? Auto restart/replace the app/server?
  ⚙️ Configuration	How to configure the app for multiple environments?	e.g. Each environment has a different domain name; computing resources settings.
  🔒🤫 Secrets management	How to pass secrets to your apps?	aka credentials - e.g. passwords, API keys
  ☦️ Load balancing	How to distribute traffic across all replicas?
  🌐🕸️ Service communication	How each app communicate/connect with each other?	aka service discovery
  	How to control/monitor the these communication/connections?	aka service mesh: authentication, authorization, encryption, error handling, observability…
  💾 Disk management	How to connect the right hard drive to the right servers?

Four types of orchestration tools

• In the pre-cloud ere, most companies has their own solutions: gluing together various scripts & tools to solve each problem.

• Nowadays, the industry standardize around four broad types of solutions:

  Type of orchestration	How do you do?
  “Server orchestration” (aka “deployment tooling”)	You have a pool of servers that you manage.	The old way from pre-cloud era, still common today.
  VM orchestration	Instead of managing servers directly, you manage VM images.
  Container orchestration	Instead of managing servers directly, you manage containers.
  Serverless orchestration	You no longer think about servers at all, and just focus on managing apps, or even individual functions.

Server Orchestration

What is Server Orchestration

server orchestration : the original approach from pre-cloud era, but still common today : setup a bunch of servers → deploy apps across these servers → changes are update in-place to these servers : there is no standardized toolings in this approach : - configuration management tools, e.g. Ansible, Chef, Puppet : - specialized deployment scripts, e.g. Capistrano, Deployer, Mina : - thousands of ad-hoc scripts

[!IMPORTANT] Key takeaway #1 Server orchestration is an older, mutable infrastructure approach where

• you have a fixed set of servers that you
  • maintain
  • update in-place.

Example: Deploy Multiple Servers in AWS Using Ansible

[!WARNING] Deploy & manage servers is not really what configuration management tools were designed to do.

• But for learning & testing, Ansible is good enough.

First, to use Ansible as a server orchestration, you need

• a bunch of servers (that will be managed, e.g. physical servers on-prem, virtual servers in the could)
• SSH access to those servers.

If you don’t have servers you can use, you can also use Ansible to deploy several EC2 instances.

The Ansible playbook to create multiples EC2 instance can be found at the example repo at ch3/ansible/create_ec2_instances_playbook.yml, which will:

• Prompt you for:
  • number_instances: The number of instances to create
  • basename: The basename for all resources created
  • http_port: The port on which the instances listen for HTTP requests
• Create a security group that opens port 22 (for SSH traffic) and http_port (for HTTP traffic)
• Create a EC2 key-pair that used to connect to the instances (that will be created) via SSH.
• Create multiple instances, each with the Ansible tag set to base_name

To run the playbook:

• Copy create_ec2_instances_playbook.yml from example repo to ch3/ansible

  mkdir -p ch3/ansible
  cd ch3/ansible
  
  cp -r <PATH_TO_EXAMPLE_REPO>/ch3/ansible/create_ec2_instances_playbook.yml .
  

• Use ansible-playbook command to run the playbook

  ansible-playbook -v create_ec2_instances_playbook.yml
  

  • Enter the values interactively & hit Enter

  • Or define the values as variables in a yaml file and pass to ansible-playbook command via -extra-vars flag.

    # examples/ch3/ansible/sample-app-vars.yml
    num_instances: 3
    base_name: sample_app_instances
    http_port: 8080
    

    ansible-playbook -v create_ec2_instances_playbook.yml \
      --extra-vars "@sample-app-vars.yml"
    

Example: Deploy an App Securely and Reliably Using Ansible

Previous chapters has basic example of deploying an app:

• Chapter 1: Example: Deploying an app using AWS: Deploy an app to AWS with “ClickOps”
• Chapter 2: Example: Configure a Server Using Ansible: Deploy an app to AWS with Ansible

There’re still several problems with both examples (e.g. root user, port 80, no automatic app restart…)

In this example, you will fix these problems and deploy the app in a more secure, reliable way.

• (As previous example) Use an Ansible Inventory plugin to discover your EC2 instances

  # examples/ch3/ansible/inventory.aws_ec2.yml
  plugin: amazon.aws.aws_ec2
  regions:
    - us-east-2
  keyed_groups:
    - key: tags.Ansible
  leading_separator: ""
  

• (As previous example) Use group variables to store the configuration for your group of servers

  # examples/ch3/ansible/group_vars/sample_app_instances.yml
  ansible_user: ec2-user
  ansible_ssh_private_key_file: ansible-ch3.key
  ansible_host_key_checking: false
  

• Use a playbook to configure your group of servers to run the Node.js sample app

  # examples/ch3/ansible/configure_sample_app_playbook.yml
  - name: Configure servers to run the sample-app
    hosts: sample_app_instances # 1️⃣
    gather_facts: true
    become: true
    roles:
      - role: nodejs-app #        2️⃣
      - role: sample-app #        3️⃣
        become_user: app-user #   4️⃣
  

  • 1️⃣: Target the group discovered by the inventory plugin (which are created in the previous example).
  • 2️⃣: Split the role into 2 smaller roles: the nodejs-app role is only responsible for configuring the server to be able to run any Node.js app.
  • 3️⃣: The sample-app role is now responsible for running the sample-app.
  • 4️⃣: The sample-app role will be executed as the OS user app-user - which is created by the nodejs-app role - instead of the root user.

• The Ansible roles

  roles
    └── nodejs-app
        └── tasks
            └── main.yml
  

• The nodejs-app role: a generic role for any Node.js app

  roles
    └── nodejs-app
        └── tasks
            └── main.yml # The Ansible role's task
  

  # examples/ch3/ansible/roles/nodejs-app/tasks/main.yml
  - name: Add Node packages to yum #                                 1️⃣
    shell: curl -fsSL https://rpm.nodesource.com/setup_21.x | bash -
  - name: Install Node.js
    yum:
      name: nodejs
  
  - name: Create app user #                                          2️⃣
    user:
      name: app-user
  
  - name: Install pm2 #                                              3️⃣
    npm:
      name: pm2
      version: latest
      global: true
  - name: Configure pm2 to run at startup as the app user
    shell: eval "$(sudo su app-user bash -c 'pm2 startup' | tail -n1)"
  

  • 1️⃣: Install Node.js
  • 2️⃣: Create a new OS user called app-user, which allows you to run yours app with an OS user with limited permissions.
  • 3️⃣: Install PM2 (a process supervisor⁴) and configure it to run on boot.

• The sample-app role: a specifically role to run the sample-app.

  roles
    ├── nodejs-app
    └── sample-app
        ├── files
        │   ├── app.config.js # The configuration file for the process supervisor - PM2
        │   └── app.js        # Your example-app code
        └── tasks
            └── main.yml      # The Ansible role's task
  

  • Clone the example-app code (from chapter 1):

    cd examples
    mkdir -p ch3/ansible/roles/sample-app/files
    cp ch1/sample-app/app.js ch3/ansible/roles/sample-app/files/
    

  • The PM2 configuration file

    # examples/ch3/ansible/roles/sample-app/files/app.config.js
    module.exports = {
      apps : [{
        name   : "sample-app",
        script : "./app.js", #       1️⃣
        exec_mode: "cluster", #      2️⃣
        instances: "max", #          3️⃣
        env: {
          "NODE_ENV": "production" # 4️⃣
        }
      }]
    }
    

    • 1️⃣: PM2 will run the script at /app.js.
    • 2️⃣: The script will be run in cluster mode⁵ (to take advantages of all the CPUs)
    • 3️⃣: Use all CPUs available
    • 4️⃣: Run Node.js script in “production” mode.

  • The sample-app role’s task

    # examples/ch3/ansible/roles/sample-app/tasks/main.yml
    - name: Copy sample app #                         1️⃣
      copy:
        src: ./
        dest: /home/app-user/
    
    - name: Start sample app using pm2 #              2️⃣
      shell: pm2 start app.config.js
      args:
        chdir: /home/app-user/
    
    - name: Save pm2 app list so it survives reboot # 3️⃣
      shell: pm2 save
    

    • 1️⃣: Copy app.js and app.config.js to home directory of app-user.
    • 2️⃣: Use PM2 (using the app.config.js configuration) to start the app.
    • 3️⃣: Save Node.js processes to restart them later.

• Run Ansible playbook

  ansible-playbook -v -i inventory.aws_ec2.yml configure_sample_app_playbook.yml
  

  Output

  PLAY RECAP ************************************
  13.58.56.201               : ok=9    changed=8
  3.135.188.118              : ok=9    changed=8
  3.21.44.253                : ok=9    changed=8
  localhost                  : ok=6    changed=4
  

  • Now you have three secured, reliable instances of your application (with 3 separated endpoint).

  [!NOTE] Your application now can be accessed via any of the those endpoints. But should your users need to decide which instance they will access?

  • No.
  • You should have a load balancer to distribute load across multiple servers of your app.

Example: Deploy a Load Balancer Using Ansible and Nginx

Introduction to Load Balancer

What is load balancer?

load balancer : a piece of software that can distribute load across multiple servers or apps : e.g. : - Apache httpd⁶, Nginx⁷, HAProxy⁸. : - Cloud services: AWS Elastic Load Balancer, GCP Cloud Load Balancer, Azure Load Balancer.

How load balancer works?

• You give your users a single endpoint to hit, which is the load balancer.

• The load balancer

  • forwards the requests it receives to a number of different endpoints.

  • uses various algorithms to process requests as efficiently as possible.

    e.g. round-robin, hash-based, least-response-time…

The example

In this example, you will run your own load balancer in a separate server (using nginx).

• (Optional) Deploy an EC2 instance for the load balancer:

  You will use the same create_ec2_instances_playbook.yml playbook deploy that EC2 instance:

  • Configure the create_ec2_instances_playbook.yml playbook

    # examples/ch3/ansible/nginx-vars.yml
    num_instances: 1
    base_name: nginx_instances
    http_port: 80
    

  • Run the create_ec2_instances_playbook.yml playbook

    ansible-playbook \
      -v create_ec2_instances_playbook.yml \
      --extra-vars "@nginx-vars.yml"
    

• Use group variables to configure your nginx_instances group

  # examples/ch3/ansible/group_vars/nginx_instances.yml
  ansible_user: ec2-user
  ansible_ssh_private_key_file: ansible-ch3.key
  ansible_host_key_checking: false
  

• Create a playbook to configure your group of servers to run Nginx

  • The playbook

    # examples/ch3/ansible/configure_nginx_playbook.yml
    - name: Configure servers to run nginx
      hosts: nginx_instances
      gather_facts: true
      become: true
      roles:
        - role: nginx
    

  • The playbook’s roles (nginx)

    roles
      ├── nginx
      │   ├── tasks
      │   │   └── main.yml
      │   └── templates
      │       └── nginx.conf.j2
      ├── nodejs-app
      └── sample-app
    

    • The Ansible role’s template for Nginx configuration

      # examples/ch3/ansible/roles/nginx/templates/nginx.conf.j2
      user nginx;
      worker_processes auto;
      error_log /var/log/nginx/error.log notice;
      pid /run/nginx.pid;
      
      events {
          worker_connections 1024;
      }
      
      http {
          log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                            '$status $body_bytes_sent "$http_referer" '
                            '"$http_user_agent" "$http_x_forwarded_for"';
      
          access_log  /var/log/nginx/access.log  main;
      
          include             /etc/nginx/mime.types;
          default_type        application/octet-stream;
      
          upstream backend { #                                       1️⃣
              {% for host in groups['sample_app_instances'] %} #     2️⃣
              server {{ hostvars[host]['public_dns_name'] }}:8080; # 3️⃣
              {% endfor %}
          }
      
          server {
              listen       80; #                                     4️⃣
              listen       [::]:80;
      
              location / { #                                         5️⃣
                      proxy_pass http://backend;
              }
          }
      }
      

      This Nginx configuration file⁹ will configure the load balancer to load balance the traffic across the servers you deployed to run the sample-app:

      • 1️⃣ Use the upstream keyword to define a group of servers that can be referenced elsewhere in this file by the name backend.
      • 2️⃣ (Ansible - Jinja templating syntax¹⁰) Loop over the servers in the sample_app_instances group.
      • 3️⃣ (Ansible - Jinja templating syntax) Configure the backend upstream to route traffic to the public address and port 8080 of each server in the sample_app_instances group.
      • 4️⃣ Configure Nginx to listen on port 80.
      • 5️⃣ Configure Nginx as a load balancer, forwarding requests to the / URL to the backend upstream.

    • The Ansible role’s task to configure Nginx

      # examples/ch3/ansible/roles/nginx/tasks/main.yml
      - name: Install Nginx #           1️⃣
        yum:
          name: nginx
      
      - name: Copy Nginx config #       2️⃣
        template:
          src: nginx.conf.j2
          dest: /etc/nginx/nginx.conf
      
      - name: Start Nginx #             3️⃣
        systemd_service:
          state: started
          enabled: true
          name: nginx
      

      • 1️⃣: Install Nginx (using yum)
      • 2️⃣: Render the Jinja template to Nginx configuration file and copy to the server.
      • 3️⃣: Start Nginx (using systemd as the process supervisor).

• Run the playbook to configure your group of servers to run Nginx

  ansible-playbook -v -i inventory.aws_ec2.yml configure_nginx_playbook.yml
  

  Output

  PLAY RECAP
  xxx.us-east-2.compute.amazonaws.com : ok=4    changed=2    failed=0
  

Example: Roll Out Updates to Servers with Ansible

[!NOTE] Some configuration management tools support various deployment strategies.

e.g.

• Rolling deployment: you update your severs in batches:
  • Some servers are being updated (with new configuration).
  • While others servers keep running (with old configuration) and serving traffic.

• With Ansible, the easiest way to have a rolling update is to add the serial parameter to the playbook.

  # examples/ch3/ansible/configure_sample_app_playbook.yml
  - name: Configure servers to run the sample-app
    hosts: sample_app_instances
    gather_facts: true
    become: true
    roles:
      - role: nodejs-app
      - role: sample-app
        become_user: app-user
    serial: 1 #               1️⃣
    max_fail_percentage: 30 # 2️⃣
  

  • 1️⃣: Apply changes to the servers in batch-of-1 (1 server at a time)
  • 2️⃣: Abort a deployment more than 30% of the servers hit an error during update.
    • For this example, it means the deployment will stop if there is any of the server fails.

• Make a change to the application

  sed -i 's/Hello, World!/Fundamentals of DevOps!/g' examples/ch3/ansible/roles/sample-app/files/app.js
  

• Re-run the playbook

  ansible-playbook -v -i inventory.aws_ec2.yml configure_sample_app_playbook.yml
  

Get your hands dirty with Ansible and server orchestration

1. How to scale the number of instances running the sample app from three to four.
2. Try restarting one of the instances using the AWS Console.
   • How does nginx handle it while the instance is rebooting?
   • Does the sample app still work after the reboot?
   • How does this compare to the behavior you saw in Chapter 1?
3. Try terminating one of the instances using the AWS Console.
   • How does nginx handle it?
   • How can you restore the instance?

VM Orchestration

What is VM Orchestration

VM orchestration : Create VM images that have your apps & dependencies fully installed & configured : Deploy the VM images across a cluster of servers : - 1 server → 1 VM image : - Scale the number of servers up/down depending on your needs : When there is an app change: : - Create new VM image 👈 Immutable infrastructure approach. : - Deploy that new VM image onto new servers; then undeploy the old servers.

VM orchestration is a more modern approach:

• works best with cloud providers (AWS, Azure, GCP…) - where you can spin up new servers & tear down old ones in minutes.
• or you an use virtualization on-prem with tools from VMWare, Citrix, Microsoft Hyper-V…

[!IMPORTANT] Key takeaway #2 VM orchestration is an immutable infrastructure approach where you deploy and manage VM images across virtualized servers.

[!NOTE] With VM orchestration, you will deploy multiple VM servers, aka a cluster (of VM servers)

Most cloud providers has a native way to run VMs across a cluster:

• AWS Auto Scaling Groups (ASG)
• Azure Scale Sets
• GCP Managed Instance Groups

The following tools are used in the examples for VM orchestration:

1. A tool for building VM images: Packer
2. A tool for orchestrating VMs: AWS Auto Scaling Group (ASG)
3. A tool for managing IaC: OpenTofu

Example: Build a More Secure, Reliable VM Image Using Packer

An introduction about building an VM image using Packer has already been available at Chapter 2 - Building a VM image using Packer.

This example will make the VM image more secure, reliable:

• Use PM2 as the process supervisor
• Create a OS user to run the app

• Copy Packer template from chapter 2

  cd examples
  mkdir -p ch3/packer
  cp ch2/packer/sample-app.pkr.hcl ch3/packer/
  

• Copy the app & PM2 configuration file from chapter 3

  cp ch3/ansible/roles/sample-app/files/app*.js ch3/packer/
  

• Update the Packer template’s build steps to make the VM image more secure, reliable

  # examples/ch3/packer/sample-app.pkr.hcl
  build {
    sources = [
      "source.amazon-ebs.amazon_linux"
    ]
  
    provisioner "file" { #                                                1️⃣
      sources     = ["app.js", "app.config.js"]
      destination = "/tmp/"
    }
  
    provisioner "shell" {
      inline = [
        "curl -fsSL https://rpm.nodesource.com/setup_21.x | sudo bash -",
        "sudo yum install -y nodejs",
        "sudo adduser app-user", #                                        2️⃣
        "sudo mv /tmp/app.js /tmp/app.config.js /home/app-user/", #       3️⃣
        "sudo npm install pm2@latest -g", #                               4️⃣
        "eval \"$(sudo su app-user bash -c 'pm2 startup' | tail -n1)\"" # 5️⃣
      ]
      pause_before = "30s"
    }
  }
  

  • 1️⃣: Copy app.js & app.config.js onto the server /tmp folder (The home folder of app-user hasn’t existed yet).
  • 2️⃣: Create app-user (and its home folder).
  • 3️⃣: Move app.js & app.config.js to app-user’s home folder.
  • 4️⃣: Install PM2.
  • 5️⃣: Run PM2 on boot (as app-user) so if your server ever restarts, pm2 will restart your app.

• Install Packer plugins (used in the Packer template)

  packer init sample-app.pkr.hcl
  

• Build image from Packer template

  packer build sample-app.pkr.hcl
  

Example: Deploy a VM Image in an Auto Scaling Group Using OpenTofu

In chapter 2, you’ve already used OpenTofu to deploy an AMI on a single EC2 instance using a root module, or using a reusable module.

In this chapter, you will use an OpenTofu reusable module asg to deploy multiples EC2 instances to a cluster

[!TIP] ASG offers a number of nice features:

• Cluster management: You can easily launch multiple instances & manually resize the cluster.
• Auto scaling: Or let ASG resize the cluster automatically (in response to load).
• Auto healing: ASG monitors all instances (in the cluster) and automatically replace any failure instances.

[!NOTE] The asg module is available in this book code repo at github.com/brikis98/devops-book (in ch3/tofu/modules/asg folder).

The asg module will creates 3 main resources:

• A launch template: ~ the blueprint for the configuration of each EC2 instance.
• An ASG: use the launch template to spin up EC2 instances (in the Default VPC)
• A security group: control the traffic in/out of each EC2 instance.

[!NOTE] A VPC - virtual private cloud, is an isolated area of your AWS account that has its own virtual network & IP address space.

• Just about every AWS resource deploys into a VPC.
• If you don’t explicitly specify a VPC, the resource will be deployed into the Default VPC, which is part of every AWS account created after 2013.

[!WARNING] It’s not a good idea to use the Default VPC for production apps, but it’s OK to use it for learning and testing.

• To use the asg module, first you need a root module live/asg-sample:

  • The root module folder

    mkdir -p examples/ch3/tofu/live/asg-sample
    cd examples/ch3/tofu/live/asg-sample
    

• The root module’s main.tf

  # examples/ch3/tofu/live/asg-sample/main.tf
  provider "aws" {
    region = "us-east-2"
  }
  
  module "asg" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/asg"
  
    name = "sample-app-asg" #                                         1️⃣
  
    ami_id        = "ami-XXXXXXXXXXXXXXXXX" #                         2️⃣
    user_data     = filebase64("${path.module}/user-data.sh") #       3️⃣
    app_http_port = 8080 #                                            4️⃣
  
    instance_type    = "t2.micro" #                                   5️⃣
    min_size         = 1 #                                            6️⃣
    max_size         = 10 #                                           7️⃣
    desired_capacity = 3 #                                            8️⃣
  }
  

  • 1️⃣ name: Base name of all resources in asg module.
  • 2️⃣ ami_id: AMI to use for each EC2 instance.
  • 3️⃣ user_data: User data script to run on each EC2 instance.
  • 4️⃣ app_http_port: Port to open in the security group (to allow the app to receive HTTP requests).
  • 5️⃣ instance_type: Type of EC2 instance.
  • 6️⃣ min_size: Minimum number of EC2 instances (to run in the ASG).
  • 7️⃣ max_size: Maximum number of EC2 instances (to run in the ASG).
  • 8️⃣ desired_capacity: The desired (initial) number of instances (to run in the ASG).

  For more information, see:

  • The asg module code
  • The Terraform docs for AWS provider’s ASG resource.

• The user data script used for EC2 instance:

  #!/usr/bin/env bash
  # examples/ch3/tofu/live/asg-sample/user-data.sh
  set -e
  
  sudo su app-user #        1️⃣
  cd /home/app-user #       2️⃣
  pm2 start app.config.js # 3️⃣
  pm2 save #                4️⃣
  

  • 1️⃣: Switch to app-user.
  • 2️⃣: Go to app-user home directory (where the Packer template copied the sample app code).
  • 3️⃣: Use PM2 to start the sample-app.
  • 4️⃣: Tell PM2 to save all processes for resurrecting them later.

• Apply the OpenTofu code

  tofu apply
  

Example: Deploy an Application Load Balancer Using OpenTofu

The problem with deployed your own load balancer using Nginx

[!WARNING] You can address all these issues of Nginx yourself, but:

• it’s a considerable amount of work.

Using cloud providers managed services for load balancing

Most cloud providers offer managed services for solving common problems, including services for load balancing.

e.g. AWS Elastic Load Balancer (ELB), Azure Load Balancer, GCP Cloud Load Balancer

These services provide lots of powerful features out-of-the-box.

For example, AWS Elastic Load Balancer (ELB):

• ELB out-of-the-box features:

  Aspect	The out-of-the-box solution from load balancing managed service	Example
  🧬 Availability	Under the hood, AWS automatically deploys multiple servers for an ELB so you don’t get an outage if one server crashes.
  ♾️ Scalability	AWS monitors load on the ELB, and if it is starting to exceed capacity, AWS automatically deploys more servers.
  🚧 Maintenance	AWS automatically keeps the load balancer up to date, with zero downtime.
  🛡️ Security	AWS load balancers are hardened against a variety of attacks, including meeting the requirements of a variety of security standards out-of-the-box.	e.g. SOC 2, ISO 27001, HIPAA, PCI, FedRAMP…
  🔒 Encryption	AWS has out-of-the-box support for encryption data	e.g. HTTPS, Mutual TLS, TLS Offloading, auto-rotated TLS certs…

• ELB even has multiple types of load balancers, you can choose the one best fit for your needs:

  • Application Load Balancer (ALB)
  • Network Load Balancer (NLB)
  • Gateway Load Balancer (GWLB)
  • Classic Load Balancer (Classic LB)

[!NOTE] An AWS ALB consists of:

• Listeners: A listener listens for requests on
  • a specific port, e.g. 80
  • protocol, e.g. HTTP
• Listener rules: A listener rule specifies
  • which requests (that come into a listener)
    • to route to which target group, based on rules that match on request parameters:
      • path, e.g. /foo
      • hostname, e.g. bar.example.com
• Target groups: A target group is a group of servers that
  • receive requests from the load balancer.
  • perform health checks on these servers by
    • sending to each server a request on a configuration interval - e.g. every 30s
    • only considering the server as healthy if it
      • returns an expected response (e.g. 200 OK)
        • within a time period (e.g. within 2s)
  • only send requests to servers that pass its health checks.

The example code

For this example, you’ll use ALB, which is simple, best fit for a small app:

• The sample code repo includes a OpenTofu module called alb (in ch3/tofu/modules/alb folder) that deploys a simple ALB.

• Configure a root module asg-sample to uses alb module:

  # examples/ch3/tofu/live/asg-sample/main.tf
  module "asg" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/asg"
  
    # ... (other params omitted) ...
  
  }
  
  module "alb" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/alb"
  
    name                  = "sample-app-alb" # 1️⃣
    alb_http_port         = 80 #               2️⃣
    app_http_port         = 8080 #             3️⃣
    app_health_check_path = "/" #              4️⃣
  }
  

  • 1️⃣ name: Base name for alb module’s resources.
  • 2️⃣ alb_http_port: The port the ALB (listener) listens on for HTTP requests.
  • 3️⃣ app_http_port: The port the app listens on for HTTP requests 👈 The ALB target group will send traffic & health checks to this port.
  • 4️⃣ app_health_check_path: The path to use when sending health check requests to the app.

• Connect the ALB to the ASG:

  # examples/ch3/tofu/live/asg-sample/main.tf
  module "asg" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/asg"
  
    # ... (other params omitted) ...
  
    target_group_arns = [module.alb.target_group_arn] # 1️⃣
  }
  

  • 1️⃣ target_group_arns: Attach the ASG to the ALB target group:

    • Register all of ASG’s instances in the ALB’s target group, which including:

      • The initial instances (when you first launch the ASG).
      • Any new instances that launch later: either as a result of a deployment/auto-healing/auto-scaling.

    • Configure the ASG to use the ALB for health checks & auto-healing.

      • By default, the auto-healing feature is simple:
        • It replaces any instances that crashed 👈 Detect hardware issues.
        • If the instance is still running, but the app is not responding, the ASG won’t know to replace it. 👈 Not detect software issues.
      • By using ALB’s health checks, the ASG will also any instance that fails the ALB - target group - health check 👈 Detect both hardware & software issues.

• Output the ALB domain name from the root module asg-sample:

  # examples/ch3/tofu/live/asg-sample/outputs.tf
  output "alb_dns_name" {
    value = module.alb.alb_dns_name
  }
  

• Apply asg-sample module:

  tofu init
  tofu apply
  

  Output

  Apply complete! Resources: 10 added, 0 changed, 0 destroyed.
  
  Outputs:
  
  alb_dns_name = "sample-app-tofu-656918683.us-east-2.elb.amazonaws.com"
  

Example: Roll Out Updates with OpenTofu and Auto Scaling Groups

[!NOTE] Most of the VM orchestration tools have support for zero-downtime deployments & various deployment strategies.

e.g. AWS ASG has a native feature called instance refresh¹¹, which can update your instances automatically by doing a rolling deployment.

In this example, you will enable instance refresh for the ASG:

• Update the asg-sample module

  module "asg" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/asg"
  
    # ... (other params omitted) ...
  
    instance_refresh = {
      min_healthy_percentage = 100 #  1️⃣
      max_healthy_percentage = 200 #  2️⃣
      auto_rollback          = true # 3️⃣
    }
  
  }
  

  • 1️⃣ min_healthy_percentage: The cluster will never have fewer than the desired number of instances.
  • 2️⃣ max_healthy_percentage: The cluster will keep all the old instances running, deploy new instances, waiting for all new instances to pass health checks, then undeploy old instances. 👈 ~ Blue/green deployments.
  • 3️⃣ auto_rollback: If new instances fail to pass health checks, the ASG will auto rollback, putting the cluster back to its previous working condition.

• Make a change to the app

  sed -i 's/Hello, World!/Fundamentals of DevOps!/g' examples/ch3/packer/app.js
  

• Build the new VM image

  cd examples/ch3/packer
  packer build sample-app.pkr.hcl
  

• Update the asg-sample module’s launch template with the new VM image

• Apply the updated asg-sample module

  cd examples/ch3/packer
  tofu apply
  

  Output

  OpenTofu will perform the following actions:
  
    # aws_autoscaling_group.sample_app will be updated in-place
    ~ resource "aws_autoscaling_group" "sample_app" {
          # (27 unchanged attributes hidden)
  
        ~ launch_template {
              id      = "lt-0bc25ef067814e3c0"
              name    = "sample-app-tofu20240414163932598800000001"
            ~ version = "1" -> (known after apply)
          }
  
          # (3 unchanged blocks hidden)
      }
  
    # aws_launch_template.sample_app will be updated in-place
    ~ resource "aws_launch_template" "sample_app" {
        ~ image_id       = "ami-0f5b3d9c244e6026d" -> "ami-0d68b7b6546331281"
        ~ latest_version = 1 -> (known after apply)
          # (10 unchanged attributes hidden)
      }
  

• Go to EC2 console to verify that the instance refreshing is progressing.

[!NOTE] During the instance refreshing, the load balancer URL should always return a successful response (because it’s zero-downtime deployment).

[!TIP] You can check with curl

while true; do curl http://<load_balancer_url>; sleep 1; done

Output

Hello, World!
Hello, World!
Hello, World!
Hello, World!
Hello, World!
Hello, World!           # 👈 Only responses from the old instances
Fundamentals of DevOps! # 👈 As new instances start to pass health checks, ALB sends requests to these instances
Hello, World!
Fundamentals of DevOps!
Hello, World!
Fundamentals of DevOps!
Hello, World!
Fundamentals of DevOps!
Hello, World!
Fundamentals of DevOps!
Hello, World!
Fundamentals of DevOps! # 👈 Only responses from the new instances
Fundamentals of DevOps!
Fundamentals of DevOps!
Fundamentals of DevOps!
Fundamentals of DevOps!
Fundamentals of DevOps!

Get your hands dirty with OpenTofu and VM orchestration

1. How to scale the number of instances in the ASG running the sample app from three to four.
   • How does this compare to adding a fourth instance to the Ansible code?
2. Try restarting one of the instances using the AWS Console.
   • How does the ALB handle it while the instance is rebooting?
   • Does the sample app still work after the reboot?
   • How does this compare to the behavior you saw when restarting an instance with Ansible?
3. Try terminating one of the instances using the AWS Console.
   • How does the ALB handle it?
   • Do you need to do anything to restore the instance?

[!WARNING] Don’t forget to run tofu destroy to undeploy all your infrastructure created by the OpenTofu module.

Container Orchestration

What is Container Orchestration

container orchestration : Create container images that have your apps & dependencies fully installed & configured : Deploy the container images across a cluster of servers : - 1 server → Multiple containers 👈 Pack the containers as efficiently as possible to each server (bin packing). : - Scale the number of servers and/or containers up/down depending on load. : When there is an app change: : - Create new container image 👈 Immutable infrastructure approach. : - Deploy that new container image onto new containers in the cluster; then undeploy the old containers.

[!NOTE] Although containers has been around for decades (from the 1970s¹²),

• container orchestration only started to explode in popularity around 2013,
  • with the emerge of Docker¹³ (2013) and Kubernetes¹⁴ (2014).

[!IMPORTANT] Key takeaway #3 Container orchestration is an immutable infrastructure approach where you deploy & manage container images across a cluster of servers.

The advantages of container orchestration

Containers and container orchestration tools

There are many tools for container and container orchestration:

• For container: Docker, Moby, CRI-O, Podman, runc, buildkit
• For container orchestration: Kubernetes, Docker Swarm, Amazon ECS, Nomad (by HashiCorp), Marathon/Mesos (by Apache), OpenShift (by RedHat).

[!NOTE] Docker & Kubernetes are the most popular.

Their name are nearly synonymous with container & container orchestration.

[!TIP] The examples in this chapter will use

• the most popular container & container orchestration tools - Docker, Kubernetes
• with the most popular cloud provider - AWS.

A Crash Course on Docker

As from Chapter 2 - Server Templating Tools - Container,

• A container image is like a self-contained “snapshots” of the operating system (OS), the software, the files, and all other relevant details.
• (A container emulates the “user space” of an OS).

Install Docker

If you don’t have Docker installed already, follow the instructions on the Docker website to install Docker Desktop for your operating system.

[!TIP] If you’re using Linux, you can install Docker Engine, which doesn’t run a VM as Docker Desktop¹⁷.

Basic Docker commands

docker run

For example, let’s run a container from ubuntu:24.04 image:

• Run the container

  docker run -it ubuntu:24.04 bash
  

  Unable to find image 'ubuntu:24.04' locally
  24.04: Pulling from library/ubuntu
  Digest: sha256:3f85b7caad41a95462cf5b787d8a04604c
  Status: Downloaded newer image for ubuntu:24.04
  
  root@d96ad3779966:/#
  

• Now you’re in Ubuntu: let’s try your new Ubuntu

  • Check the version of Ubuntu

    root@d96ad3779966:/# cat /etc/os-release
    PRETTY_NAME="Ubuntu 24.04 LTS"
    NAME="Ubuntu"
    VERSION_ID="24.04"
    (...)
    

    [!NOTE] Isn’t it magic? What just happened?

    • First, Docker searches your local filesystem for the ubuntu:24.04 image.
    • If you don’t have that image downloaded already, Docker downloads it automatically from Docker Hub, which is a Docker Registry that contains shared Docker images.
      • The ubuntu:24.04 image happens to be a public Docker image — an official one maintained by the Docker team — so you’re able to download it without any authentication.
    • Once the image is downloaded, Docker runs the image, executing the bash command, which starts an interactive Bash prompt, where you can type.

  • List the files

    root@d96ad3779966:/# ls -al
    total 56
    drwxr-xr-x   1 root root 4096 Feb 22 14:22 .
    drwxr-xr-x   1 root root 4096 Feb 22 14:22 ..
    lrwxrwxrwx   1 root root    7 Jan 13 16:59 bin -> usr/bin
    drwxr-xr-x   2 root root 4096 Apr 15  2020 boot
    drwxr-xr-x   5 root root  360 Feb 22 14:22 dev
    drwxr-xr-x   1 root root 4096 Feb 22 14:22 etc
    drwxr-xr-x   2 root root 4096 Apr 15  2020 home
    lrwxrwxrwx   1 root root    7 Jan 13 16:59 lib -> usr/lib
    drwxr-xr-x   2 root root 4096 Jan 13 16:59 media
    (...)
    

    • That’s not your filesystem.

    [!NOTE] Docker images run in containers that are isolated at the user-space level:

    • When you’re in a container, you can only see the filesystem, memory, networking, etc., in that container.
      • Any data in other containers, or on the underlying host operating system, is not accessible to you,
      • Any data in your container is not visible to those other containers or the underlying host operating system.

    [!NOTE] In other words, the image format is self-contained, which means Docker images run the same way anywhere. 👈 This is one of the things that makes Docker useful for running applications.

  • Write some text to a file

    root@d96ad3779966:/# echo "Hello, World!" > test.txt
    

• Exit the container by hitting Ctrl+D¹⁹

  [!TIP] You will be back in your original command prompt on your underlying host OS

  If you look for the test.txt file you’ve just wrote, you’ll see it doesn’t exist.

• Try running the same Docker image again:

  docker run -it ubuntu:24.04 bash
  

  root@3e0081565a5d:/#
  

  This time,

  • Since the ubuntu:24.04 image is already downloaded, the container starts almost instantly.

  [!NOTE] Unlike virtual machines, containers are lightweight, boot up quickly, and incur little CPU or memory overhead.

  👉 This is another reason Docker is useful for running applications.

  • The command prompt looks different. 👈 You’re now in a totally new container
  • Any data you wrote in the previous container is no longer accessible to you (👈 Containers are isolated from each other)

• Exit the second container by hitting Ctrl+D.

docker ps

You’ve just run 2 containers, let’s see them:

$ docker ps -a
CONTAINER ID   IMAGE            COMMAND    CREATED          STATUS
3e0081565a5d   ubuntu:24.04     "bash"     5 min ago    Exited (0) 16 sec ago
d96ad3779966   ubuntu:24.04     "bash"     14 min ago   Exited (0) 5 min ago

[!NOTE] Use docker ps -a to show all the containers on your system, including the stopped ones.

docker start

You can start a stopped container again using docker start <CONTAINER_ID>.

• Start the first container that you wrote to the text file

  $ docker start -ia d96ad3779966
  root@d96ad3779966:/#
  

  [!NOTE] Using -ia flags with docker start to have an interactive shell and allow you type in. (It has same effect as -it of docker run)

• Confirm that it’s the first container:

  root@d96ad3779966:/# cat test.txt
  Hello, World!
  

Example: Create a Docker Image for a Node.js app

In this example, you will use a container to run the Node.js sample-app:

• The source code of this example is in examples/ch3/docker

  mkdir -p examples/ch3/docker
  

• Copy the sample-app source code:

  cp example/ch3/ansible/roles/sample-app/files/app.js example/ch3/docker
  

• Create a file named Dockerfile

  [!NOTE] The Dockerfile is a template that defines how to build a Docker image.

  # examples/ch3/docker/Dockerfile
  FROM node:21.7         # 1️⃣
  WORKDIR /home/node/app # 2️⃣
  COPY app.js .          # 3️⃣
  EXPOSE 8080            # 4️⃣
  USER node              # 5️⃣
  CMD ["node", "app.js"] # 6️⃣
  

  [!WARNING] Dockerfile doesn’t support a comment that is in the middle of a line.

  • 1️⃣ FROM: Create a new build stage from a base image: Use the official Node.js Docker image from Docker Hub as the base.

    [!NOTE] With Docker, it’s easy to share container image.

    • You don’t need to install Node.js yourself.
    • There are lots of official images, which are maintained by the official teams, community, e.g. The Node.js Docker Team⁠

  • 2️⃣ WORKDIR: Change working directory: Set the working directory for the rest of the image build.

  • 3️⃣ COPY: Copy files and directories: Copy app.js into the Docker image.

  • 4️⃣ EXPOSE: Describe which ports your application is listening on: When someone uses this Docker image, they know which ports they wish to expose.

  • 5️⃣ USER: Set user and group ID: (Instead of the root user), use the node user - created by the Node.js Docker image - to run the app.

  • 6️⃣ CMD: Specify default commands: The default command to be executed by container orchestration tool (Docker, Kubernetes).

    [!IMPORTANT] With containers, you typically do not need to use a process supervisor.

    • The container orchestration tools take care of
      • process supervisor
      • resource usage (CPU, memory…)
      • …

    [!NOTE] Most of container orchestration tools expect your containers to

    • run apps in the “foreground” - blocking until they exit
    • log directly to stdout, stderr

• Build a Docker image for your sample app from a Dockerfile

  docker build -t sample-app:v1 .
  

  • Use -t (--tag) flag to specify the Docker image name & tag in the format name:tag

    For this example:

    • name sample-app²⁰
    • tag v1

    Later on, if you make change to the sample app, you’ll build a new Docker image with:

    • the same name sample-app²¹
    • a different tag e.g. v2, v3

  • The dot (.) at the end tells docker build to run the build in the current directory (which should be the folder that contains your Dockerfile).

• When the build finishes, you can use docker run command to run your new image

  docker run --init sample-app:v1
  

  Listening on port 8080
  

  • Use --init flag to ensure Node.js app will exit correctly if you hit Ctrl+C.

  [!NOTE] Node.js doesn’t handle kernel signals properly, by using --init flag, you wrap your Node.js process with a lightweight init system that properly handles kernel signals, e.g. SIGINT (CTRL-C)

  For more information, see Docker and Node.js best practices

• Your app is "listening on port 8080", let’s try your app

  $ curl localhost:8080
  

  curl: (7) Failed to connect to localhost port 8080: Connection refused
  

  • You still can’t connect to your app. Why?

    • Your app is up and running, but it’s running inside the container, which is isolated from your host OS - not only for the filesystem but also for networking…
      • Your app is listening on port 8080 inside the container., which isn’t accessible from the host OS.

  • If you want to access your app, which is running inside the container, from the host OS:

    • You need to expose the port, which is listening on (by your app) inside the container, to the outside of the container (to your host OS).

• Publish a container’s port [to the host] (with docker run --publish)

  docker run --publish 8081:8080 --init sample-app:v1
  

  Server listening on port 8080
  

  • The port mapping of a container is available via:

    • docker ps output’s PORTS column

      docker ps
      

      CONTAINER ID   IMAGE           COMMAND                  CREATED          STATUS          PORTS                                       NAMES
      ecf2fb27c512   sample-app:v1   "docker-entrypoint.s…"   19 seconds ago   Up 18 seconds   0.0.0.0:8081->8080/tcp, :::8081->8080/tcp   elegant_hofstadter
      

    • docker port

      docker port
      

      8080/tcp -> 0.0.0.0:8081
      8080/tcp -> [::]:8081
      

[!NOTE] There are a different in the order of the container port & the host port:

• When you run a container (docker run) or list containers (docker ps), the perspective is from the host (outside the container):
  • --publish [hostPort:]containerPort
  • 0.0.0.0:hostPort->containerPort/tcp
• When you list the port mappings of a container (docker port), the perspective is from inside the container:
  • containerPort/tcp -> 0.0.0.0:hostPort

• Now you can retry your app:

  curl localhost:8081
  

  Hello, World!
  

[!WARNING] Using docker run is fine for local testing & learning,

• but not for Dockerized apps in production (which typically require a container orchestration tool, e.g. Kubernetes).

[!NOTE] Don’t forget to clean up stopped containers:

• Every time you run docker run and exit, you leave behind stopped container, which take up disk space.

You can clean up stopped containers by:

• Manually run docker rm <CONTAINER_ID>.
• Having docker run automatically do it for you with --rm flag.

A Crash Course on Kubernetes

What is Kubernetes?

Kubernetes (K8s) : a container orchestration tool, that solves almost all orchestration problems for running containers.

Kubernetes consists of 2 main pieces: control plane & worker nodes:

• Control plane 🧠:

  • Responsible for managing the Kubernetes cluster:
    • Storing the states
    • Monitoring containers
    • Coordinating actions across the cluster
  • Runs the API server, which provides an API - to control the cluster - that can be accessed from:
    • CLI tools, e.g. kubectl
    • Web UIs, e.g. Kubernetes dashboard, Headlamp
    • IaC tools, e.g. OpenTofu/Terraform

• Worker nodes 👐:

  • The servers that are used to actually run your containers.
  • Entirely managed by the control plane.

Why Kubernetes?

In additional to solving almost all the orchestration problems for running containers:

• Kubernetes is open source
• Kubernetes can be run anywhere: in the cloud, in your data-center, on your PC.

Run Kubernetes on personal computer

• If you’re using Docker Desktop, you’re just a few clicks away from running a Kubernetes cluster locally:

  • Docker Desktop’s Dashboard / Settings / Kubernetes / Enable Kubernetes / Apply & restart

• After having the running Kubernetes cluster, you need to install kubectl - the CLI tool for managing the cluster:

  • Following the instruction on Kubernetes website to install kubectl.

• Configure the kubeconfig (Kubernetes configuration) to access the Kubernetes cluster.

  [!TIP] If you’re running the Kubernetes cluster via Docker Desktop, the Docker Desktop has already update the config for you.

  • Tell kubectl to use the context that Docker Desktop added

    kubectl config use-context docker-desktop
    

    Switched to context "docker-desktop".
    

    [!NOTE] The kubeconfig can consists of multiple contexts, each context is corresponding to the configuration for a Kubernetes cluster. e.g.

    • The context added by Docker Desktop is named docker-desktop.

    [!NOTE] By default, kubeconfig is at $HOME/.kube/config.

• Check if your Kubernetes is working - e.g. by using get nodes commands:

  kubectl get nodes
  

  NAME             STATUS   ROLES           AGE   VERSION
  docker-desktop   Ready    control-plane   67m   v1.29.2
  

  The Kubernetes cluster created by Docker Desktop has only 1 node, which:

  • runs the control plane
  • also acts as a worker node

How to use Kubernetes?

• To deploy something in Kubernetes:

  • You

    • declare your intent
      • by creating Kubernetes objects
    • record your intent
      • by writing these Kubernetes object to the cluster (via api server)

  • The Kubernetes cluster runs a reconciliation loop, which continuously

    • checks the objects you’ve stored in the it
    • works to make the state of the cluster match your intent.

• There are many types of Kubernetes objects available:

  • To deploy an application, e.g. the sample app, you use Kubernetes Deployment - a declarative way to manage application in Kubernetes:
    • Which Docker images to run
    • How many copies of them to run (replicas)
    • Many settings for those image, e.g. CPU, memory, port numbers, environment variables…

• A typical workflow when using Kubernetes:

  • Create YAML file to define Kubernetes objects
  • Use kubectl apply to submit those objects to the cluster

[!NOTE] Kubernetes: Object & Resource & Configuration & Manifest TODO

Example: Deploy a Dockerized App with Kubernetes

• Create a folder to store the YAML files for the dockerized app

  cd examples
  mkdir -p ch3/kubernetes
  

• Create the Kubernetes Deployment object

  # example/ch3/kubernetes/sample-app-deployment.yml
  apiVersion: apps/v1 #               0️⃣
  kind: Deployment #                  1️⃣
  metadata: #                         2️⃣
    name: sample-app-deployment
  spec:
    replicas: 3 #                     3️⃣
    template: #                       4️⃣
      metadata: #                     5️⃣
        labels:
          app: sample-app-pods
      spec:
        containers: #                 6️⃣
          - name: sample-app #        7️⃣
            image: sample-app:v1 #    8️⃣
            ports:
              - containerPort: 8080 # 9️⃣
            env: #                    10
              - name: NODE_ENV
                value: production
    selector: #                       11
      matchLabels:
        app: sample-app-pods
  

  • 1️⃣ kind: Specify the “kind” of this Kubernetes object.

  • 2️⃣ metadata: Specify the metadata of this Kubernetes object, that can be used to identify & target it in API calls.

    [!NOTE] Kubernetes makes heavy use of metadata (& its labels) to keep the system flexible & loosely coupled.

  • 3️⃣: This Deployment will run 3 replicas.

  • 4️⃣: The pod template - the blueprint - that defines what this Deployment will deploy & manage.

    With pod template, you specify:

    • The containers to run
    • The ports to use
    • The environment variables to set
    • …

    [!TIP] The pod template is similar to the launch template of AWS Auto Scaling Group

    [!NOTE] Instead of deploying one container at a time, in Kubernetes you deploy pods, groups of containers that are meant to be deployed together. e.g.

    • You can deploy a pod with:
      • a container to run a web app, e.g. the sample-app
      • another container that gathers metrics on the web app & send them to a central service, e.g. Datadog.

  • 5️⃣: The pods (deployed & managed by this pod template) have its own metadata (so Kubernetes can also identify & target them).

  • 6️⃣: The containers run inside the pod.

  • 7️⃣: The pod in this example run a single container named sample-app.

  • 8️⃣: The Docker image to run for the sample-app container.

  • 9️⃣: Tells Kubernetes that the Docker image listens for request on port 8080.

    [!NOTE] Isn’t this port already specified in the Dockerfile?

    • The port specified with EXPOSE in the Dockerfile acts a document from the person who builds the image.
    • The person who runs the containers, using that information to run the containers, .e.g.
      • docker run --publish hostPort:containerPort
      • Kubernetes’ Pod spec.containers.[].port.containerPort

  • 10 env: Set the environment for the container.

  • 11 selector: Tells Kubernetes Deployment what to target (which pod to deploy & manage)

    [!NOTE] Why doesn’t Deployment just assume that the pod defined within that Deployment is the one you want to target.

    Because Kubernetes is trying to be flexible & decoupled:

    • The Deployment & the pod template can be defined completely separately.
    • But you always need to specify a selector to tell Kubernetes what to target.

• Use kubectl apply to apply the Deployment configuration

  kubectl apply -f sample-app-deployment.yml
  

• Interact with the deployments

  • Display the deployment

    kubectl get deployments
    

    [!TIP] The field metadata.name’s value is used as the name of the deployment.

  • Show details about the deployment

    kubectl describe deployments <NAME>
    

  • Display the pods

    kubectl get pods
    

  • Show details about the pods

    kubectl describe pods <NAME>
    

  • Print the logs for a container in a pod

    kubectl logs <POD_NAME>
    

Example: Deploy a Load Balancer with Kubernetes

[!NOTE] Kubernetes has a built-in support for load-balancing via Service object.

• The Service object is a way to expose an app running in Kubernetes as a service you can tale over the network.

• Create the Kubernetes Service object

  # examples/ch3/kubernetes/sample-app-service.yml
  apiVersion: v1
  kind: Service #                 1
  metadata: #                     2
    name: sample-app-loadbalancer
  spec:
    type: LoadBalancer #          3
    selector:
      app: sample-app-pods #      4
    ports:
      - protocol: TCP
        port: 80 #                5
        targetPort: 8080 #        6
  

  • 1: Specify that this object is a Kubernetes Service
  • 2: Specify the name of the Service (via the metadata).

  [!NOTE] Every Kubernetes object MUST have the metadata with the name field

  • 3: Configure the Service to be a load balancer.

  [!NOTE] The actual type of load balancer you get will be different, depending on:

  • What sort of Kubernetes cluster you’re running
  • How you configure that cluster

  e.g. If you’re run this code

  • In AWS, you’ll get an AWS ELB
  • In GCP, you’ll get an Cloud Load Balancer
  • Locally, you’ll get a simple load balancer (built into the Kubernetes distribution in Docker Desktop)

  • 4: Distribute traffic across the pods with the label app: sample-app-pods (the pods you defined in previous Deployment)
  • 5: The Service will receive requests on port 80 (the default HTTP port).
  • 6: The Service will forward requests to port 8080 of the pods.

• Use kubectl apply to apply the Service configuration

  kubectl apply -f sample-app-service.yml
  

• Interact with the services

  • Display the service

  kubectl get services
  

  • Show details of the service

  kubectl describe services <NAME>
  

Example: Roll Out Updates with Kubernetes

Just as Ansible, ASG, Kubernetes has built support for rolling updates.

• Add strategy section to sample-app-deployment.yaml

  # example/ch3/kubernetes/sample-app-deployment.yml
  # ...
  spec:
    # ...
    strategy:
      type: RollingUpdate
      rollingUpdate:
        maxSurge: 3 #       1
        maxUnavailable: 0 # 2
  

  • 1: maxSurge: The Deployment can deploy up to 3 extra pods during deployment.
  • 2: maxUnavailable: The Deployment only undeploy an old pod after a new one is deployed.

• Apply the updated Deployment

  kubectl apply -f sample-app-deployment.yml
  

• Make a change to the sample-app (the app.js file)

  sed -i 's/Hello, World!/Fundamentals of DevOps!/g' examples/ch3/docker/app.js
  

• To make a change to a containerized app, you need to build the new image, then deploy that new image:

  • Build a new image (tag sample-app:v2) with the new changes

    docker build -t sample-app:v2
    

  • Update the Deployment to use sample-app:v2 image

    # examples/ch3/kubernetes/sample-app-deployment.yml
    # (...)
    spec:
      # (...)
      spec:
        containers:
          - name: sample-app
            image: sample-app:v2 # Change to the new tag image
    

  • Run kubectl apply to deploy the change:

    kubectl apply -f sample-app-deployment.yml
    

  • Display the pods (to see the rolling updates)

    kubectl get pods
    

Get your hands dirty with Kubernetes and YAML template tools

[!NOTE] Using YAML (and kubectl) is a great way to learn Kubernetes, and it is used in the examples in this chapter to avoid introducing extra tools,

• but raw YAML is not a great choice for production usage.
  • In particular, YAML doesn’t have support for variables, templating, for-loops, conditionals, and other programming language features that allow for code reuse.

When using Kubernetes in production, instead of raw YAML, try out one of the following tools that can solve these gaps for you:

• Helm
• OpenTofu with the Kubernetes provider
• Pulumi with the Kubernetes provider
• Kustomize
• kapp

A Crash Course on AWS Elastic Kubernetes Service (EKS)

Why use a managed Kubernetes service

• Running Kubernetes is great for learning & testing, but not for production.

• For production, you’ll need to run a Kubernetes cluster on servers in a data center:

  • Kubernetes is a complicated system
  • Setting up & maintaining a Kubernetes cluster is a significant undertaking.

• Most cloud providers have managed Kubernetes services that makes setting up & maintaining a Kubernetes cluster a lot easier.

What is EKS

EKS is the manage Kubernetes service from AWS, which can

• deploy & manage
  • the control plane
  • worker nodes

Example: Deploy a Kubernetes Cluster in AWS Using EKS

[!CAUTION] Watch out for snakes: EKS is not part of the AWS free tier!

• While most of the examples in this book are part of the AWS free tier, Amazon EKS is not: as of June 2024, the pricing is $0.10 per hour for the control plane.
• So please be aware that running the examples in this section will cost you a little bit of money.

The eks-cluster OpenTofu module

The sample code repo contains an OpenTofu module named eks-cluster (in ch3/tofu/modules/eks-cluster folder) that can be used to deploy a simple EKS cluster, which includes:

• A fully-managed control plane

• Full-manged worker nodes

  [!NOTE] EKS supports several types of worker nodes:

  • EKS managed node groups
  • Self managed nodes
  • AWS Fargate

  This example uses an EKS manage node group, which deploys worker nodes in an ASG (VM orchestration).

• IAM roles with the minimal permissions required by the control plane & worker nodes

  [!NOTE] An IAM role

  • is similar to an IAM user: it’s an entity in AWS that can be granted IAM permissions.
  • is not associated with any person, and do not have permanent credentials (password, access keys)
    • but can be assumed by other IAM entities, e.g. EKS control plane

  IAM role is a mechhanism for granting services permissions to make certian API calls in AWS account.

• (Everything will be deployed into the Default VPC).

Using the OpenTofu module to deploy an Kubernetes cluster using EKS

• Create the eks-sample OpenTofu module folder

  cd examples
  mkdir -p examples/ch3/tofu/live/eks-sample
  

• Configure the eks-sample module to use the eks-cluster module

  # examples/ch3/tofu/live/eks-sample/main.tf
  
  provider "aws" {
    region = "us-east-2"
  }
  
  module "cluster" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/eks-cluster"
  
    name        = "eks-sample"
    eks_version = "1.29"
  
    instance_type        = "t2.micro"
    min_worker_nodes     = 1
    max_worker_nodes     = 10
    desired_worker_nodes = 3
  }
  

• (Optional) Authenicate to AWS

• Init the OpenTofu module

  tofu init
  

• Apply OpenTofu configuration to create infrastructure (the eks-cluster’s resources)

  tofu apply
  

  • The cluster deployment takes 3-5 minutes

• Interact with your Kubernetes cluster

  • Configure Kubenetes configuration to authenticate to the cluster

    # aws eks update-kubeconfig --region <REGION> --name <CLUSTER_NAME>
    aws eks update-kubeconfig --region us-east-2 --name eks-tofu
    

  • Display the nodes

  kubectl get nodes
  

Example: Push a Docker Image to AWS Elastic Container Registry (ECR)

Container registry and ECR

If you want to deploy your sample-app to the EKS cluster, the Docker image for the sample-app need to be pushed to a container registry that EKS can read from.

There are lots of container registries:

• Docker Hub
• AWS Elastic Container Registry (ECR)
• Azure Container Registry
• Google Artifact Registry
• JFrog Docker Registry
• GitHub Container Registry.

You’ve used AWS for the examples, so ECR is the easiest option.

• For each Docker image you want to store in ECE, you have to create an ECR repository (ECR repo).

• The book’s sample code repo includes a module called ecr-repo (in ch3/tofu/modules/ecr-repo folder) that you can use to create an ECR repo.

Using ecr-repo OpenTofu module to create an ECR repo

• Create the ecr-sample OpenTofu module folder

  cd examples
  mkdir -p examples/ch3/tofu/live/ecr-sample
  

• Configure the ecr-sample module to use the eks-repo module

  • main.tf

    # examples/ch3/tofu/live/ecr-sample/main.tf
    provider "aws" {
      region = "us-east-2"
    }
    
    module "repo" {
      source = "github.com/brikis98/devops-book//ch3/tofu/modules/ecr-repo"
    
      name = "sample-app"
    }
    

  • output.tf

    # examples/ch3/tofu/live/ecr-sample/outputs.tf
    output "registry_url" {
      value       = module.repo.registry_url
      description = "URL of the ECR repo"
    }
    

• Init the OpenTofu module

  tofu init
  

• Apply OpenTofu configuration to create infrastructure (the ecr-repo’s resources)

  tofu apply
  

[!NOTE] By default, docker build command builds the Docker image for whatever CPU architecture that it’s running on.

e.g.

• On a Macbook with ARM CPU (M1, M2…), the Docker image is built for arm64 architecture.
• On a PC running Linux, it’s for amd64 architecture.

[!NOTE] You need to ensure that you build your Docker images for whatever architecture(s) you plan to deploy on.

• Docker now ships with the buildx command which makes it easy to build Docker images for multiple architecture.

• (The very first time you use buildx) Create a builder named multi-platform-builder for your target architectures:

  docker buildx create \
    --use \                              # Set the current builder instance
    --platform=linux/amd64,linux/arm64 \ # Fixed platforms for current node
    --name=multiple-platform-build       # Builder instance name
  

• Use the multiple-platform-build builder to build Docker image sample-app:v3 for multiple platforms

  docker buildx build \
    --platform=linux/amd64,linux/arm64 \
    -t sample-app:v3 \
    .
  

• Re-tag the image using the registry URL of the ECR repo (registry_url)

  docker tag \
    sample-app:v3 \
    <YOUR_ECR_REPO_URL>:v3
  

• Authenticate docker to the ECR repo:

  aws ecr get-login-password --region us-east-2 | \
    docker login \
      --username AWS \
      --password-stdin \
      <YOUR_ECR_REPO_URL>
  

• Push Docker image to your ECR repo

  docker push <YOUR_ECR_REPO_URL>:v3
  

  [!TIP] The first time you push, it may take longer than a minute to update the image.

  Subsequent pushes - due to Docker’s layer caching - will be faster.

Example: Deploy a Dockerized App into an EKS Cluster (With Load Balancer)

After having the sample-app Docker image on your ECR repo, you’re ready to deploy the sample-app to EKS cluster:

• Update the Deployment to use the Docker image from your ECR repo

  # examples/ch3/kubernetes/sample-app-deployment.yml
  # (...)
  spec:
    # (...)
    spec:
      containers:
        - name: sample-app
          image: <YOUR_ECR_REPO_URL>:v3
  

• Apply both the Kubernetes object into your EKS cluster:

  kubectl apply -f sample-app-deployment.yml
  kubectl apply -f sample-app-service.yml
  

• Interact with Kubernetes cluster on EKS (and your app)

  • Display the pods

    kubectl get pods
    

  • Display the service

    kubectl get services
    

    • The sample-app-loadbalancer has an EXTERNAL-IP of the domain name of an AWS ELB.

    [!TIP] The EXTERNAL-IP column is showing the domain name, isn’t it weird?

Get your hands dirty with Kubernetes and container orchestration

1. By default, if you deploy a Kubernetes Service of type LoadBalancer into EKS, EKS will create a Classic Load Balancer, which is an older type of load balancer that is not generally recommended anymore.

   • In most cases, you actually want an Application Load Balancer (ALB), as you saw in the VM orchestration section.
   • To deploy an ALB, you need to make a few changes, as explained in the AWS documentation.

2. Try terminating one of the worker node instances using the AWS Console.

   • How does the ELB handle it?
   • How does EKS respond?
   • Do you need to do anything to restore the instance or your containers?

3. Try using kubectl exec to get a shell (like an SSH session) into a running container.

Serverless Orchestration

What is Serverless?

serverless : focus entirely on your app : - without having to think about servers at all : - (the servers are fully managed by someone not you)

How Serverless works?

The origiral model referred to as “serverless” was Functions as a Service (FaaS), which works as follows:

1. Create a deployment package, which contains just the source code to run a function (instead of the whole app).

2. Upload that deployment package to your serverless provider, which is typically also a cloud provider, e.g. AWS, GCP, Azure.

   [!NOTE] You can use tools like Knative to add support for serverless in your on-prem Kubernetes cluster.

3. Configure the serverless provider to trigger your function in response to certain events, e.g. an HTTP request, a file upload, a new message in a queue.

4. When the trigger goes off, the serverless provider:

   • Execute your function
   • Passing it information about the event as an input
   • (In some case), taking the data the function returns as an output; and passing it on elsewhere (e.g. sending it as an HTTP response).

5. When you need to deploy an update, repeat step 1 and 2: create a new deployment package; upload that deployment package to the cloud provider.

Serverless pros and cons

• Pros:

• Cons:

Type of serverless

Nowaday, serverless has become so popular, the term “serverless” is being applied to many models:

• Serverless functions - FaaS (the original model of serverless), e.g. AWS Lambda (2015), GCP Cloud Functions, Azure Serverless
• “Serverless web-app”, e.g. Google App Engine (GAE - 2008)
• Serverless containers, e.g. AWS Fargate.
• Serverless database, e.g. Amazon Aurora Serverless.

[!IMPORTANT] Key takeaway #4 Serverless orchestration is an immutable infrastructure approach where you deploy and manage functions without having to think about servers at all.

Example: Deploy a Serverless Function with AWS Lambda

The lambda OpenTofu module

The book sample code repo includes an OpenTofu module named lambda (in ch3/tofu/modules/lambda) that do the following:

• Zip up a folder - you specify - into a deployment package.
• Upload the deployment package as an AWS Lambda function.
• Configure various settings for the Lambda function, e.g. memory, CPU, environment variables.

Using the lambda OpenTofu module to deploy a AWS Lambda function

• Create folder live/lambda-sample to use as a root module

  cd examples
  mkdir -p ch3/tofu/live/lambda-sample
  cd ch3/tofu/live/lambda-sample
  

• Configure the lambda module

  # examples/ch3/tofu/live/lambda-sample/main.tf
  provider "aws" {
    region = "us-east-2"
  }
  
  module "function" {
    name = "lambda-sample" #         1
  
    src_dir = "${path.module}/src" # 2
    runtime = "nodejs20.x" #         3
    handler = "index.handler" #      4
  
    memory_size = 128 #              5
    timeout     = 5 #                6
  
    environment_variables = { #      7
      NODE_ENV = "production"
    }
  
    # ... (other params omitted) ...
  }
  

  • 1 name: Base name of all resources of the lambda module

  • 2 src_dir: The directory which contains the code for the Lambda function.

  • 3 runtime: The runtime used this Lambda function.

    [!NOTE] AWS Lambda supports

    • several different runtimes: Node.js, Python, Go, Java, .NET.
    • create custom runtimes for any languague

  • 4 handler: The handler to call your function, aka entrypoint.

    [!NOTE] The handler format is <FILE>.<FUNCTION>:

    • <FILE>: The file in your deployment package.
    • <FUNCTION>: The name of the function to call in that file.

    Lambda will pass the event information to this function.

    For this example, Lambda will call the hanlder function the index.js file.

  • 5 memory_size: The amount of memory to give the Lambda function.

    [!NOTE] Adding more memory also proportionally increases:

    • the amount of CPU available
    • the cost to run the function.

  • 6 timeout: The maximum amount of time the Lambda function has to run.

    [!NOTE] The timeout limit of Lambda is 15 minutes.

  • 7 environment_variables: The environment variables to set for the function.

• Add the handler code at lambda-sample/src/index.js

  # examples/ch3/tofu/live/lambda-sample/src/index.js
  exports.handler = (event, context, callback) => {
    callback(null, {statusCode: 200, body: "Hello, World!"});
  };
  

• Init & apply the OpenTofu module

  tofu init
  tofu apply
  

• Verify that the Lambda function has been deployed by:

  • Open the Lambda console
  • Click on the function called sample-app-lambda
  • You should see your Lambda function & handler code.
  • Currently, the function has no triggers:
    • You can manually trigger it by clicking the Test button.

[!NOTE] In this example, you deploy a Lambda function without a trigger, which isn’t very useful.

• Because the function cannot be triggered by anything or anyone except you.

A Crash course on AWS Lambda triggers

You can configure a variety of events to trigger your Lambda function.

You can have AWS automatically run your Lambda function:

• each time a file is uploaded to Amazon’s Simple Storage Service (S3),

• each time a new message is written to a queue in Amazon’s Simple Queue Service (SQS),

• each time you get a new email in Amazon’s Simple Email Service (SES)

  [!NOTE] AWS Lambda is a great choice of building event-driven systems and background processing jobs.

• each time you receive an HTTP request in API Gateway

  [!NOTE] API Gateway is a managed service you can use to expose an entrypoint for your apps, managing routing, authentication, throttling, and so on. You can use API Gateway and Lambda to create serverless web apps.

Example: Deploy an API Gateway in Front of AWS Lambda

The api-gateway OpenTofu module

The book’s sample code repo includes a module called api-gateway in the ch3/tofu/modules/api-gateway folder that can deploy an HTTP API Gateway, a version of API Gateway designed for simple HTTP APIs, that knows how to trigger a Lambda function.

Using api-gateway OpenTofu module to deploy an API Gateway in Front of AWS Lambda

• Configure the api-gateway module to trigger the Lambda function

  # examples/ch3/tofu/live/lambda-sample/main.tf
  
  module "function" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/lambda"
    # ... (other params omitted) ...
  }
  
  module "gateway" {
    source = "github.com/brikis98/devops-book//ch3/tofu/modules/api-gateway"
  
    name               = "lambda-sample" #              1
    function_arn       = module.function.function_arn # 2
    api_gateway_routes = ["GET /"] #                    3
  }
  

  • 1 name: The base name to use for the api-gateway’s resources.

  • 2 function_arn: The ARN of the Lambda function the API Gateway should tirgger when it gets HTTP requests.

    In this example, function_arn is set to the output from the lambda module.

  • 3 api_gateway_routes: The routes that should trigger the Lambda function

    In this example, the API Gateway has only 1 route: HTTP GET to / path.

• Add an output variable

  # examples/ch3/tofu/live/lambda-sample/outputs.tf
  
  output "api_endpoint" {
    value = module.gateway.api_endpoint
  }
  

• Init & apply OpenTofu configuration

  tofu init
  tofu apply
  

• Your API Gateway is now routing requests to your Lambda function.

  • As load goes up & down,
    • AWS will automatically scale your Lambda functions up & down.
    • API Gateway will automatically distribute traffic across these functions/
  • When there’no load:
    • AWS will automatically scale to zero. So it won’t cost you a cent.

Example: Roll Out Updates with AWS Lambda

[!NOTE] By default, AWS Lambda natively supports a nearly instantaneous deployment model:

• If you upload a new deployment package, all new requests will start executing the code in that deployment package almost immediately.

• Update the Lambda function response text

  // examples/ch3/tofu/live/lambda-sample/src/index.js
  exports.handler = (event, context, callback) => {
    callback(null, { statusCode: 200, body: "Fundamentals of DevOps!" });
  };
  

• Rerun apply to deploy the changes

  tofu apply
  

[!TIP] Under the hood, AWS Lambda does an instanteneous switchover from the old to the new version (~ blue-green deployment).

Get your hands dirty with serverless web-apps and Serverless Orchestration

[!NOTE] To avoid introducing too many new tools, this chapter uses OpenTofu to deploy Lambda functions

• which works great for functions used for background jobs and event processing,
• but for serverless web apps where you use a mix of Lambda functions and API Gateway, the OpenTofu code can get very verbose (especially the API Gateway parts).
  • Moreover, if you’re using OpenTofu to manage a serverless webapp, you have no easy way to run or test that webapp (especially the API Gateway endpoints) locally.

If you’re going to be building serverless web apps for production use cases, try out one of the following tools instead, as they are purpose-built for serverless web apps, keep the code more concise, and give you ways to test locally:

• Serverless Framework
• SAM

Comparing Orchestration Options

In terms of the core orchestration problems

For more information,see Orchestration - Core Problems Comparison

In terms of core orchestration attributes

For more information, see Orchestration - Attributes Comparison

Conclusion

• You learn how to run your apps in a way that more closely handles the demand of production (“in a scalable way”):

  • ⛓️‍💥 avoid a single point of failure

    • by using multiple replicas

  • ☦️ distribute traffic across the replicas

    • by deploying load balancers

  • 🎢 roll out updates to your replicas without downtime 🔛

    • by using deployment strategies

• You’ve seen a number of orchestration approaches to handle all of the above problems:

  Orchestration approach	…infrastructure approach	How it works?	Example
  Server orchestration	Mutable … (Old way)	A fixed set of servers are maintained, updated in place.	Deploy code onto a cluster of servers (using Ansible)
  VM …	Immutable …	VM images are deployed & managed across virtualized servers.	Deploy VMs into an Auto Scaling Group.
  Container …	Immutable …	Containers images are deployed & managed across a cluster of servers.	Deploy containers into a Kubernetes cluster.
  Serverless …	Immutable …	Functions are deploy & managed without thinking about servers at all.	Deploy functions using AWS Lambda.

Chapter 4: How to Version, Build, and Test Your Code

With most real-world code, software development is a team sport, not a solo effort.

You need to figure out how to support many developers collaborating safety and efficiently on the same codebase.

In particular, you need to solve the following problems:

These problems are all key part of your software development life cycle (SDLC).

• In the pasts, many companies came up with their own ad-hoc, manual SDLC processes:
  • Email code changes back & forth
  • Spend weeks integrating changes together manually
  • Test everything manually (if they did any testing at all)
  • Release everything manually, e.g. Using FTP to upload code to a server
• Now a day, we have far better tools & techniques for solving these problems:
  • Version control
  • Build system
  • Automated testing

Version Control

What is Version Control

version control system (VCS) : a tool that allows you to : - store source code : - share it with your team : - integrate your work together : - track changes over time

Version Control Primer

Visualizing your process with Word documents as version control

Your normal workflow with the an essay - copying, renaming, emailing… - is a type of version control, but not a manual version control system.

There are better version control tools, in which you commit, branch, merge… your works.

Version Control Concepts

Repositories : You store files (code, documents, images, etc.) in a repository (repo for short).

Branches : (You start with everything in a single branch, often called something like main.) : At any time, you can create a new branch from any existing branch, and work in your own branch independently.

Commits : Within any branch, : - you can edit files, : - when you’re ready to store your progress in a new revision, you create a commit with your updates (you commit your changes) : The commit typically records : - not only the changes to the files, : - but also who made the changes, and a commit message that describes the changes.

Merges : At any time, you can merge branches together. : e.g. : - It’s common to create a branch from main, work in that branch for a while, and then merge your changes back into main.

Conflicts : (VCS tools can merge some types of changes completely automatically), : But if there is a conflict (e.g., two people changed the same line of code in different ways), : - the VCS will ask you to resolve the conflict manually.

History : The VCS tracks every commit in every branch in a commit log, which lets you see : the full history of how the code changed: : - all previous revisions of every file, : - what changed between each revision : - who made each change.

[!NOTE] There are many version control systems:

• CVS, Subversion, Perforce…
• Mercurial, Git…

These days, the most popular is Git.

Example: A Crash Course on Git

Git basics

• Install Git: Follow the office guide

• Let Git know your name & email

  git config --global user.name "<YOUR NAME>"
  git config --global user.email "<YOUR EMAIL>"
  

• Initialize a Git repo

  [!NOTE] Before initial a Git repo, you need to create a empty folder:

  mkdir /tmp/git-practice
  cd /tmp/git-practice
  

  (or you can use an existing repo)

  git init
  

  [!NOTE] Now, create a text file that will be including in your first commit:

  echo 'Hello, World!' > example.txt
  

  [!TIP] The contexts of the git-practice folder looks like this

  $ tree -aL 1
  .
  ├── .git
  └── example.txt
  

  [!NOTE] The .git folder is where Git record all information about your branches, commits, revisions…

• Show the working tree status

  git status
  

  [!NOTE] The git status show the working tree status:

  • What branch you’re on.
  • Any commits you’ve made.
  • Any changes that haven’t been committed yet.

• Before commit your changes, you first need to add the file(s) you want to commit to the staging area using git add

  git add example.txt
  

• Re-run git status

  git status
  

  On branch main
  
  No commits yet
  
  Changes to be committed:
    (use "git rm --cached <file>..." to un-stage)
  	new file:   example.txt
  

  • The example.txt is now in the staging area, ready to be committed.

• To commit the staged changes, use the git commit

  git commit -m "Initial commit"
  

  • Use the -m flag to pass in the commit message.

• Check the commit log using git log

  git log
  

  For each commit in the log, you’ll see

  • commit ID
  • author
  • date
  • commit message.

  [!NOTE] Each commit has a different ID that you can use to uniquely identify that commit, and many Git commands take a commit ID as an argument.

  [!TIP] Under the hood, a commit ID is calculated by taking the SHA-1 hash of:

  • the contents of the commit,
  • all the commit metadata (author, date, and so on), and
  • the ID of the previous commit

  [!TIP] Commit IDs are 40 characters long,

  • but in most commands, you can use just
    • the first 7 characters, as that will be unique enough to identify commits in all but the largest repos.

Let’s make another change and another commit:

• Make a change to the example.txt

  echo 'New line of text' >> example.txt
  

• Show your working tree status

  git status
  

  On branch main
  Changes not staged for commit:
    (use "git add <file>..." to update what will be committed)
    (use "git restore <file>..." to discard changes in working directory)
  	modified:   example.txt
  

  • Git is telling you that the changes is current “in working directory” (working tree), and is “not staged for commit”
  • Git also tell you the changed files: modified: example.txt

• To see what exactly these changes are, run git diff

  git diff
  

  $ git diff
  diff --git a/example.txt b/example.txt
  index 8ab686e..3cee8ec 100644
  --- a/example.txt
  +++ b/example.txt
  @@ -1 +1,2 @@
   Hello, World!
  +New line of text
  

  [!NOTE] Use git diff frequently to check what changes you’ve made before committing them:

  • If the changes look good:
    • Use git add <file>... to stage the changes.
    • Then use git commit to commit them.
  • If the changes don’t look good:
    • Continue to modify the changes
    • Or use "git restore <file>..." to discard changes in working directory.

• Re-stage the changes and commit:

  git add example.txt
  git commit -m "Add another line to example.txt"
  

• Use git log once more with --oneline:

  git log --oneline
  

  02897ae (HEAD -> main) Add another line to example.txt
  0da69c2 Initial commit
  

[!IMPORTANT] The commit log is very powerful 👈 It’s has the commit IDs and commit messages:

• Debugging:

  Something breaks -> “What changed?” -> Check commit log’s messages

• Reverting:

  • You can use git revert <COMMIT_ID> to create a new commit that reverts all the changes in the commit <COMMIT_ID>.

    (in other words, undoing the changes in that commit while still preserving your Git history)

  • You yan use git reset --hard <COMMIT_ID> to get rid of:

    • all commits after COMMIT_ID.
    • including the history about them.

• Comparison:

  You can use git diff to compare not only local changes, but also to compare any two commits.

• Author:

  You can use git blame to annotate each line of a file with information about the last commit that modified that file, (including the date, the commit message, and the author).

  • Don’t use this to blame someone for causing a bug, as the name implies. It may be war!
  • The more common use case is to help you understand where any give piece of code came from, and why that change was made.

Git branching and merging

• To create a new branch and switch to it, use git checkout -b

  git checkout -b testing
  

  [!NOTE] If you want to make sure you never lost your code, you can use git switch -c to create a new branch and switch to it.

• Check the you’re on new branch with git status

  git status
  

• You can also list all the branches (and see which one you’re on) with git branch

  git branch
  

  [!TIP] The branch which you’re on is mark with asterisk (*)

• Any changes you commit now will go into the testing branch:

  • Try it with the example.txt

    echo 'Third line of text' >> example.txt
    

  • Stage and commit the changes

    git add example.txt
    git commit -m "Added a 3tr line to example.txt"
    

  • You git log to check that you have three commits on testing branch:

    git log --oneline
    

• Switch back to main branch to see that main branch still has only 2 commits

  git switch main
  git log --oneline
  

• Merge the work in your testing branch back to the main branch

  git merge testing # Merge testing branch (to current branch - main)
  

  Updating c4ff96d..c85c2bf
  Fast-forward
   example.txt | 1 +
   1 file changed, 1 insertion(+)
  

  • It’s a Fast-forward, Git was able to merge all the changes automatically, as there were no conflicts between main & testing branches.

Get your hands dirty with Git

• Learn how to use the git tag command to create tags.

• Learn to use git rebase.

  • When does it make sense to use it instead of git merge?

Example: Store your Code in GitHub

• Git is a distributed VSC:

  Every team member can

  • have a full copy of the repository.
  • do commits, merges, branches completely locally.

• But the most common way to use Git is using one of the repositories as a central repository, which acts as your source of truth.

  • Everyone will initially get their code from this central repo
  • As someone make changes, he/she always pushes them back to this central repo.

• There are many way to run such a central repo:

  • Hosting yourself

  • Use a hosting service, which is the most common approach:

    • Not only host Git repos
    • But also provide:
      • Web UIs
      • User management
      • Development workflows, issue tracking, security tools…

    The most popular hosting service for Git are GitHub, GitLab, BitBucket.

  [!NOTE] GitHub is the most popular, and what made Git popular.

  • GitHub provides a great experience for hosting repos & collaboration with team members.
  • GitHub has become de facto home for most open source projects.

In this example, you will push the example code you’ve worked in while reading this book/blog post series to GitHub.

• Go the folder where you have your code

  cd devops-books
  

• The contents of the folder should look like this:

  tree -L 2
  

  .
  ├── ch1
  │   ├── ec2-user-data-script
  │   └── sample-app
  ├── ch2
  │   ├── ansible
  │   ├── bash
  │   ├── packer
  │   └── tofu
  └── ch3
      ├── ansible
      ├── docker
      ├── kubernetes
      ├── packer
      └── tofu
  

• Initialize an empty Git repository in .git/

  git init
  

• Show working tree status

  git status
  

  • There is “no commits yet”, and only “untracked files”.

• Create gitignore file (.gitignore)

  *.tfstate            # 1
  *.tfstate.backup
  *.tfstate.lock.info
  
  .terraform           # 2
  
  *.key                # 3
  
  *.zip                # 4
  
  node_modules         # 5
  coverage
  

  • 1: Ignore OpenTofu state.
  • 2: Ignore .terraform, used by OpenTofu as a scratch directory.
  • 3: Ignore the SSH private keys used in Ansible examples.
  • 4: Ignore build artifact created by lambda module.
  • 5: Ignore Node.js’s scratch directories.

  [!TIP] Commit the .gitignore file first to ensure you don’t accidentally commit files that don’t belong in version control.

• Stage and commit .gitignore

  git add .gitignore
  git commit -m "Add .gitignore"
  

• Stage all files/folders in root of the repo:

  git add .
  git commit -m "Example for first few chapters"
  

  • The code in now in a local Git repo in your computer.
  • In the next section, you’ll push it to a Git repo on GitHub

• Create a GitHub account if you haven’t one

• Authenticate to GitHub on the CLI: Follow the official docs

• Create a new repository in GitHub

• Add that GitHub repository as a remote to your local Git repository:

  [!NOTE] A remote is a Git repository hosted somewhere, i.e. somewhere on the Internet

  git remote add origin https://github.com/<USERNAME>/<REPO>.git
  

  • This will add your GitHub repo as a remote named origin

  [!TIP] Your remote GitHub repo can be any where, but anyone that access your repo, which now acts as a central repository can refer to it as origin.

• Push your local branch to your GitHub repo

  git push origin main
  

  [!TIP] You push to REMOTE a LOCAL_BRANCH with:

  git push REMOTE LOCAL_BRANCH
  

• Refresh your repo in GitHub, you should see your code there.

[!NOTE] You’ve just push your changes to a remote endpoint, which being halfway to be able to collaborate with other developers.

• You can click the Add a README button, then:

  • Fill in the README content.
  • And commit changes directly to the Git repo.

• If you do that, your GitHub repo now has a README.md file, but the local repo on your computer doesn’t.

• To get the latest code from the origin, use git pull:

  git pull origin main
  

  [!NOTE] The command git pull REMOTE REMOTE_BRANCH will:

  • “Fetch” from REMOTE the REMOTE_BRANCH.
  • Merge that REMOTE_BRANCH to current branch (in the local repository).

• If your haven’t have a local copy of the central repository, first you need to clone that repo:

  git clone https://github.com/<USERNAME>/<REPO>
  

  This command will

  • checkout a copy of the repo <REPO> to a folder called <REPO> in your current working directory.
  • automatically add the repo’s URL as a remote named origin

You’ve just seen the basic Git workflows when collaboration:

• git clone: Check out a fresh copy of a repo.
• git push origin <LOCAL_BRANCH>: Share your changes to other team members.
• git pull origin <REMOTE_BRANCH>: Get changes from other team members.

A Primer on Pull Request

pull request : a request to merge one branch into another branch : ~ you’re requesting the owner runs git pull on your repo/branch

[!TIP] GitHub popularized the PR workflow as the de facto way to make changes to open source repos

And these days, many companies use PRs to make changes to private repos as well.

A pull request processes is as a follows:

• You check out a copy of repo R, create a branch B, and commit your changes to this branch.

  • If you have write access to repo R, you can create branch B directly in repo R.
  • However, if you don’t have write access, which is usually the case if repo R is an open source repo in someone else’s account, then you
    • first create a fork of repo R, which is a copy of the repo in your own account,
    • then you create branch B in your fork.

• When you’re done with your work in branch B, you open a pull request against repo R:

  • Requesting that the maintainer of that repo merges your changes from branch B into some branch in repo R (typically main).

• The owner of repo R then

  • uses GitHub’s PR UI to review your changes,
  • provide comments and feedback,
  • and ultimately, decide to either
    • merge the changes in,
    • or close the PR unmerged.

Example: Open a Pull Request in GitHub

• Create a new branch named update-readme and switch to it

  git switch -c update-readme
  

• Make a change to the README.md file

  echo "https://www.fundamentals-of-devops.com/" >> README.md
  

• Show un-staged changed

  git diff
  

• Stage & commit the changes

  git add README.md
  git commit -m "Add URL to README"
  

• Push your update-readme branch to the origin remote

  git push origin update-readme
  

  [!TIP] In the git push output, GitHub conveniently shows you a URL for creating a pull request.

  You can also create PRs by

  • going to the Pull Requests tab of your repo in GitHub Web UI
  • clicking New Pull Request button.

• Open the URL in a web browser, then

  • Fill in the pull request’s title, description.
  • Scroll down to see the changes between your update-readme & main branches.
  • If those changes look OK, click Create pull request button.
  • You’ll end up in the GitHub PR UI.

• You and your team members cana use the Github PR page to

  • see the changes
  • discuss the changes
  • request reviewers, modifies to those changes…

• If the PR looks gook:

  • Click Merge pull request
  • Then Confirm merge to merge the changes in.

Version Control Best Practices

• Always use version control

  • Using version control brings massive benefits for software engineering.
  • Version control’s easy, cheap/free.

  [!IMPORTANT] Key takeaway #1 Always manage your code with a version control system.

• Write good commit messages

  When you’re trying to figure out what caused a bug, an outage, git log and git blame can help you, but only if the commit message are well written.

  [!NOTE] What is a good commit message?

  • Summary: Short, clear summary of the change (< 50 characters).
  • Context:
    • If you need more than a summary, put a new line after the summary, then provide more information to understand the context.
    • Focus on what changed; why it changed (How it changed should be clear from the the code itself).

  e.g.

  Fix bug with search auto complete
  
  A more detailed explanation of the fix, if necessary. Provide
  additional context that may not be obvious from just reading the
  code.
  
  - Use bullet points
  - If appropriate
  
  Fixes #123. Jira #456.
  

  [!TIP] You can go a little further with the commit messages by:

  • Following How to Write a Good Commit Message
  • Adopting Conventional Commits

• Commit early and often

  Committing as you’re solving a large problem, break it down to small, manageable parts.

  [!NOTE] What to commit and PR?

  Atomic commit/PR.

  In other words, each commit or pull request should do exactly one small, relatively self-contained thing.

  [!TIP] Atomic commit: You should be able to describe the commit in one short sentence and use it as the commit message’s summary.

  e.g. Instead of a single, massive commit that implements an entire large feature,

  • aim for a series of small commits, where each one implements some logical part of that feature:
    • a commit for backend logic
    • a commit for UI logic
    • a commit for search logic

  [!TIP] Atomic PR:

  • A single PR can contain multiple commits, but it should still represent a single set of cohesive changes - changes that naturally & logically go together.
  • If your PR contains unrelated changes, you should break it up into multiple PRs.

  e.g. Follow the Boy Scout Rule¹ is a good idea, but

  • don’t make a PR that contains a new feature, a bug fix, and a refactor
    • put each of these changes into its own PR:
      • a PR for the new feature
      • a PR for the bug fix
      • a PR for the refactor

  [!NOTE] What is the benefit of atomic commits, PRs?

  Benefit	Description
  More useful Git history	Each commit/PR can fit in oneline in the history.
  Cleaner mental model	Force you to break the work down.
  Less risk	Easy to revert.
  Easier code reviews	Quick to approve.
  Less risky refactors	You can try something new then go back to any commits quickly without losing much work.
  Lower risk of data loss	Commit (and push) act as a data backup.
  More frequent integration	Quick to merge, release.

• Use a code review process

  [!NOTE] Why any one should have their code review?

  In the writing world, even if you’re the smarted, most capable, most experienced, you can’t proofread your own work:

  • You’re too close to the concept.
  • You can’t put yourself in the shoes of someone who is hearing them for the first time.

  The same applies for writing code.

  [!TIP] Having your code review by someone else is a highly effective way to catch bugs, reducing defect rates by as much as 55-80% - which is even a higher rate than automated test.

  [!NOTE] Code reviews are also an efficient mechanism to

  • spread knowledge, culture, training
  • provide a sense of ownership throughout the team

  [!NOTE] How to do code reviews?

  • Enforce a pull request workflow

    You can enforce that

    • all changes are done through pull requests
      • so the maintainers of each repo can asynchronously review each change before it gets merged.

  • Use pair programming

    Pair programming:

    • a development technique where two programmers work together at one computer:

      • one person as the driver, responsible for writing the code
      • the other as the observer, responsible for
        • reviewing the code and
        • thinking about the program at a higher level

      (the programmers regularly switch roles)

    • results in code review process happens all the time:

      • driver will also try to make clear what the code is doing

    Pair programming is used:

    • by some companies for all their coding
    • by other companies for only complex tasks, or ramping up a new hire.

  • Use formal inspections

    Formal inspection is when you schedule a live meeting for a code review where you:

    • present the code to multiple developers
      • go through it together, line-by-line.

    Formal inspection can be apply for mission critical parts of your systems.

  [!TIP] Whatever process you pick for code reviews, you should

  • define your code preview guidelines up front,
    • so everyone can have a process that is consistent & repeatable across the entire team:
      • what to look for, e.g. design, functionality, complexity, tests.
      • what not to look for, e.g. code formatting (should be automated)
      • how to communicate feedback effectively

  For example, have a look at Google’s Code Review Guidelines.

• Protect your code:

  For many companies these day, the code you write is:

  • your most important asset.
  • a highly sensitive asset: if someone can slip malicious code into the codebase, it would be a nightmare.

  [!NOTE] How to protect your code?

  • Signed commits:

    By default, any one can set the email used by Git to any email they want.

    • What if a bad actor introduces some malicious code in your name (email).

    • Fortunately, most VSC hosts (GitHub, GitLab…) allow you to enforce signed commits on your repos, where they reject any commit that doesn’t have a valid cryptographic signature.

      Under the hood:

      • You give Git the private key; and give the VSC host the public key.
      • When you commit, Git will sign that your commits with the private key.
      • When you push to central repo on VSC host, VSC host will use the public key to verify that these commit are signed by your private key.

  • Branch protection:

    Most VCS hosts (GitHub, GitLab, etc.) allow you to enable branch protection, where you can

    • enforce certain requirements before code can be pushed to certain branches (e.g., main)

    For example, you can require that all changes to main branch:

    • Submitted via pull requests
    • Those pull requests are review by at least N other developers.
    • Certain checks - e.g. security scans - pass

    before these pull requests can be merged.

Get your hands dirty with Git amend, squash

Build System

What is Build System?

build system (build tools) : the system used by most software project to automate important operations, e.g. : - Compiling code : - Downloading dependencies : - Packaging the app : - Running automated tests…

Why use Build System?

The build system serves 2 audiences:

• The developers on your team, who run the build steps as part of local development.
• The automated tools (scripts), which run the build steps as part of automating your software delivery process.

Which Build System to use?

You can:

• create your own build system from ad-hoc scripts, duct tape & glue.
• or use an off-the-shelf build system.

There are many off-the-shelf build systems out there:

• Some were originally designed for use with a specific programming language, framework. e.g
  • Rake for Ruby
  • Gradle, Mavan for Java
  • SBT for Scale
  • NPM for JavaScript (Node.js)
• Some are language agnostic:
  • Make: granddad of all build systems.
  • Bazel: fast, scalable, multi-language and extensible build system.

[!TIP] Usually, the language-specific tools will give you the best experience with that language.

You should only go with the language-agnostic ones in specific circumstances, such as:

• Massive teams
• Dozens of languages
• Gigantic monorepo

[!IMPORTANT] Key takeaway #2 Use a build system to capture, as code, important operations and knowledge for your project, in a way that can be used both by developers and automated tools.

Example: Configure your Build Using NPM

The example-app is written is JavaScript (Node,js), so NPM is a good choice for build system.

• The code for this example will be in examples/ch4/sample-app

  cd examples
  mkdir -p ch4/sample-app
  

• Clone the app.js from previous example

  cp ch1/sample-app/app.js ch4/sample-app/app.js
  

• Install Node.js which comes with NPM

• To use NPM as a build system, you need a package.json file.

  [!NOTE] The package.json file can be

  • created manually
  • scaffold by running npm init

  In this example, you will run npm init

  npm init
  # npm will prompt you for the package name, version, description...
  

  You should have a package.json file looks like this:

  {
    "name": "sample-app",
    "version": "1.0.0",
    "description": "Sample app for 'Fundamentals of DevOps and Software Delivery'",
    "scripts": {
      "test": "echo \"Error: no test specified\" && exit 1"
    }
  }
  

  [!NOTE] NPM has a number of built-in scripts, such as npm install, npm start, npm test, and so on.

  All of these have default behaviors, but in most cases, you can define what these script do by

  • adding them to the scripts block.
  • specify which commands that script should run.

  For example

  • npm init gives you an initial test script in the scripts block that just run a command that exits with an error.

• Add a script named start to the script block in package.json

  {
    "scripts": {
      "start": "node app.js"
    }
  }
  

• Now you run the npm start script to run your app.

  npm start
  

  [!NOTE] By using npm start to run your app, you’re using a well-known convention:

  • Most Node.js and NPM users know to use npm start on a project.
  • Most tools that work with Node.js know to use npm start to start a Node.js app.

  In other words, you capture how to run your app in the build system.

• Create a Dockerfile

  # examples/ch4/sample-app/Dockerfile
  FROM node:21.7
  
  WORKDIR /home/node/app
  
  # 1
  COPY package.json .
  COPY app.js .
  
  EXPOSE 8080
  
  USER node
  
  # 2
  CMD ["npm", "start"]
  

  This Dockerfile is identical to the one in previous example, except:

  • 1: In addition to app.js, you also copy the package.json to the Docker image.
  • 2: Instead of using node app.js, you use npm start to start the app.

• Create a script called build-docker-image.sh

  # examples/ch4/sample-app/build-docker-image.sh
  #!/usr/bin/env bash
  set -e
  
  # (1)
  version=$(npm pkg get version | tr -d '"')
  
  # (2)
  docker buildx build \
    --platform=linux/amd64,linux/arm64 \
    -t sample-app:"$version" \
    .
  

  • 1: Run npm pkg get version to get the value of the version key in package.json.
  • 2: Run docker buildx, setting version to the value from 1.

• Make the script executable

  chmod u+x build-docker-image.sh
  

• Add a dockerize script to the scripts block in package.json

  {
    "scripts": {
      "dockerize": "./build-docker-image.sh"
    }
  }
  

• Now instead of trying to figure out how to build the Docker image, your team members can execute npm run dockerize to build the Docker image.

  npm run dockerize
  

  [!NOTE] Notice it’s npm run dockerize (with the extra run) as dockerize is a custom script, not a built-in script of NPM.

Dependency Management

dependencies : software packages & libraries that your code uses.

Kind of dependencies

• Code in the same repo

  You can

  • break your code in a single repo into multiple modules/packages
  • have these modules depend on each other

  These modules/packages allow you

  • develope different parts of your codebase in
    • isolation from the others,
    • (possible with completely separate teams working on each part)

• Code in different repos

  You can store code across multiple repos, which

  • give you more isolation between different parts of your software
    • make it even easier for separate teams to take ownership for each part.

  Typically, when code in repo A depends on code in repo B:

  • it’s a specific version of the code in repo B, which may correspond to a specific Git tag.

  • or it’s a versioned artifact published form the repo B

    e.g.

    • a Jar file in the Java world
    • a Ruby Gem in the Ruby world

• Open source code

  Most common type of dependency these days. A type of code in different repos.

Why use a dependency?

Yoy use a dependency so

• you can reply on someone else to solve certain problems for you
• instead of having to
  • solve everything yourself from scratch
  • (maintain it)

[!IMPORTANT] Key takeaway #3 Use a dependency management tool to pull in dependencies—not copy & paste.

The problems with copy-paste dependency

• Transitive dependencies

  Copy/pasting a single dependency is easy, but if

  • that dependency has its own dependencies, and
    • those dependencies have their own dependencies, and
      • so on (collectively known as transitive dependencies),

  then copy/pasting becomes rather hard.

• Licensing

  Copy/pasting may violate the license terms of that dependency, especially if you end up modifying that code (because it now sits in your own repo).

  [!WARNING] Be especially aware of dependencies that uses GPL-style licenses (known as copyleft or viral licenses),

  • if you modify the code in those dependencies,
    • you need to release your own code under the same license i.e. you’ll be forced to open source your company’s proprietary code!.

• Staying up to date

  If you copy/paste the code, to get any future updates, you’ll have to

  • copy/paste new code, and new transitive dependencies, and
  • make sure you don’t lose any changes your team members made along the way.

• Private APIs

  (Since you can access those files locally), you may end up

  • using private APIs
    • instead of the public ones that were actually designed to be used,

  which can lead to unexpected behavior, (and make staying up to date even harder)

• Bloating your repo

  Every dependency you copy/paste into your version control system makes it larger and slower.

How to use dependencies

• Instead of copy-paste, use a dependency management tool, which is usually built-in with build systems.

• You define your dependencies

  • as code
  • in the build configuration
  • including the version (of the dependencies)

  the dependency management tools is then responsible for:

  • downloading those dependencies (plus any transitive dependencies)
  • making them available to your code.

Example: Add Dependencies in NPM

So far, the Node.js example-app has not any dependencies other than the http standard library built in Node.js.

In this example, you will introduce an dependency named Express, which is a popular web framwork for Node.js

• Install Express & save it to dependencies in package.json

  npm install express --save
  

  • The package will now have a new dependencies section:

    {
      "dependencies": {
        "express": "^4.19.2"
      }
    }
    

• There will be 2 new file/folder next to the package.json file:

  • node_modules folder: where NPM download & install dependencies

    • Should be in your .gitignore; anyone check out this repo the first time can run npm install to install the dependencies.

  • package-lock.json file: a dependency lock file, which captures the exact dependencies what were installed.

    • In package.json, you can specify a version range instead of a specific version.
    • Without the package-lock.json, every time you run npm install,
      • you may get a new version of the dependencies,
        • which make the builds not reproducible
    • With the package-lock.json file, you can use npm clean-install (npm ci in short) to
      • tell NPM to perform a clean install (and install the exact versions in the lock file)
        • so the build is reproducible (every time)

• Re-write the code in app.js to use Express framework

  const express = require("express");
  
  const app = express();
  const port = 8080;
  
  app.get("/", (req, res) => {
    res.send("Hello, World!");
  });
  
  app.listen(port, () => {
    console.log(`Example app listening on port ${port}`);
  });
  

  [!TIP] By using the Express framework, it’ll be a lot easier to evolve this code into a real app by leverage all the features built into Express e.g. routing, templating, error handling, middleware, security…

• Update the Dockerfile to run npm ci

  FROM node:21.7
  
  WORKDIR /home/node/app
  
  # (1)
  COPY package.json .
  COPY package-lock.json .
  
  # (2)
  RUN npm ci --only=production
  
  COPY app.js .
  
  EXPOSE 8080
  
  USER node
  
  CMD ["npm", "start"]
  

  • 1: Copy not only package.json, but also package-lock.json into the Docker image.
  • 2: Run npm ci to have a clean install with the exact dependencies in the lock file.

  [!NOTE] The --only=production flag tells NPM to only install the production dependencies.

  • An NPM package can also have dev-dependencies - which are only used in the dev environment.
  • When running in production environment, these dev dependencies are not needed.

Get your hands dirty with modern frontend build systems

• PNPM
• Yarn
• Turborepo
• Lerna
• Parcel

Automated Testing

Why use automated testing

legacy code : spaghetti code without automated tests, documentation : code that you don’t have the confidence to make changes

To prevent legacy code, you use automated testing, where you:

• write test code to validate that
  • your production code works
    • the way you expect it to.

By writing automated tests, you might catch some of the bugs,

• but the most important benefit of having a good suite of automated tests is, you have the confidence to make changes quickly, because:

  • you don’t have to keep the whole program in your head
  • you don’t have to worry about breaking other people’s
  • you don’t have to repeat the same boring, error-prone manual testing over & over agian.

[!IMPORTANT] Key takeaway #4 Use automated tests to give your team the confidence to make changes quickly.

Types of automated tests

There’re a lot of type of automated tests:

• Compiler

  If you’re using a statically-typed language (e.g., Java, Scala, Haskell, Go, TypeScript), you can pass your code through the complier (compile) to automatically identify

  • (a) syntactic issues
  • (b) type errors.

  If you’re using a dynamically-typed language (e.g., Ruby, Python, JavaScript), you can pass the code through the interpreter to identify syntactic issues.

• Static analysis / linting

  These are tools that read & check your code “statically” — that is, without executing it — to automatically identify potential issues.

  Examples:

  • ShellCheck for Bash
  • ESLint for JavaScript
  • SpotBugs for Java
  • RuboCop for Ruby.

• Policy tests

  In the last few years, policy as code tools have become more popular as a way to define and enforce company policies & legal regulations in code.

  Examples: Open Policy Agent, Sentinel, Intercept.

  • Many of these tools are based on static analysis, except they give you flexible languages to define what sorts of rules you want to check.
  • Some rely on plan testing, as described next.

• Plan tests

  Whereas static analysis is a way to test your code without executing it at all, plan testing is a way to partially execute your code. This typically only applies to tools that can generate an execution plan without actually executing the code.

  For example:

  • OpenTofu has a plan command that shows you what changes the code would make to your infrastructure without actually making those changes: so in effect, you are running all the read operations of your code, but none of the write operations.

  You can write automated tests against this sort of plan output using tools such as Open Policy Agent and Terratest.

• Unit tests

  This is the first of the test types that fully execute your code to test it.

  The idea with unit tests is to execute only a single “unit” of your code:

  • What a unit is depends on the programming language, but it’s typically a small part of the code, such as one function or one class. - You typically mock any dependencies outside of that unit (e.g., databases, other services, the file system), so that the test solely executes the unit in question.

  To execute the unit tests:

  • Some programming languages have unit testing tools built in e.g. testing for Go; unittest for Python
  • Whereas other languages rely on 3rd party tools for unit testin e.g. JUnit for Java; Jest for JavaScript

• Integration tests

  Just because you’ve tested a unit in isolation and it works, doesn’t mean that multiple units will work when you put them together. That’s where integration testing comes in.

  With integeration tests, you test

  • multiple units of your code (e.g., multiple functions or classes),
  • often with a mix of
    • real dependencies (e.g., a database)
    • mocked dependencies (e.g., a mock remote service).

• End-to-end (E2E) tests

  End-to-end tests verify that your entire product works as a whole, which mean you:

  • run
    • your app,
    • all the other services you rely on,
    • all your databases and caches, and so on,
  • test them all together.

  These often overlap with the idea of acceptance tests, which verify your product works from the perspective of the user or customer (“does the product solve the problem the user cares about”).

• Performance tests

  Most unit, integration, and E2E tests verify the correctness of a system under ideal conditions: one user, low system load, and no failures.

  Performance tests verify the stability & responsiveness of a system in the face of heavy load & failures.

Example: Add Automated Tests for the Node.js App

• How to know if the the Node.js example-app work?

  const express = require("express");
  
  const app = express();
  const port = 8080;
  
  app.get("/", (req, res) => {
    res.send("Hello, World!");
  });
  
  app.listen(port, () => {
    console.log(`Example app listening on port ${port}`);
  });
  

• So far, you will do it through manual testing:

  • Manually ran the app with npm start
  • Then open the app URL in the brower.
  • Verify that the output is matched.

• What if you have

  • hundreds of URLs?
  • hundreds of developers making changes?

[!NOTE] The idea with automated testing is to

• write code that
  • performs the testings steps for you.

Then the computer can run these test code and test your app faster, more reliable.

Add unit tests for the Node.js App

• You’ve start with unit test. To add a unit test, first you need a unit of code, which you will introduce in this example

• For this example, create a basic module with 2 functions that reverve characters & words in a string. Those 2 functions acts as the unit of code to be tested.

  # 1
  function reverseWords(str) {
    return str.split(" ").reverse().join(" ");
  }
  
  # 2
  function reverseCharacters(str) {
    return str.split("").reverse().join("");
  }
  
  module.exports = { reverseCharacters, reverseWords };
  

  • 1: reverseWords reverses the words in a string. e.g. hell world will be reversed to world hello
  • 2: reverseCharacters reverses the characters in a string e.g. abcde will be reversed to edcba

[!NOTE] How do you know this code actually works?

1. Imagine how the code runs in your head?
2. Test the code manually?

• Fire up a REPL - an interactive shell - to manuallt execute code.
  • Import the reverve file
  • Run the reverseWords, reverseCharacters function with your input> , and check the output.
  • (When you’re done with the REPL, use Ctrl+D to exit).

3. Capture the steps you did in a REPL in an automated test.

• In this example, you will use Jest as the testing framework, and SuperTest as the library for testing HTTP apps.

• Intstall Jest and Supertest (and save them as dev dependencies with --save-dev flag)

  npm install --save-dev jest supertest
  

  Your package.json should looks like this:

  {
    "dependencies": {
      "express": "^4.19.2"
    },
  
    "devDependencies": {
      "jest": "^29.7.0",
      "supertest": "^7.0.0"
    }
  }
  

• Update the test script (in package.json) to run Jest

  {
    "scripts": {
      "test": "jest --verbose"
    }
  }
  

• Writing tests for reserveWords function

  const reverse = require("./reverse");
  
  //                                                         1
  describe("test reverseWords", () => {
    //                                                       2
    test("hello world => world hello", () => {
      const result = reverse.reverseWords("hello world"); // 3
      expect(result).toBe("world hello"); //                 4
    });
  });
  

  • 1: Use descibe function to group server tests together.
    • The first argument: description of the group of tests.
    • The second argument: a function that will run the tests for this group.
  • 2: Use test function to define individual tests
    • The first argument: description of the test.
    • The second argument: a function that will run the test
  • 3: Call the reverseWords function and store the result in the variable result.
  • 4: Use the expect matcher to check that the result matches “world hello”. (If it doesn’t match, the test will fail.)

• Use npm test to run the tests

  npm test
  

  • The test PASS without any error.

• Add a second unit test for the reverseWords function

  describe("test reverseWords", () => {
    test("hello world => world hello", () => {});
  
    test("trailing whitespace   => whitespace trailing", () => {
      const result = reverse.reverseWords("trailing whitespace   ");
      expect(result).toBe("whitespace trailing");
    });
  });
  

• Re-run npm test

  npm test
  

  • The test FAIL

• Fix whitespace handling in reverseWords

  function reverseWords(str) {
    return str
      .trim() // 1
      .split(" ")
      .reverse()
      .join(" ");
  }
  

  • 1: Use the trim functon to strip leading & trailing whitespace.

• Re-run npm test; it should pass now.

This is a good example of the typical way you write code

• when you have a good suite of automated test to lean on:
  • make a change
  • re-run the tests
  • make another changes
  • re-run the tests
  • add new tests
  • …

With each iteration,

• your test suite gradually improves
  • you build more & more confidence in your code
    • you can go faster & faster

The automated tests

• provides a rapid feedback loop that help you being more productive
• acts as regression tests prevent old bugs

[!IMPORTANT] Key takeaway #5 Automated testing makes you more productive while coding by providing a rapid feedback loop: make a change, run the tests, make another change, re-run the tests, and so on.

[!IMPORTANT] Key takeaway #6 Automated testing makes you more productive in the future, too: you save a huge amount of time not having to fix bugs because the tests prevented those bugs from slipping through in the first place.

Using code coverage tools to improve unit tests

code coverage : the percent of code got executed by your tests : can be measured by many automated testing tools

• Update test script to also measure code coverage

  {
    "scripts": {
      "test": "jest --verbose --coverage"
    }
  }
  

• Run npm test to see the extra information about code coverage

  npm test
  

  • There is also a new coverage folder (next to package.json), which contains HTML reports about code coverage.
  • Open the HTML reports, you can see:
    • How many time each part of the code were executed
    • The part of code that wasn’t executed at all

• Now, you know which parts of the code wasn’t tested, you can add unit test for them:

  describe("test reverseCharacters", () => {
    test("abcd => dcba", () => {
      const result = reverse.reverseCharacters("abcd");
      expect(result).toBe("dcba");
    });
  });
  

• Re-run the test and now the code coverage is 100%.

Add end-to-end tests for the Node.js App

In this example, you will add an end-to-end test for the Node.js sample-app: a test that makes an HTTP request to the app, and chcek the response.

• First, split out the part of the app that listen on a port

  // app.js
  const express = require("express");
  
  const app = express();
  
  app.get("/", (req, res) => {
    res.send("Hello, World!");
  });
  
  module.exports = app;
  

  // server.js
  const app = require("./app");
  
  const port = 8080;
  
  app.listen(port, () => {
    console.log(`Example app listening on port ${port}`);
  });
  

• Update the start script in package.json

  {
    "scripts": {
      "start": "node server.js"
    }
  }
  

• Add a end-to-end test for the app

  // app.test.js
  const request = require("supertest");
  const app = require("./app"); // 1
  
  describe("Test the app", () => {
    test("Get / should return Hello, World!", async () => {
      const response = await request(app).get("/"); // 2
      expect(response.statusCode).toBe(200); //        3
      expect(response.text).toBe("Hello, World!"); //  4
    });
  });
  

  • 1: Inport the app code from app.js
  • 2: Use the SuperTest libary (imported under the name request) to fire up the app and make an HTTP GET request to it at the / URL.
  • 3: Check that the reponse status code is a 200 OK
  • 4: Check that the response body is the text "Hello, World!"

• Re-run npm test

  npm test
  

Get your hands dirty with end-to-end test for Node.js app

• Add a new endpoint to the sample app
• Add a new automated test to validate the endpoint works as expected.

Example: Add Automated Tests for the OpenTofu Code

[!NOTE] You can write automated tests not only for app code, but also for infrastructure code, too.

The tooling for infrastructure tests isn’t as mature, and the tests take longer to run, but the tests five all the same benefits.

In this example, you will add an automated tests for the lambda-sample OpenTofu module in Chapter 3.

[!NOTE] There are several approaches to test OpenTofu code:

• Static analysis: Terrascan, Trivy, tflint
• Policy testing: Open Policy Agent, Sentinel
• Plan testing: build-in test command, Open Policy Agent, Terratest
• Unit, integration, end-to-end testing:
  • Build-in test command: for simple modules, tests.
  • Terratest : for more complex modules, tests.

• Copy that module

  cd examples
  mkdir -p ch4/tofu/live
  cp -r ch3/tofu/live/lambda-sample ch4/tofu/live
  cd ch4/tofu/live/lambda-sample
  

Add static analysis for your OpenTofu code using Terrascan

• Create a config file for Terrascan called terrascan.toml

  [severity]
  level = "high"
  

• Install Terrscan

• Run terrascan in the lambda-sample folder

  terrascan scan \
    --iac-type terraform \
    --non-recursive \
    --verbose \
    -c terrascan.toml
  

  • --iac-type terraform: Analyze only Terraform or OpenTofu code.

  • --non-recursive:

    By default, Terrascan tries to scan everything in the current folder and all subfolders.

    This flag avoids Terrascan scanning the src folder within lambda-sample and complaining that folder doesn’t contain OpenTofu code.

  • --verbose: This gives a bit of extra log output, including Rule IDs for any policies that have been violated.

  • -c terrascan.toml: Use the settings in the configuration file terrascan.toml you created.

Add unit tests for your OpenTofu code using the test command

[!NOTE] The test in this example will deploy real resources into your AWS accounts.

• It’s closer to integration tests
• But it still test just a single unit - so it’s still a unit test

• Use the test-endpoint module (in example code repo at ch4/tofu/modules/test-endpoint) to make an HTTP request to an endpoint (from your OpenTofu code)

[!NOTE] Currently, the test command can only use local modules, so use need to make a copy of it in your test.

• Clone test-endpoint module

  cd examples
  mkdir -p ch4/tofu/modules
  cp -r ../../<EXAMPLE_CODE_REPO>/ch4/tofu/modules/test-endpoint ch4/tofu/modules
  

• In the lambda-sample module, create a test file

  # examples/ch4/tofu/live/lambda-sample/deploy.tftest.hcl
  run "deploy" {
    command = apply
  }
  
  # (2)
  run "validate" {
    command = apply
  
    # (3)
    module {
      source = "../../modules/test-endpoint"
    }
  
    # (4)
    variables {
      endpoint = run.deploy.api_endpoint
    }
  
    # (5)
    assert {
      condition     = data.http.test_endpoint.status_code == 200
      error_message = "Unexpected status: ${data.http.test_endpoint.status_code}"
    }
  
    # (6)
    assert {
      condition     = data.http.test_endpoint.response_body == "Hello, World!"
      error_message = "Unexpected body: ${data.http.test_endpoint.response_body}"
    }
  }
  

  • 1: The first run block will run apply on the lambda-sample module itself.

  • 2: The second run block will run apply as well, but this time on a test-endpoint module, as described in (3).

  • 3: This module block is how you tell the run block to run apply on the test-endpoint module (the module you copied from the blog post series’s sample code repo).

  • 4: Read the API Gateway endpoint output from the lambda-sample module and pass it in as the endpoint input variable for the test-endpoint module.

  • 5: assert blocks are used to check if the code actually works as you expect. This first assert block checks that the test-endpoint module’s HTTP request got a response status code of 200 OK.

  • 6: The second assert block checks that the test-endpoint module’s HTTP request got a response body with the text “Hello, World!”

• (Authenticate to AWS)

• Run tofu test

  tofu test
  

  • OpenTofu will
    • run apply, deploy your real resources, and then
    • at the end of the test, run destroy to clean everthing up again.

Get your hands dirty with Infrastructure Test

• Figure out how to encrypt the environment variables in the lambda module, which is a better fix for the Terrascan error.

• Add a new endpoint in your lambda module and add a new automated test to validate the endpoint works as expected.

Testing Best Practices

Which type of test to use? - The test pyramid

The first question with testing: Which testing approach should you use? Unit tests? Integration tests? E2E test?

The answer: A mix of all of them.

• Each type of test can catch different type of errors; and have different strengths & weaknesses.
• The only way to be confident your code works as expected is to combine multiple types of tests. In most cases, he proportion of tests follow the test pyramid.

For more information, see:

• The test pyramid
• A comparison of different types of automated tests

What to test

The second question with testing: What should you test?

• Some believe that every line of code must be tested (or you must achieve 100% code coverage).
• But remember, each test has a cost, does the cost bring enough values?

Before deciding if a part of your code should be test, evaluating your testing strategy & making trade-offs between the following factors:

• The code of bugs

  e.g.

  • A prototype that will be throw away in a week -> the cost of bug is low
  • A payment processing system -> the cost of bug is very high.

  Usually, the cost of bug is high for systems that

  • touches data storage
  • relates to securiy
  • cannot be break…

• The likelehood of bugs

  e.g.

  • If there is a lot of people working on the same code, there might be more bugs (integration bugs…).
  • Math problems.
  • Your own distributed consensus algorithm

• The cost of testing

  • Usually, unit tests has low cost
  • Integration tests, end-to-end tests, performance tests are more expensive to write, run.

• The cost of not having tests

  Many companies make analysis about cost/benefit of test and conlcude that tests aren’t worth it.

  But not have tests has a big cost: FEAR.

  • The company may end up having a paralyzed dev team.

When to test

The third question about testing: When to test?

• Add tests several years after you write the code: much expensive, but not as beneficial.
• Add tests a day after you write the code: cheaper, more beneficial.
• Add tests before you write the code: lowest cost, most beneficial.

Test-Driven Development (TDD)

TDD (Test-Driven Development) : You write the test before you write the implementation code : Isn’t it weird? How can you test something not-existed?

With TDD, The tests

• will test the implementation code
• provide a feedback that leads to a better design

By trying to write tests for your code (before you write the implementation codes), you’re forced to take a step back & ask important questions:

• How do I structure the code so I can test it?
• What dependencies do I have?
• What are the common use cases? Corner cases?

[!TIP] If you find that your code is hard to test, it’s almost always

• a sign that it needs to be refactored (for some other reasons) too.

e.g.

• The code uses a lot of mutable state & side effects -> Hard to test & hard to reuse, understand.
• The code has many ocmplex interactions with its dependencies -> It’s tightly coupld & hard to change.
• The code has many use cases to cover -> It’s doing too much, needs to be broken up.

TDD cycle:

0. Add basic placeholders for the new functionality (e.g., the function signatures):

• just enough for the code to compile
• but don’t actually implement the functionality.

1. Add tests for the new functionality.

1.2. (RED) Run all the tests. The new tests should fail, but all other tests should pass.

2. Implement the new functionality.

2.2. (GREEN) Rerun the tests. Everything should now pass.

3. (REFACTOR) Refactor the code until you have a clean design, re-running the tests regularly to check everything is still working.

[!TIP] A TDD cycle is also known as Red - Green - Refactor.

[!NOTE] When using TDD, the design of your code emerges as a result of a repeated test-code-test cycle.

• Without TDD, you often come up with a design and make it your final design.
• With TDD:
  • you need to figure how to pass new tests (in each cycle), which forces you to iterate on your design.
  • you often ship something more effective.

Which type of test to apply TDD?

• You can apply TDD for many type of tests:
  • Unit tests -> Force you consider how to design the small parts of your code.
  • Integration tetsts -> Force you to consider how your different parts communicate with each other.
  • End-to-end tests -> Force you to consider how to deploy everything.
  • Performance tests -> Force you to think what is the bottlenecks are & which metrics you need gather to identify them.

For more information about TDD, see:

• Growing Object-Oriented Software, Guided by Tests by Steve Freeman and Nat Pryce (Addison-Wesley Professional)
• Hello, Startup: A Programmer’s Guide to Building Products, Technologies, and Teams (O’Reilly)

The other benefits of TDD:

• By writing tests first, you increase the chance of having thorough test converage.
  • Because you’re forced to write code incrementally. Each incremental code can be tested more easy.

When not to use TDD?

• If you’re doing exploratory coding:
  • you don’t yet know exactly what you’re building
  • you’re just exploring the problem space by coding & messing with data

How TDD works with legacy codebase (that doesn’t have any tests)?

• You can use TDD for any changes you make to the codebase

  It’s a standard TDD cycle with some extra steps at the front:

  A. Write a test for the functionality you’re about to modify. B. Run all the tests. They should all pass. C. Use the standard TDD process for new changes you’re about to make.

  So it’s GREEN + Red-Green-Refactor.

[!TIP] TDD can also be used for bug fixing.

• If there’s a bug in production, it’s mean there was no test that caught the bug.
  • So you can do Test-Driven Bug Fixing.

Conclusion

To allow your team members to collaborate on your code:

• Always manage your code with a version control system.

• Use a build system to capture, as code, important operations and knowledge for your project, in a way that can be used both by developers and automated tools.

• Use a dependency management tool to pull in dependencies — not copy & paste.

• Use automated tests to give your team the confidence to make changes quickly.

  • Automated testing makes you more productive while coding by providing a rapid feedback loop: make a change, run the tests, make another change, re-run the tests, and so on.

  • Automated testing makes you more productive in the future, too: you save a huge amount of time not having to fix bugs because the tests prevented those bugs from slipping through in the first place.