GitOps Cookbook: Kubernetes automation in practice

Note: The following is an excerpt from GitOps Cookbook by Natale Vinto and Alex Soto Bueno (O'Reilly Media, December 2022). Download the e-book for practical, hands-on examples to help you implement GitOps and CI/CD in a cloud native environment.

With the advent of practices such as infrastructure as code (IaC), software development has pushed the boundaries of platforms where you can run applications. This becomes more frequent with programmable, API-driven platforms such as public clouds and open source infrastructure solutions. While some years ago developers were only focusing on application source code, today they also have the opportunity to code the infrastructure where their application will run. This gives control and enables automation, which significantly reduces lead time.

A good example is with Kubernetes, a popular open source container workload orchestration platform and the de facto standard for running production applications, either on public or private clouds. The openness and extensibility of the platform enables automation, which reduces risks of delivery and increases service quality. Furthermore, this powerful paradigm is extended by another increasingly popular approach called GitOps.

What is GitOps?

GitOps is a methodology and practice that uses Git repositories as a single source of truth to deliver infrastructure as code. It takes the pillars and approaches from DevOps culture and provides a framework to start realizing the results. The relationship between DevOps and GitOps is close, as GitOps has become the popular choice to implement and enhance DevOps, platform engineering, and SRE.

GitOps is an agnostic approach, and a GitOps framework can be built with tools such as Git, Kubernetes, and CI/CD solutions. The three main pillars of GitOps are:

• Git is the single source of truth
• Treat everything as code
• Operations are performed through Git workflows

There is an active community around GitOps, and the GitOps Working Group defines a set of GitOps Principles (currently in version 1.0.0) available at OpenGitOps:

• Declarative: A system managed by GitOps must have its desired state expressed declaratively.
• Versioned and immutable: The desired state is stored in a way that enforces immutability and versioning and retains a complete version history.
• Pulled automatically: Software agents automatically pull the desired state declarations from the source.
• Continuously reconciled: Software agents continuously observe the actual system state and attempt to apply the desired state.

Why GitOps?

Using the common Git-based workflows that developers are familiar with, GitOps expands upon existing processes from application development to deployment, app lifecycle management, and infrastructure configuration.

Every change throughout the application lifecycle is traced in the Git repository and is auditable. This approach is beneficial for both developers and operations teams as it enhances the ability to trace and reproduce issues quickly, improving overall security. One key point is to reduce the risk of unwanted changes (drift) and correct them before they go into production.

Here is a summary of the benefits of the GitOps adoption in four key aspects:

• Standard workflow: Use familiar tools and Git workflows from application development teams
• Enhanced security: Review changes beforehand, detect configuration drifts, and take action
• Visibility and audit: Capture and trace any change to clusters through Git history
• Multicluster consistency: Reliably and consistently configure multiple environments and multiple Kubernetes clusters and deployment

Kubernetes CI/CD

Continuous integration (CI) and continuous delivery (CD) are methods used to frequently deliver apps by introducing automation into the stages of app development.

CI/CD pipelines are one of the most common use cases for GitOps. In a typical CI/CD pipeline, submitted code checks the CI process while the CD process checks and applies requirements for things like security, infrastructure as code, or any other boundaries set for the application framework. All code changes are tracked, making updates easy while also providing version control should a rollback be needed. CD is the GitOps domain and it works together with the CI part to deploy apps in multiple environments, as you can see in Figure 1-1.

With Kubernetes, it’s easy to implement an in-cluster CI/CD pipeline. You can have CI software create the container image representing your application and store it in a container image registry. Afterward, a Git workflow such as a pull request can change the Kubernetes manifests illustrating the deployment of your apps and start a CD sync loop, as shown in Figure 1-2.

This cookbook will show practical recipes for implementing this model on Kubernetes acting as a CI/CD and GitOps platform.

App deployment with GitOps on Kubernetes

As GitOps is an agnostic, platform-independent approach, the application deployment model on Kubernetes can be either in-cluster or multicluster. An external GitOps tool can use Kubernetes just as a target platform for deploying apps. At the same time, in-cluster approaches run a GitOps engine inside Kubernetes to deploy apps and sync manifests in one or more Kubernetes clusters.

The GitOps engine takes care of the CD part of the CI/CD pipeline and accomplishes a GitOps loop, which is composed of four main actions as shown in Figure 1-3:

• Deploy: Deploy the manifests from Git.
• Monitor: Monitor either the Git repo or the cluster state.
• Detect drift: Detect any change from what is described in Git and what is present in the cluster.
• Take action: Perform an action that reflects what is on Git (rollback or three-way diff). Git is the source of truth, and any change is performed via a Git workflow.

In Kubernetes, application deployment using the GitOps approach makes use of at least two Git repositories: one for the app source code, and one for the Kubernetes manifests describing the app’s deployment (Deployment, Service, etc.).

The following list outlines the items in the workflow:

1. App source code repository
2. CI pipeline creating a container image
3. Container image registry
4. Kubernetes manifests repository
5. GitOps engine syncing manifests to one or more clusters and detecting drifts

Figure 1-4 illustrates the structure of a GitOps project on Kubernetes.

DevOps and agility

GitOps is a developer-centric approach to continuous delivery and infrastructure operations, and a developer workflow through Git for automating processes. As DevOps is complementary to Agile software development, GitOps is complementary to DevOps for infrastructure automation and application lifecycle management. As you can see in Figure 1-5, it’s a developer workflow for automating operations.

One of the most critical aspects of the Agile methodology is to reduce the lead time, which is described more abstractly as the time elapsed between identifying a requirement and its fulfillment. Reducing this time is fundamental and requires a cultural change in IT organizations.

Seeing applications live provides developers with a feedback loop to redesign and improve their code and make their projects thrive. Similarly to DevOps, GitOps also requires a cultural adoption in business processes. Every operation, such as application deployment or infrastructure change, is only possible through Git workflows. And sometimes, this means a cultural shift.

The "Teaching Elephants to Dance (and Fly!)" speech from Burr Sutter gives a clear idea of the context. The elephant is where your organization is today. There are phases of change between traditional and modern environments powered by GitOps tools. Some organizations have the luxury of starting from scratch, but for many businesses, the challenge is teaching their lumbering elephant to dance like a graceful ballerina.

Download GitOps Cookbook

Ready to explore the latest Kubernetes automation insights for pipelines and CI/CD workloads? Download the full GitOps Cookbook from Red Hat Developer.