RHEL 9 delivers latest container technologies for development and production

Three years ago, with the release of Red Hat Enterprise Linux 8 (RHEL 8), we delivered a new set of container tools with a new concept called Application Streams. These new container tools enabled RHEL users to find, run, build and share containers. For more information on why RHEL moved from Docker to Podman (and the journey it took us to get there) see RHEL 8 enables containers with the tools of software craftsmanship.

Red Hat named a Leader in the 2023 Gartner® Magic Quadrant™

Red Hat was positioned highest for ability to execute and furthest for completeness of vision in the Gartner 2023 Magic Quadrant for Container Management.

Read the report

In our previous release, What's new in Red Hat Enterprise Linux 8.5 Container Tools?, we introduced a lot of the foundational features and capabilities needed to get to RHEL 9.

With the release of RHEL 9, we continue to deliver the same container tools based on Pod Manager (podman), Buildah, Skopeo, Udica, CRIU and other Linux utilities. RHEL 9 continues with the philosophy of delivering what we believe is the best tool for the job, making it easier for container users to upgrade from RHEL 8 to RHEL 9. This article digs into the latest technologies and changes in how RHEL 9 packages container tools.

• RHEL 8.6 and RHEL 9.0 - what’s changed?

• Centralized management of identity mappings

• Support for NFS for container storage

• Advanced network stack for Podman 4.0

• Portable certificate and signing container

• crun becomes the default container runtime in RHEL 9.0

• Control Group v2 (cgroup 2) becomes the default in RHEL 9 and Podman

RHEL 8.6 and RHEL 9.0 - what’s changed?

First, let’s start with some RHEL background information. RHEL 8.6 and RHEL 9.0 are what we refer to as synchronized releases. They were released at approximately the same time and will continue in synchronization until RHEL 8 reaches 8.10, and goes into maintenance support phase, and RHEL 9 will continue on with new features. This gives users approximately two years of feature updates on both platforms and a window to upgrade. The RHEL life cycle is intentionally designed to make upgrading easier, which also applies to container tools.

Notice in the drawing below, that the following version of RHEL 8 and 9 are in sync:

• RHEL 9 Alpha -> RHEL 8.4

• RHEL 9 Beta -> RHEL 8.5

• RHEL 9.0 GA -> RHEL 8.6

• RHEL 9.1 -> RHEL 8.7

• RHEL 9.2 -> RHEL 8.8

• RHEL 9.3 -> RHEL 8.9

• RHEL 9.4 -> RHEL 8.10 

This synchronization between RHEL 8 and 9 simplifies upgrades and extends to the versions of Podman, Buildah and Skopeo. That’s right, the fast and stable versions of Podman, Buildah and Skopeo are aligned between versions. For example, notice that the latest version of Podman is the same between RHEL 8 and 9:

On RHEL 8

cat /etc/redhat-release 
Red Hat Enterprise Linux release 8.6 (Ootpa)
[root@lance ~]# podman --version
podman version 4.0.2

On RHEL 9

cat /etc/redhat-release 
Red Hat Enterprise Linux release 9.0 (Plow)
podman --version
podman version 4.0.2

The synchronization of important software like Podman between major versions of RHEL simplifies upgrades, but there are still some changes you should be aware of. In RHEL 8.X we released two application streams, one to give developers access to the latest versions of Podman, Buildah and Skopeo, and a second, stable stream to provide operations teams with a two-year support life cycle. 

RHEL 8 Application Streams

There were a couple of challenges with the methodology used in RHEL 8. First, we didn’t see a large uptake of the stable stream, which broke our initial assumption when we launched RHEL 8. We launched the fast and stable streams in RHEL 8 thinking people wanted access to a stable container API (Podman) while still consuming the latest-greatest operating system bits (Linux kernel, systemd, etc.). This has historically been the desire with container-based operating systems.

This assumption turned out to be false. Instead, RHEL users primarily sought access to the stable container tools stream in conjunction with two-year extended update support (EUS) for RHEL as a whole. It turns out that people who build on RHEL want access to an entire operating system with a longer life cycle. As such, we changed the way we released container tools in RHEL 9. 

RHEL 9 Rolling Application Stream and EUS

In RHEL 9, we also provide two ways to consume container tools, one focused on moving quickly, and the other focused on stability. Developers and users who want access to the latest and greatest versions of Podman, Buildah and Skopeo can consume an Application Stream which is released up to every 12 weeks (just like RHEL 8). By design, this stream is kept in sync with the fast stream on RHEL 8 (until 8.10, when RHEL 8 slows down development), making it easy to upgrade/downgrade between major versions. 

If RHEL 9 users need access to a stable stream that is supported for two years with security backports, they can access it through RHEL Extended Update Support (EUS), although this is a separate subscription. By design, each version of container tools in EUS releases of RHEL 9 is kept in sync with a corresponding stable stream in RHEL 8. Again, this makes it easy to upgrade from RHEL 8 to RHEL 9, maintaining a consistent version of Podman, and reducing the chances of introducing regressions.

It’s important to note that RHEL 8 will continue to be delivered exactly as it was designed. It will not convert to the methodology used in RHEL 9. If your developers, admins, or architects have planned a roll-out of RHEL 8 based on stable streams, you can continue to rely on them and will not need EUS.

It should also be noted that in RHEL 8.6 the container-tools:2.0 module is now deprecated and you should move to a newer version (3.0, 4.0, etc.) to continue receiving security patches. For more information, consult the RHEL Application Streams Life Cycle page. 

Centralized management of identity mappings

Rootless Podman is a great technology. It improves container security by running them as a non-root user, the same way regular processes run on a system. This means an attacking workload would need to break out of an extra layer of security, first getting through the container controls and then figuring out a way to become root. This is great for large fleets of laptops/desktops, in HPC environments and even for developers on shared servers.

But, managing a large number of non-root users across large fleets of RHEL workstations, HPC nodes, or shared servers has historically been pretty tough to do, because an administrator needed to manage /etc/subuid and /etc/sugid files manually on every node.

Not anymore. With RHEL 9, we’ve introduced a feature in Identity Management (IdM) that allows administrators to easily manage rootless podman users across a fleet of users and RHEL nodes. Users can assign subuids/subgids to a single user, or to all the users in a directory server. It’s really convenient. For more information, see Chapter 29. Managing subID ranges manually.

Support for NFS for container storage

As mentioned, Rootless Podman is a great feature, and often administrators have many users across many nodes (workstations, HPC, shared developer servers, etc.). In these scenarios, users want to bring their data with them. For example, if they do a “podman pull” on one node, they want to have that image available on any node in the cluster. An easy way to do this with normal processes is with NFS, but this hasn’t historically worked with Podman/containers.

With this new feature, Podman can now store data on any NFS server which supports extended attributes (xattrs). Non-root/rootless users can pull an image once and use it anywhere their home directory is available. This is extremely convenient with workstations, HPC nodes, or even shared development servers where CI/CD is done. For more information, see this upstream article: New features for running containers on NFS with rootless Podman 

Advanced network stack for Podman 4.0

In the release of RHEL 9 with Podman 4.X, a new network stack is available to users. It comes with new features like better IPv6 support, improved support for containers in multiple networks and improved performance. 

The following article does a great job of giving an overview: Podman 4.0's new network stack: What you need to know.

Portable certificate and signing container

With the release of RHEL 8.4, we introduced UBI Micro, one of the industry's smallest and fastest container images (Introduction to Red Hat's UBI Micro).

With the release of RHEL 9, we’ve built on this technology to create a tiny (12.5MB) OpenSSL container image which can be used for simple cryptographic use cases like generating SSL certificate requests, verifying SSL certificates, or even signing files.

This gives developers a standardized way to perform trusted cryptographic use cases whether in production or on their desktops/laptops. Like all Red Hat Universal Base Images, you and your developers can use and distribute this new portable certificate and signing container anywhere you need.

For more information, check out the listing on the Red Hat Ecosystem Catalog

• ubi9/openssl 

• ubi8/openssl

crun becomes the default container runtime in RHEL 9.0

Back in 2020, Red Hat contributors introduced crun, a fast, low-memory container runtime which is OCI compliant. In RHEL 9, we’re making crun the default container runtime. Both crun and runc will be supported for the full life cycle of RHEL 9.

The move to crun as the default simplifies many tasks for administrators for many low-level configuration tasks, improves performance and memory utilization and unblocks all kinds of cool use cases. For more information, see: An introduction to crun, a fast and low-memory footprint container runtime

Control Group v2 (cgroup 2) becomes the default in RHEL 9 and Podman

The cgroup 2 project describes itself as, “a Linux kernel component that provides a mechanism to isolate, measure and control the distribution of resources for a collection of processes on a server.” This gives administrators and infrastructure software, like Container Engines and Runtimes, a powerful mechanism to limit the resources used by any given process, which is especially useful with containers. 

While cgroup 2 was first supported in RHEL 8, a user needed to enable it and reboot. With RHEL 9, cgroup 2 is the default mechanism out of the box, which gives finer-grained control over rootless containers (First Look: Rootless Containers and cgroup v2 on Fedora 31). For a nice introduction to cgroup 2,  see: World domination with cgroups in RHEL 8: welcome cgroups v2!

Conclusion

There are a lot of great new container capabilities in RHEL 9.0 with Podman 4.0.2, but many of these capabilities are also available in RHEL 8.6. Whether you want to jump to the latest and greatest or get more out of an existing installation, the design and architecture of the container tools application stream has you covered.

With RHEL 9, we continue to deliver quick access to the latest and greatest Podman, Buildah and Skopeo, but now we also give you access to a stable stream through EUS. We’ve tried to make RHEL 9 even easier to use for our customers, and we hope you enjoy it. We’d love to hear your feedback.

Feel free to provide feedback to our new product manager for Container Tools, Mark Russell (https://www.linkedin.com/in/marrusl/), our RHEL Server product manager, Scott McCarty (@fatherlinux), our technical marketing manager, Eric Hendricks (@itguyeric), or Tweet at our official Red Hat Enterprise Linux account, @rhel).