Build Kubernetes pods with Podman play kube

Whenever Podman developers talk about its future, they always mention one topic—making it easier to test workloads with Podman and deploy them into Kubernetes. The primary way users jump between Podman and Kubernetes is by using Podman's generate kube and play kube subcommands. As the names imply, generate kube creates a YAML description of a Podman pod or container to run in Kubernetes. Conversely, the play kube subcommand allows you to run Podman pods based on a Kubernetes YAML file.

Recently, Podman received a series of enhancements that enhance this experience by adding the ability to:

• Build images with play kube
• Tear down pods with play kube
• Support Kubernetes-style init containers

Build images with play kube

Users of podman play kube told us they want to build images as part of the play process. Because Kubernetes does not have a similar concept, we were at first hesitant to implement the idea. The more play kube gets used, the more it gets compared to Docker compose. That was the tipping point. Our users were right.

The new podman play kube feature looks for a directory with the same name as the image used in the YAML file. If that directory exists and there is a Containerfile or Dockerfile present in that directory, Podman builds the container image.

For example, a sample YAML file that has Apache and PHP could look like:

# Save the output of this file and use kubectl create -f to import
# it into Kubernetes.
#
# Created with podman-4.0.0-dev
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2021-09-20T17:40:19Z"
  labels:
	app: php
  name: php
spec:
  containers:
  - args:
	- apache2-foreground
	command:
	- docker-php-entrypoint
	env:
	- name: PATH
  	value: /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
	- name: TERM
  	value: xterm
	- name: container
  	...
	- name: PHP_EXTRA_BUILD_DEPS
  	value: apache2-dev
	- name: APACHE_ENVVARS
  	value: /etc/apache2/envvars
	image: php-7.2-apache-mysqli:latest
	name: apache
	ports:
	- containerPort: 80
  	hostPort: 8080
  	protocol: TCP
	resources: {}
	securityContext:
  	allowPrivilegeEscalation: true
  	capabilities:
    	drop:
    	- CAP_MKNOD
    	- CAP_NET_RAW
    	- CAP_AUDIT_WRITE
  	privileged: false
  	readOnlyRootFilesystem: false
  	seLinuxOptions: {}
	tty: true
	workingDir: /var/www/html
  dnsConfig: {}
  restartPolicy: Never
status: {}

Notice in the YAML file that the container image is referred to as php-7.2-apache-mysqli:latest. If there is a Containerfile in that directory and the image is not in the image store, Podman builds the image.

Now look at an example where you want to use docker.io/library/php:7.2-apache but the PHP extension for mysqli is not installed or enabled. Create a Containerfile inside the php-7.2-apache-mysqli directory. To give some perspective, here's the directory layout:

├── mariadb-conf
│   ├── Containerfile
│   └── my.cnf
├── php-7.2-apache-mysqli
│   ├── Containerfile
│   └── index.php
└── php.yaml

And the contents of the Containerfile are:

$ cat php-7.2-apache-mysqli/Containerfile  
FROM docker.io/library/php:7.2-apache
RUN docker-php-ext-install mysqli
COPY index.php /var/www/html/index.php

And now, execute the play kube command, citing the YAML file:

$ podman play kube php.yaml  
-->  /home/baude/myproject/php-7.2-apache-mysqli
STEP 1/3: FROM docker.io/library/php:7.2-apache
STEP 2/3: RUN docker-php-ext-install mysqli
Configuring for:
PHP Api Version:     	20170718
Zend Module Api No:  	20170718
Zend Extension Api No:   320170718
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
checking for cc... cc
checking whether the C compiler works... yes
checking for C compiler default output file name... a.out
checking for suffix of executables...  
checking whether we are cross compiling... no
checking for suffix of object files... o
…
Build complete.
Don't forget to run 'make test'.
 
Installing shared extensions: 	/usr/local/lib/php/extensions/no-debug-non-zts-20170718/
Installing header files:      	/usr/local/include/php/
find . -name \*.gcno -o -name \*.gcda | xargs rm -f
find . -name \*.lo -o -name \*.o | xargs rm -f
find . -name \*.la -o -name \*.a | xargs rm -f
find . -name \*.so | xargs rm -f
find . -name .libs -a -type d|xargs rm -rf
rm -f libphp.la   	modules/* libs/*
STEP 3/3: COPY index.php /var/www/html/index.php
COMMIT php-7.2-apache-mysqli:latest
--> 096882adf84
Successfully tagged localhost/php-7.2-apache-mysqli:latest
096882adf845274f8c6546cf52a77c7fc78b3fa20c659cfc2b73753972bd5f90
Pod:
d8774760bc3a0bdb1b405c57880f3872b0a71f3434b8d4ab6d8d18c8e6e44ffa
Container:
25162b2da0e2b9078c69404f6d6e1250f6ee4fc032b47564f9bbb971d838cf29

During the build, you can see the remnants of mysqli getting installed, and to prove it, you can verify the mysql module was installed. The index file for this container displays what modules are enabled:

$ curl http://localhost:8080
Array
(
	[0] => Core
	[1] => date
	[2] => libxml
	...
	[30] => mysqlnd
	[31] => apache2handler
	[32] => mysqli
	[33] => sodium
)

When performing a build during play kube, the directory with the image's name becomes the context directory for the build. It also adds a command-line parameter, --build, which forces a rebuild of all images used in the YAML file if they have the directory and Containerfile present. Because Kubernetes will not build the image, you need to push your newly created image to a container registry before using your YAML file with Kubernetes.

Tear down pods with play kube

The podman play kube command can create and run multiple pods with multiple containers in the pods. Managing the pods that play kube creates and runs has always been a manual process using Podman's pod commands. Moreover, play kube is fast becoming an alternative to docker compose, which is very service-oriented. For convenience and compatibility, it made sense to provide a way to tear down what was created by the YAML input file.

[ Learn more about How to use Podman inside of Kubernetes. ]

As of Podman v3.4, the podman play kube command has a new --down flag. Using this new flag mimics the docker compose "down" because it is more of a teardown. When you give the --down flag, all pods (and their containers) get stopped and subsequently removed. If a volume is in use, it does not get removed.

Revisiting the PHP container example above, once the play kube command executes, you can observe a running pod with two containers, including the pod's infra container:

$ podman pod ps
POD ID    	NAME    	STATUS  	CREATED   	INFRA ID  	# OF CONTAINERS
cc97c8c8a07d  php     	Running 	20 hours ago  bd22e1434d3a  2
$ podman ps -a
CONTAINER ID  IMAGE                               	COMMAND           	CREATED   	STATUS       	PORTS             	NAMES
bd22e1434d3a  k8s.gcr.io/pause:3.5                                      	22 hours ago  Up 22 hours ago  0.0.0.0:8080->80/tcp  cc97c8c8a07d-infra
b97893b79bb9  localhost/php-7.2-apache-mysqli:latest  apache2-foregroun...  22 hours ago  Up 22 hours ago  0.0.0.0:8080->80/tcp  php-greatnapier

Suppose you want to stop this pod. You can "replay" the YAML file by adding the --down flag:

$ podman play kube --down php.yaml  
Pods stopped:
cc97c8c8a07db0f26114022a71ee59771134244a8b465147a513b28f9b7d171b
Pods removed:
cc97c8c8a07db0f26114022a71ee59771134244a8b465147a513b28f9b7d171b
$ podman ps -a
CONTAINER ID  IMAGE   	COMMAND 	CREATED 	STATUS  	PORTS   	NAMES
$ podman pod ps
POD ID      NAME        STATUS      CREATED     INFRA ID    # OF CONTAINERS

The pod is stopped and then removed. Confirm the pod's containers are also removed with podman ps -a or podman pod ps.

Init containers support

As mentioned earlier, Podman added support for init containers inspired and based on Kubernetes init containers. The basic idea behind init containers is that these special containers run before any other regular container in a pod. Init containers are considered to be blocking because no other pod containers run until the init container exits. A pod can have multiple init containers, but they run serially in the order you added them to the pod because they are blocking.

One unique distinction between Podman init containers and Kubernetes init containers is that Podman offers two kinds of init containers: always and once. As their names suggest, an always init container runs every time the pod starts. A once init container runs at Pod startup and is deleted upon container exit. This is because Podman pods can be restarted, unlike pods in Kubernetes, which are only ever replaced.

Init containers can be handy for starting services, initializing databases (on a volume), or blocking on some condition that gets met before other containers in the pod start. When init containers are combined with podman play kube and its ability to build images, you can really see the maturation of containers. You can also generate YAML for init containers if podman play kube gets run on a pod containing them.

An example: Apache, MariaDB, and PHP

The LAMP (Linux, Apache, MySQL, and PHP) stack has long been a shining example for Linux. The following example will implement a LAMP stack inside a Podman pod. It will use init containers and volumes to prepare the respective containers with their content. In the case of the database volume, the one-time init container prepares the basic configuration and database files and then preloads a database with data. The web volume init container, which always runs before the Apache and PHP container, populates the webserver content with the latest from a Git repository.

Image

First, I will perform the setup manually, then I'll follow with a play kube example where the images get built on the fly. To follow along, clone my project from Git:

$ git clone http://github.com/baude/my-lamp-project

Once the clone completes, you will have the following directory structure:

├── lamp.yaml
├── mariadb-conf
│   ├── Containerfile
│   └── my.cnf
├── mydbstuff
├── mywebstuff
├── php-7.2-apache-mysqli
│   └── Containerfile
├── php-7.2-apache-mysqli-ext
│   ├── Containerfile
│   └── index.php
└── php.yaml

Next, you need to build a couple of images manually. Remember, this version demonstrates init containers used manually in Podman:

$ cd mariadb-conf
$ podman build -t mariadb-conf .
STEP 1/2: FROM docker.io/library/mariadb
Trying to pull docker.io/library/mariadb:latest...
Getting image source signatures
Copying blob 7275e59ecb3d done   
...  
Copying config 6b01262bc7 done   
Writing manifest to image destination
Storing signatures
STEP 2/2: COPY my.cnf /etc/mysql/my.cnf
COMMIT mariadb-conf
--> c62d6a4978a
Successfully tagged localhost/mariadb-conf:latest
c62d6a4978a42b649e3468a12d210197dabae29b5daed2d1f0ead778f70a5fc3
$ cd ../php-7.2-apache-mysqli-ext/
$ podman build -t php-7.2-apache-mysqli-ext .
STEP 1/3: FROM docker.io/library/php:7.2-apache
Trying to pull docker.io/library/php:7.2-apache...
Getting image source signatures
Copying blob c2199db96575 done   
... 
Copying config c61d277263 done   
Writing manifest to image destination
Storing signatures
STEP 2/3: RUN docker-php-ext-install mysqli
Configuring for:
PHP Api Version:     	20170718
Zend Module Api No:  	20170718
Zend Extension Api No:   320170718
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for a sed that does not truncate output... /bin/sed
...
 
Build complete.
Don't forget to run 'make test'.
 
Installing shared extensions: 	/usr/local/lib/php/extensions/no-debug-non-zts-20170718/
Installing header files:      	/usr/local/include/php/
find . -name \*.gcno -o -name \*.gcda | xargs rm -f
find . -name \*.lo -o -name \*.o | xargs rm -f
find . -name \*.la -o -name \*.a | xargs rm -f
find . -name \*.so | xargs rm -f
find . -name .libs -a -type d|xargs rm -rf
rm -f libphp.la   	modules/* libs/*
--> ec9a8979871
STEP 3/3: COPY index.php /var/www/html/index.php
COMMIT php-7.2-apache-mysqli-ext
--> d17ae4015b7
Successfully tagged localhost/php-7.2-apache-mysqli-ext:latest
d17ae4015b7c0af918e9bb5e0d4818e11b394ea3f3f7481d0a558eaa1cfb443

Now that you've built the base images, you can create your pod with containers. The order you create the containers is not important except when one or more init containers rely on another container. In that case, make sure the creation order is correct. The first container created will also create the lamp pod and provide the port mappings:

$ podman create -t --pod new:lamp -p 8080:80 --init-ctr=once -v ./mydbstuff:/var/lib/mysql:Z -e MYSQL_ROOT_PASSWORD=mypass mariadb-conf sh -c "mysql_install_db -u root && (mysqld -u root &) && apt-get update && apt-get install -y curl && curl --output - -L  https://github.com/baude/mysql-example/blob/main/world.sql.gz?raw=true  | gunzip -cd | mysql -u root --password='${MYSQL_ROOT_PASSWORD}'"
C14d30bcc6eed259236916c230c0faf64123b9855160c3895c87a77cb49df653

It happens that the first container is an init container that is flagged to run once. It sets up the database volume, including preloading it with data. The next container is the regular database volume that runs inside the pod:

$ podman create --pod lamp -t -v ./mydbstuff:/var/lib/mysql:Z mariadb-conf
Aa7a013f0bb1b748df1a0a1f1e691824cd9e6334f65129312514b513a86e8a51

Like the init container, the database container also has a bind mount for mydbstuff where the init container populates it with content. The next container is also an init container, but it runs each time the pod starts. At each startup, this container clones a simple website to pick up any changes. The content is stored on the mywebstuff volume:

$ podman create -t --pod lamp --init-ctr=always -v ./mywebstuff:/var/www/html:Z php-7.2-apache-mysqli:latest sh -c "git clone http://github.com/baude/php-example /var/www/html || cd /var/www/html && git pull origin"

And the last container is the Apache and MySQL container. Like its init container, it bind mounts mywebstuff to serve the content from the Git clone:

$ podman create -t --pod lamp  -v ./mywebstuff:/var/www/html:Z php-7.2-apache-mysqli:latest
Ea5821fdd4cde1d4de197ff3269fb5f89875fd2f78aac4b638e2661acbc7695f

The pod is fully created and can now start:

$ podman pod start lamp
2ba762fc1edcbef4a8c607a598d9d0b3219fc49e765ba5fb795516bc66e73eac
$ podman pod ps
POD ID    	NAME    	STATUS  	CREATED     	INFRA ID  	# OF CONTAINERS
2ba762fc1edc  lamp    	Running 	43 seconds ago  a513f190e1a2  4
$ podman ps -a
CONTAINER ID  IMAGE                               	COMMAND           	CREATED     	STATUS                 	PORTS             	NAMES
a513f190e1a2  k8s.gcr.io/pause:3.5                                      	52 seconds ago  Up 29 seconds ago      	0.0.0.0:8080->80/tcp  2ba762fc1edc-infra
aa7a013f0bb1  localhost/mariadb-conf:latest       	mysqld            	45 seconds ago  Up 17 seconds ago      	0.0.0.0:8080->80/tcp  bold_aryabhata
acd05fc2e9af  localhost/php-7.2-apache-mysqli:latest  sh -c git clone h...  37 seconds ago  Exited (0) 18 seconds ago  0.0.0.0:8080->80/tcp  angry_northcutt
ea5821fdd4cd  localhost/php-7.2-apache-mysqli:latest  apache2-foregroun...  33 seconds ago  Up 17 seconds ago      	0.0.0.0:8080->80/tcp  ecstatic_yalow

The pod is now running.

[ Examine 10 considerations for Kubernetes deployments. ]

Notice that the init container for the database is absent because it was a one-shot run. Because the other init container was set to always, it is still present but shows as exited. Using a browser, you should now be able to see a simple query from the website returning information about two cities in Minnesota:

$ curl http://localhost:8080
Returned 2 rows.
 
3837 Minneapolis USA Minnesota 382618
3851 Saint Paul USA Minnesota 287151

Note the tree view of the LAMP project directory. All the database content is in mydbstuff. Likewise for mywebstuff, except for the content to serve:

$ tree
├── lamp.yaml
├── mariadb-conf
│   ├── Containerfile
│   └── my.cnf
├── mydbstuff
│   ├── aria_log.00000001
│   ├── aria_log_control
│   ├── ddl_recovery-backup.log
│   ├── ddl_recovery.log
│   ├── ib_buffer_pool
│   ├── ibdata1
│   ├── ib_logfile0
│   ├── ibtmp1
│   ├── multi-master.info
│   ├── mysql [error opening dir]
│   ├── performance_schema [error opening dir]
│   ├── sys [error opening dir]
│   ├── test [error opening dir]
│   └── world [error opening dir]
├── mywebstuff
│   ├── index.php
│   └── LICENSE
├── php-7.2-apache-mysqli
│   └── Containerfile
├── php-7.2-apache-mysqli-ext
│   ├── Containerfile
│   └── index.php
└── php.yaml

Put it all together: play kube, init containers, and building images

You can continue the example by using a single play kube file to bring up a similar LAMP stack. I removed all existing pods and images from this example so that I can demonstrate play kube's power more fully.

Begin with the Git project context you cloned earlier (my-lamp-project). Using Podman, verify that there are no images or containers present in local storage:

$ podman images
REPOSITORY  TAG     	IMAGE ID	CREATED 	SIZE
$ podman ps -a
CONTAINER ID  IMAGE   	COMMAND 	CREATED 	STATUS  	PORTS   	NAMES

Now play the YAML file in the Git clone. If you try to use the YAML file provided by the Git repository, you have to manually change the paths for the bind mounts as they get preceded with /home/baude. Early in the process, Podman builds the two images:

$ podman play kube lamp.yaml  
STEP 1/2: FROM docker.io/library/mariadb
STEP 2/2: COPY my.cnf /etc/mysql/my.cnf
COMMIT localhost/mariadb-conf:latest
--> e4e4e63a5e4
Successfully tagged localhost/mariadb-conf:latest
e4e4e63a5e4fcd3eff0c5c0901d2dbd3bacee99001a60c06046829944592ea0e
STEP 1/2: FROM docker.io/library/php:7.2-apache
STEP 2/2: RUN docker-php-ext-install mysqli && apt-get update && apt-get install -y git
Configuring for:
PHP Api Version:     	20170718
Zend Module Api No:  	20170718
Zend Extension Api No:   320170718
checking for grep that handles long lines and -e... /bin/grep
...
Installing shared extensions: 	/usr/local/lib/php/extensions/no-debug-non-zts-20170718/
Installing header files:      	/usr/local/include/php/
find . -name \*.gcno -o -name \*.gcda | xargs rm -f
find . -name \*.lo -o -name \*.o | xargs rm -f
find . -name \*.la -o -name \*.a | xargs rm -f
find . -name \*.so | xargs rm -f
find . -name .libs -a -type d|xargs rm -rf
rm -f libphp.la   	modules/* libs/*
Get:1 http://security.debian.org/debian-security buster/updates InRelease [65.4 kB]
Get:2 http://deb.debian.org/debian buster InRelease [122 kB]
Get:3 http://deb.debian.org/debian buster-updates InRelease [51.9 kB]
Get:4 http://security.debian.org/debian-security buster/updates/main amd64 Packages [305 kB]
Get:5 http://deb.debian.org/debian buster/main amd64 Packages [7907 kB]
Get:6 http://deb.debian.org/debian buster-updates/main amd64 Packages [15.2 kB]
Fetched 8466 kB in 2s (4933 kB/s)
Reading package lists...
Reading package lists...
Building dependency tree...
Reading state information...
...
Setting up git (1:2.20.1-2+deb10u3) ...
Setting up xauth (1:1.0.10-1) ...
Processing triggers for libc-bin (2.28-10) ...
Processing triggers for mime-support (3.62) ...
COMMIT localhost/php-7.2-apache-mysqli:latest
--> f0454e4fbfe
Successfully tagged localhost/php-7.2-apache-mysqli:latest
f0454e4fbfe3e59da53255553e019e23380e9006c247a2c218dd4499cb69caf7
Pod:
75aa5664f1e8c2e9b819b8377337dc245911cfbf15e8dc49fb9ee002c8329704
Containers:
c439bffa796f0a7743ad6bcf43e4c88cf485bd7494f5352b237fa9e048dc10c2
2fadb344560a904cb1b0dc2a92fd2cbcfda3c944180c941ffaf4c89793849968

Upon completion, play kube displays the pod ID. If all ran correctly, you should be able to point a web client to your HTTP mapped port, which in turn interacts with the database and returns two results to a precanned query:

$ curl http://localhost:8080
Returned 2 rows.
 
3837 Minneapolis USA Minnesota 382618
3851 Saint Paul USA Minnesota 287151

And now, if you want to tear down all pods associated with your YAML file, you can simply pass the --down flag to play kube:

$ podman pod ps
POD ID    	NAME    	STATUS  	CREATED    	INFRA ID  	# OF CONTAINERS
75aa5664f1e8  lamp    	Running 	4 minutes ago  ae59b18e6dea  5
$ podman play kube --down lamp.yaml  
Pods stopped:
75aa5664f1e8c2e9b819b8377337dc245911cfbf15e8dc49fb9ee002c8329704
Pods removed:
75aa5664f1e8c2e9b819b8377337dc245911cfbf15e8dc49fb9ee002c8329704
$ podman pod ps
POD ID    	NAME    	STATUS  	CREATED    	INFRA ID  	# OF CONTAINERS

Wrapping up

Skip to the bottom of list

Image

Download now

Podman's play kube command continues to get new features. play kube and generate kube are a growth area for the upstream Podman team and feature-rich ways of recording and replaying pods. We would love to hear your ideas for future updates.

The introduction of building images on the fly and using init containers to prepare pod setup and to tear down your pods brings unique use cases to Podman.