Starting with Fedora 27 Atomic Host, the RPMs for Kubernetes, Flannel and Etcd are no longer included in the host’s image, but are installable instead either as system containers or via package layering.
System containers can serve as drop-in replacements for components that had been included in the Fedora Atomic image. Once installed, these components will be manageable using the same
systemctl commands that apply to regular RPM-installed components. System containers are very flexible; you can easily run system container images based on CentOS, or on older (or newer) versions of Fedora on a Fedora 27 Atomic host.
Package layering makes it possible to install regular RPM packages from configured repositories. These additional
layered packages are persistent across upgrades, rebases, and deploys. You must typically reboot after layering on packages, and not all packages may be installed in this way. For instance, RPMs that install content to
/opt aren’t currently installable via package layering. Unlike with system containers, the packages you layer onto your host must be compatible with the version of Fedora the host is running.
If you’re running a Kubernetes cluster on Fedora Atomic Host that depends on the baked-in versions of these components, such as a cluster installed via the Ansible scripts in the kubernetes/contrib repo, you’ll need to choose one of these methods to migrate your cluster when upgrading to Fedora Atomic 27.
Migrating Kubernetes and related components using System Containers
To replace Kubernetes, Flannel, and Etcd with system containers, you would run the following commands. You could run these commands on a Fedora 26 Atomic Host, and then upgrade to 27. Upon rebooting, your components and any cluster based on them should be up and running.
System containers for master nodes
# atomic install --system --system-package=no --name kube-apiserver registry.fedoraproject.org/f27/kubernetes-apiserver # atomic install --system --system-package=no --name kube-controller-manager registry.fedoraproject.org/f27/kubernetes-controller-manager # atomic install --system --system-package=no --name kube-scheduler registry.fedoraproject.org/f27/kubernetes-scheduler
kube-apiserver system container provides the
System containers for worker nodes
# atomic install --system --system-package=no --name kubelet registry.fedoraproject.org/f27/kubernetes-kubelet # atomic install --system --system-package=no --name kube-proxy registry.fedoraproject.org/f27/kubernetes-proxy
System container for etcd
# atomic install --system --system-package=no --storage=ostree --name etcd registry.fedoraproject.org/f27/etcd
When installed with the name etcd, the etcd system container expects to find stores etcd data in
/var/lib/etcd/etcd.etcd. The etcd RPM is configured by default to store data in
/var/lib/etcd/default.etcd, and the ansible scripts in kubernetes/contrib use
/var/lib/etcd. On a system running etcd as configured by the kubernetes/contrib ansible scripts, you’d move your data as follows:
# systemctl stop etcd # cp -r /var/lib/etcd/member /var/lib/etcd/etcd.etcd/
Note: the etcd container provides the
etcdctl client. There will be more about the etcd container on this blog on Friday.
System container for flannel
# atomic install --system --system-package=no --name flanneld registry.fedoraproject.org/f27/flannel
Updating system containers
System container updates are independent of host updates. You can update a system container by pulling an updated version of the image, and then running the
atomic containers update command.
# atomic pull registry.fedoraproject.org/f27/etcd # atomic containers update etcd
You can then roll back to the previous system container version by running
atomic containers rollback.
# atomic containers rollback etcd
Migrating Kubernetes and related components using RPM Package Layering
During the upgrade to 27, you can opt to layer on particular packages by appending
--install PACKAGE to the
rpm-ostree rebase commands. Upon rebooting into 27, your components and any cluster based on them should be up and running.
To layer packages on master and etcd nodes, run the following command:
# rpm-ostree rebase fedora-atomic-27:fedora/27/x86_64/atomic-host --install kubernetes-master --install flannel --install etcd -r
To layer packages on worker nodes, run the following command:
# rpm-ostree rebase fedora-atomic-27:fedora/27/x86_64/atomic-host --install kubernetes-node --install flannel -r
Updating package layers
During regular rpm-ostree upgrades (with
rpm-ostree upgrade or
atomic host upgrade), your host will fetch updated package versions from your configured repositories.