In July we put out the first and second releases of Fedora 26 Atomic Host. In this blog post we’ll cover updating an existing Fedora 25 Atomic Host system to Fedora 26. We’ll cover preparing the system for upgrade and performing the upgrade.
NOTE: If you really don’t want to upgrade to Fedora 26 see the later section: Appendix B: Fedora 25 Atomic Host Life Support.
Preparing for Upgrade
Before we update to Fedora 26 Atomic Host we should check to see that we have at least a few GiB of free space in our root filesystem. The update to Fedora 26 can cause your system to retrieve more than 1GiB of new content (not shared with Fedora 25) and thus we’ll need to make sure we have plenty of free space.
NOTE: Upstream OSTree has implemented some filesystem checks to make sure that upgrades will stop themselves before filling up the filesystem and possibly corrupting your system.
The system we are upgrading today is a Vagrant box. Let’s see how much free space we have:
[vagrant@host ~]$ sudo df -kh / Filesystem Size Used Avail Use% Mounted on /dev/mapper/atomicos-root 3.0G 1.4G 1.6G 47% /
1.6G free means we probably need to expand our root filesystem
to make sure we don’t run out of space. Let’s check to see if we have
any free space:
[vagrant@host ~]$ sudo vgs VG #PV #LV #SN Attr VSize VFree atomicos 1 2 0 wz--n- 40.70g 22.60g [vagrant@host ~]$ sudo lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert docker-pool atomicos twi-a-t--- 15.09g 0.13 0.10 root atomicos -wi-ao---- 2.93g
We can see that we have
22.60g free and that our
logical volume is
2.93g in size. We’ll go ahead and increase the
size of the root volume group by 3 GiB.
[vagrant@host ~]$ sudo lvresize --size=+3g --resizefs atomicos/root Size of logical volume atomicos/root changed from 2.93 GiB (750 extents) to 5.93 GiB (1518 extents). Logical volume atomicos/root successfully resized. meta-data=/dev/mapper/atomicos-root isize=512 agcount=4, agsize=192000 blks = sectsz=512 attr=2, projid32bit=1 = crc=1 finobt=1 spinodes=0 rmapbt=0 = reflink=0 data = bsize=4096 blocks=768000, imaxpct=25 = sunit=0 swidth=0 blks naming =version 2 bsize=4096 ascii-ci=0 ftype=1 log =internal bsize=4096 blocks=2560, version=2 = sectsz=512 sunit=0 blks, lazy-count=1 realtime =none extsz=4096 blocks=0, rtextents=0 data blocks changed from 768000 to 1554432 [vagrant@host ~]$ sudo lvs LV VG Attr LSize Pool Origin Data% Meta% Move Log Cpy%Sync Convert docker-pool atomicos twi-a-t--- 15.09g 0.13 0.10 root atomicos -wi-ao---- 5.93g
As part of that command we also resized the filesystem all in one shot. We can see that by checking again the filesystem usage.
[vagrant@host ~]$ sudo df -kh / Filesystem Size Used Avail Use% Mounted on /dev/mapper/atomicos-root 6.0G 1.4G 4.6G 24% /
Now we should be ready for the upgrade. If you are hosting any services on your instance you may want to prepare for them to have some downtime.
NOTE: If you are running Kubernetes you should check out the later section on Kubernetes: Appendix A: Upgrading Systems with Kubernetes.
NOTE: If you are running OpenShift Origin (i.e. via being set up by the openshift-ansible installer) the upgrade should not need any preparation.
Currently we are on Fedora 25 Atomic Host using the
[vagrant@host ~]$ rpm-ostree status State: idle Deployments: ● fedora-atomic:fedora-atomic/25/x86_64/docker-host Version: 25.154 (2017-07-04 01:38:10) Commit: ce555fa89da934e6eef23764fb40e8333234b8b60b6f688222247c958e5ebd5b
In order to do the upgrade we need to add the location of
the Fedora 26 repository as a new remote (similar to a
git remote) for
ostree to know about:
[vagrant@host ~]$ sudo ostree remote add --set=gpgkeypath=/etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-26-primary fedora-atomic-26 https://kojipkgs.fedoraproject.org/atomic/26
You can see from the command that we are adding a new remote known as
fedora-atomic-26 with a remote url of
We are also setting the
gpgkeypath variable in the configuration for
the remote. This tells OSTree that we want commit signatures to be
verified when we download from a remote. This is something new that was
enabled for Fedora 26 Atomic Host.
Now that we have our
fedora-atomic-26 remote we can do the upgrade!
[vagrant@host ~]$ sudo rpm-ostree rebase fedora-atomic-26:fedora/26/x86_64/atomic-host Receiving metadata objects: 0/(estimating) -/s 0 bytes Signature made Sun 23 Jul 2017 03:13:09 AM UTC using RSA key ID 812A6B4B64DAB85D Good signature from "Fedora 26 Primary <email@example.com>" Receiving delta parts: 0/27 5.3 MB/s 26.7 MB/355.4 MB Signature made Sun 23 Jul 2017 03:13:09 AM UTC using RSA key ID 812A6B4B64DAB85D Good signature from "Fedora 26 Primary <firstname.lastname@example.org>" 27 delta parts, 9 loose fetched; 347079 KiB transferred in 105 seconds Copying /etc changes: 22 modified, 0 removed, 58 added Transaction complete; bootconfig swap: yes deployment count change: 1 Upgraded: GeoIP 1.6.11-1.fc25 -> 1.6.11-1.fc26 GeoIP-GeoLite-data 2017.04-1.fc25 -> 2017.06-1.fc26 NetworkManager 1:1.4.4-5.fc25 -> 1:1.8.2-1.fc26 ... ... setools-python-4.1.0-3.fc26.x86_64 setools-python3-4.1.0-3.fc26.x86_64 Run "systemctl reboot" to start a reboot [vagrant@host ~]$ sudo reboot Connection to 192.168.121.217 closed by remote host. Connection to 192.168.121.217 closed.
After reboot we can log in and see the status:
$ vagrant ssh [vagrant@host ~]$ rpm-ostree status State: idle Deployments: ● fedora-atomic-26:fedora/26/x86_64/atomic-host Version: 26.91 (2017-07-23 03:12:08) Commit: 0715ce81064c30d34ed52ef811a3ad5e5d6a34da980bf35b19312489b32d9b83 GPGSignature: 1 signature Signature made Sun 23 Jul 2017 03:13:09 AM UTC using RSA key ID 812A6B4B64DAB85D Good signature from "Fedora 26 Primary <email@example.com>" fedora-atomic:fedora-atomic/25/x86_64/docker-host Version: 25.154 (2017-07-04 01:38:10) Commit: ce555fa89da934e6eef23764fb40e8333234b8b60b6f688222247c958e5ebd5b [vagrant@host ~]$ cat /etc/fedora-release Fedora release 26 (Twenty Six)
We are now on Fedora 26 Atomic Host. Now is a good time to check your
services (most likely running in containers) to see if they are still
working. If not, then you always have the rollback command:
NOTE: Over time you can see updated commands for upgrading Atomic Host between releases by visiting this wiki page.
Appendix A: Upgrading Systems with Kubernetes
Fedora 25 Atomic Host ships with Kubernetes v1.5.3, and Fedora 26 Atomic Host ships with Kubernetes v1.6.7. Before upgrading systems participating in an existing Kubernetes cluster from 25 to 26, there are a few configuration changes to make.
In Kubernetes 1.6, the
--config argument is no longer valid. If
/etc/kubernetes/kubelet point to the manifests
directory using the
--config argument, then you need to change
the argument name to
--pod-manifest-path. Also in
need to add the argument
For example, if your
/etc/kubernetes/kubelet file starts with the
KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig --config=/etc/kubernetes/manifests --cluster-dns=10.254.0.10 --cluster-domain=cluster.local"
Then it should be changed to be:
KUBELET_ARGS="--kubeconfig=/etc/kubernetes/kubelet.kubeconfig --pod-manifest-path=/etc/kubernetes/manifests --cluster-dns=10.254.0.10 --cluster-domain=cluster.local --cgroup-driver=systemd"
Staying With etcd2
From Kubernetes 1.5 to 1.6 upstream
from using version 2 of the etcd API to version 3. The
instructs users to add two arguments to the
This will ensure that any pods, services or other objects stored in etcd will continue to be found by Kubernetes once you’ve completed your upgrade.
Moving To etcd3
If you later wish to migrate your etcd data to the v3 API, stop your etcd and kube-apiserver services and, run the following command to migrate to etcd3:
NOTE: The following command assumes your data is stored in
# ETCDCTL_API=3 etcdctl --endpoints https://YOUR-ETCD-IP:2379 migrate --data-dir=/var/lib/etcd
Then remove the
arguments from the
/etc/kubernetes/apiserver file and restart the etcd
and kube-apiserver services.
Appendix B: Fedora 25 Atomic Host Life Support
We have decided
to keep updating the
ref every day when Bodhi runs within Fedora. A new update will
get created every day. However, we recommend you upgrade to Fedora 26,
because we are focusing future testing and development efforts on on
Fedora 26 Atomic Host and thus the Fedora 25 OSTrees don’t get
The transition to Fedora 26 Atomic Host should be a smooth process. If you have issues or want to be involved in the future direction of Atomic Host please join us in IRC (#atomic on freenode) or on the atomic-devel mailing list.