Upgrading
<span class="p-notification__title">Note:</span>
<p class="p-notification__message">This page describes the general upgrade process. It is important to follow the specific upgrade pages for each release, as these may include additional steps and workarounds for safely upgrading.</p>
It is recommended that you keep your Kubernetes deployment updated to the latest available stable version. You should also update the other applications which make up the Charmed Kubernetes. Keeping up to date ensures you have the latest bug-fixes and security patches for smooth operation of your cluster.
New minor versions of Kubernetes are set to release once per quarter. You can check the latest release version on the Kubernetes release page on GitHub. Charmed Kubernetes is kept in close sync with upstream Kubernetes: updated versions will be released within a week of a new upstream version of Kubernetes.
<span class="p-notification__title">Note:</span>
<p class="p-notification__message"><strong>Kubernetes</strong> will automatically handle patch releases. This means that the cluster will perform an unattended automatic upgrade between patch versions, e.g. 1.23.1 to 1.23.2. Attended upgrades are only required when you wish to upgrade a minor version, e.g. 1.22.x to 1.23.x.</p>
You can see which version of each application is currently deployed by running
juju status
The ‘App’ section of the output lists each application and its version number. Note that this is the version of the upstream application deployed. The version of the Juju charm is indicated under the column titled ‘Rev’. The charms may be updated in between new versions of the application.
Before you begin
As with all upgrades, there is a possibility that there may be unforeseen difficulties. It is highly recommended that you make a backup of any important data, including any running workloads. For more details on creating backups, see the separate documentation on backups.
You should also make sure:
- The machine from which you will perform the backup has sufficient internet access to retrieve updated software
- Your cluster is running normally
- Your Juju client and controller/models are running the latest versions (see the Juju docs)
- You read the Upgrade notes to see if any caveats apply to the versions you are upgrading to/from
- You read the Release notes for the version you are upgrading to, which will alert you to any important changes to the operation of your cluster
- You read the Upstream release notes for details of deprecation notices and API changes for Kubernetes 1.24 which may impact your workloads.
It is also important to understand that Charmed Kubernetes will only upgrade and if necessary migrate, components relating specifically to elements of Kubernetes installed and configured as part of Charmed Kubernetes. This may not include any customised configuration of Kubernetes, or user generated objects (e.g. storage classes) or deployments which rely on deprecated APIs.
Specific upgrade instructions
We have never recommended skipping a version in an upgrade and no longer provide generic upgrade instructions for any version of Charmed Kubernetes. Please follow the specific upgrade documentation for the version you are upgrading to.
Supported versions
Older versions
- Upgrading from 1.20.x to 1.21.x
- Upgrading from 1.19.x to 1.20.x
- Upgrading from 1.18.x to 1.19.x
- Upgrading from 1.17.x to 1.18.x
Verify an Upgrade
Once an upgrade is complete and units settle, the output from:
juju status
… should indicate that all units are active and the correct version of Kubernetes is running.
It is recommended that you run a cluster validation to ensure that the cluster is fully functional.
Known Issues
A current bug in Kubernetes could prevent the upgrade from properly deleting old pods. You can see such an issue here:
kubectl get po --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
default nginx-ingress-kubernetes-worker-controller-r8d2v 0/1 Terminating 0 17m
ingress-nginx-kubernetes-worker default-http-backend-kubernetes-worker-5d9bb77bc5-76c8w 1/1 Running 0 10m
ingress-nginx-kubernetes-worker nginx-ingress-controller-kubernetes-worker-5dcf47fc4c-q9mh6 1/1 Running 0 10m
kube-system heapster-v1.6.0-beta.1-6db4b87d-phjvb 4/4 Running 0 16m
kube-system kube-dns-596fbb8fbd-bp8lz 3/3 Running 0 18m
kube-system kubernetes-dashboard-67d4c89764-nwxss 1/1 Running 0 18m
kube-system metrics-server-v0.3.1-67bb5c8d7-x9nzx 2/2 Running 0 17m
kube-system monitoring-influxdb-grafana-v4-65cc9bb8c8-mwvcm 2/2 Running 0 17m
In this case the nginx-ingress-kubernetes-worker-controller-r8d2v has been stuck in the Terminating state for roughly 10 minutes. The workaround for such a problem is to force a deletion:
kubectl delete po/nginx-ingress-kubernetes-worker-controller-r8d2v --force --grace-period=0
This will result in output similar to the following:
warning: Immediate deletion does not wait for confirmation that the running resource has been terminated. The resource may continue to run on the cluster indefinitely.
pod "nginx-ingress-kubernetes-worker-controller-r8d2v" force deleted
You should verify that the pod has been sucessfully removed:
kubectl get po --all-namespaces
NAMESPACE NAME READY STATUS RESTARTS AGE
ingress-nginx-kubernetes-worker default-http-backend-kubernetes-worker-5d9bb77bc5-76c8w 1/1 Running 0 11m
ingress-nginx-kubernetes-worker nginx-ingress-controller-kubernetes-worker-5dcf47fc4c-q9mh6 1/1 Running 0 11m
kube-system heapster-v1.6.0-beta.1-6db4b87d-phjvb 4/4 Running 0 17m
kube-system kube-dns-596fbb8fbd-bp8lz 3/3 Running 0 19m
kube-system kubernetes-dashboard-67d4c89764-nwxss 1/1 Running 0 19m
kube-system metrics-server-v0.3.1-67bb5c8d7-x9nzx 2/2 Running 0 18m
kube-system monitoring-influxdb-grafana-v4-65cc9bb8c8-mwvcm 2/2 Running 0 18m