Minikube high CPU usage even with no workload on Linux machine
Minikube is a tool that allows you to run Kubernetes for local development. It’s adviced to run all over the k8s documentation, in the tech blogs. Still Minikube looks like a piece of software that is not quite ready for a smooth development experience. At the time of writing minikube’s got a bunch of issues. Some of them are open for a long time, hardly reproducible and quite frustrating. CPU usage with no workload to services is one of them.
Recently I’ve bumped into a problem when an idle minikube process consumes up to 100% of CPU. After a while apiserver stops responding making your cluster not really functioning. It turned out I’m not alone. I run minikube on my Lenovo Thinkpad X1 Carbon machine with 16G RAM and 8 cores (i7 8th gen) on board. I use Ubuntu 18.04 with kvm2 VM driver. So I found this high CPU consumption as quite unexpected and annoying. I’ve solved this problem. But as it sometimes happens in the software development, I’m still not sure what exactly helped. So taking into account some black magic, let’s see what witch brew’s ingredients may help.
Before you begin: monitoring
“If you can’t measure it, you can’t improve it”. Before making any tweaks start measuring CPU usage. Altough top tool is cool, it just cannot beat its successor htop. It allows you to tag processes, filter and sort quickly, see processes and threads in a flat and a tree modes.
On GNU/Linux you most likely want to track QEMU processes resource usage.
Minikube’s dashboard plugin is also a great tool for monitoing resources — per pod (Overview section) or for the whole cluster (Cluster section). To use dashboard, enable the addon and open it:
minikube addons enable dashboard minikube dashboard
Use recommended Virtual Machine drivers
Since I’m on Ubuntu GNU/Linux I went for a kvm2 driver. Get the recommended VM driver for your specific platform.
Follow excellent Arch Linux Wiki KVM article to make sure you have hardware and Linux kernel support for KVM. You may enable virtualisation support in your BIOS and/or load KVM kernel modules needed.
Use latest OS virtualisation libraries, QEMU tools, VM drivers
This is what I believe really fixed my problem. Make sure you:
Update your OS package information:
# in my case of Ubuntu 18.04 I use sudo apt update
Install the latest libs and tools (like libvirt, qemu), enable systemD services needed as per KVM drivers guide
Install the latest driver
Run Minikube with correct VM drivers and resources
If you had already run minikube with the default virtualbox drivers, remove your cluster with:
# don't forget ``--p <your-profile-name>`` # if you had started minikube with custom profile name minikube delete
Make sure you start your new cluster with drivers needed and resources allocation that meets your services requirements. In my case I used more memory and cpus than it’s set by default:
minikube start --vm-driver kvm2 --memory 4096 --cpus 4
Use only the addons you really need
Minikube’s shipped with a dozen of addons like dashboard for web UI or heapster for Grafana-based monitoring. They may be cool and useful, but don’t try to enable them all at once.
In my case enabling efk addons has led to a high CPU usage, so that I have to disable it.
Make sure you enabled only the addons you really need:
# get list of addons minikube addons list # check for addons in the global config too # for the case of recreating your cluster minikube config view
Conclusion
Minikube’s idea is cool. You can really save some money by running k8s cluster on you laptop instead of setting it up in a cloud service for testing. But all the abstractions that Kubernetes introduces, virtualisation with KVM and QEMU on GNU/Linux make Minikube complex. Minikube’s niche (a developer’s local machine) prevents it from being quickly fixed as, say, cloud production-ready solutions that earn their owners piles of money. So let’s hope that magic I’ve posted here will work out for you too. I also hope the issue with high CPU usage will be fixed by someone with proper skills.