Running Minikube on Apple Silicon with Corporate VPN

After changing my company laptop to a MacBook Pro with Apple silicon, I had to overcome difficulties setting up a working Minikube environment. A combination of two factors created the challenges:

  1. Supported drivers: Once you remove the drivers, which do not have support for Apple Silicon or require a separate license for commercial use, only QEMU and SSH remain as possible options.

  2. Cisco AnyConnect VPN with Umbrella: Cisco AnyConnect installs a local dns proxy listening on port 53. Unfortunately, QEMU has issues with the Cisco AnyConnect setup. (see QEMU stops working with minikube. The suggested workaround to install socket_vmnet is marked as experimental, and I couldn’t get it to work on my machine.

Terminal showing log out saying that there was a DNS error.

As an act of desperation, I wanted to try out Minikube with podman as its driver. Unfortunately, Minikube does not list podman as a supported driver for macOS, but I had nothing to lose.

And it works like a charm 🎉! Follow these steps to setup minikube with the podman driver on your Apple computer. (It works on Intel Macs as well.)

  • podman
    • client version: 4.3.1
    • server version: 4.3.0
  • Minikube
    • version: v1.28.0
    • commit: 986b1ebd987211ed16f8cc10aed7d2c42fc8392f

Setup podman

  1. Install podman with brew: brew install podman
  2. Initialize podman and prefer rootful container execution : podman machine init --rootful --cpus 4 --memory 8192
  3. Start podman: podman machine start

Note:

  • rootful resolves various network problems, when a pod needs to access external services.
  • Minikube minimal requirements: 2 CPUs or more, 2GB of free memory 20GB of free disk space

Setup Minikube

  1. Install Minikube with brew: brew install minikube
  2. Start minikube: minikube start --driver=podman --cpus=max

Note:

  • If you run into strange issues during minikube start, clean-up with minikube delete --all --purge and try again.
  • Set podman as the default driver: minikube config set driver podman

Verify Setup

  1. Create a new pod: kubectl run test --image=docker.io/alpine:3 -- sleep 86400
  2. Check Internet connectivity with wget: kubectl exec test -- wget -S --spider https://www.google.com
  3. Remove test pod: kubectl delete pod test