Skip to content

Troubleshooting

Below are some fixes for commons issues you may encounter while bootstrapping your cluster

Unable to access the Juno Web Interface

This may be caused by a running firewall on a cluster node blocking the traffic.

See the k3s node requirements for OS specific instructions on disabling the local node's software firewall

Certificate is not yet valid

When accessing the Juno web interface you may see an error with the SSL certificate being "not yet valid" or has expired. This may be due to a large clock skew between one or more of your Juno cluster nodes.

This can be fixed in multiple ways depending on your setup but can normally be resolved with restarting your node's NTP service.

Genesis stuck in Unknown: cluster DNS can't resolve external hosts

Symptoms

The genesis app in ArgoCD shows sync status Unknown with an error like:

lookup github.com on 10.43.0.10:53: server misbehaving

The CoreDNS logs (kubectl logs -n kube-system deploy/coredns) show:

dial udp [2600:xxxx::1]:53: connect: network is unreachable

Cause

The host's /etc/resolv.conf lists only IPv6 nameservers, which the IPv4-only k3s pod network can't reach. This is common with a static IPv4 address that has no DNS server configured.

To check, run:

grep nameserver /etc/resolv.conf

If every entry is IPv6, you're affected. On hosts that use systemd-resolved, check /run/systemd/resolve/resolv.conf instead.

Fix

echo "nameserver 1.1.1.1" | sudo tee /etc/rancher/k3s/resolv.conf
echo 'resolv-conf: /etc/rancher/k3s/resolv.conf' | sudo tee -a /etc/rancher/k3s/config.yaml
sudo systemctl restart k3s
kubectl -n kube-system rollout restart deploy/coredns

Find your connection name with nmcli con show --active, then run:

sudo nmcli con mod "<connection-name>" ipv4.dns "1.1.1.1"
sudo nmcli con up "<connection-name>"  # briefly drops the connection
sudo systemctl restart k3s
kubectl -n kube-system rollout restart deploy/coredns

Tip

In either option, 8.8.8.8 works instead of 1.1.1.1. If your network blocks public DNS, use your router's IPv4 address, which you can find with ip -4 route show default.

Verify

kubectl get application genesis -n argocd

genesis should show Synced / Healthy within about 3 minutes. You can also click Refresh on it in the ArgoCD UI.