home-server/talos/runbooks/rbac.md

Runbook: Kubernetes RBAC (noble)

Headlamp (clusters/noble/bootstrap/headlamp/values.yaml): the chart’s ClusterRoleBinding uses the built-in edit ClusterRole — not cluster-admin. Break-glass changes use kubectl with an admin kubeconfig.

Headlamp OIDC + kube-apiserver (401 on /clusters/main/version, 200 on /me)
Headlamp sends your IdP JWT to the Kubernetes API. /me is answered by Headlamp; /clusters/.../version is proxied to kube-apiserver. 401 there means authentication failed at the API server (RBAC would normally be 403 after a successful auth). You must:

1. Roll out Talos control-plane config that sets cluster.apiServer.extraArgs for the same Authentik app as Headlamp — see the second patches entry in talos/talconfig.yaml (oidc-issuer-url, oidc-client-id: headlamp, oidc-username-claim, oidc-groups-claim). After edits: talhelper genconfig -o out, then talosctl apply-config on each control plane (rolling).
2. Ensure control planes can reach https://auth.apps.noble.lab.pcenicni.dev/... (JWKS / discovery). If that URL is unreachable from nodes, OIDC validation fails.
3. Apply cluster RBAC for OIDC groups: kubectl apply -k clusters/noble/bootstrap/headlamp (includes oidc-noble-admins-clusterrolebinding.yaml). Your user must be in Authentik group noble-admins and the id_token should carry a groups claim if you rely on that binding.

Headlamp OIDC: nodes / CPU-memory metrics / plugins look broken (403 or empty)
The chart binds only the pod ServiceAccount to headlamp-metrics-reader unless you also bind your IdP group. metrics-clusterrolebinding.yaml binds noble-admins to the same additive ClusterRole as the SA (metrics API, nodes, read-only CRDs). Without metrics-server (or another metrics.k8s.io provider), CPU and memory stay empty even with RBAC. Plugin catalogs that load from the public internet can still fail from the browser (network, ad blockers) unrelated to RBAC.

Quick discovery check (any machine with DNS to Authentik):

curl -fsS "https://auth.apps.noble.lab.pcenicni.dev/application/o/headlamp/.well-known/openid-configuration" | head -c 400; echo

Argo CD (clusters/noble/bootstrap/argocd/values.yaml): policy.default: role:readonly — new OIDC/Git users get read-only unless you add g, <user-or-group>, role:admin (or another role) in configs.rbac.policy.csv. Local user admin stays role:admin via g, admin, role:admin.

Audits

kubectl get clusterrolebindings -o custom-columns='NAME:.metadata.name,ROLE:.roleRef.name,SA:.subjects[?(@.kind=="ServiceAccount")].name,NS:.subjects[?(@.kind=="ServiceAccount")].namespace' | grep -E 'NAME|cluster-admin|headlamp|argocd'

References: Headlamp chart RBAC, Argo CD RBAC.