Refactor noble cluster configurations by removing deprecated Argo CD application management files and transitioning to a streamlined Ansible-driven installation approach. Update kustomization.yaml files to reflect the new structure, ensuring clarity on resource management. Introduce new namespaces and configurations for cert-manager, external-secrets, and logging components, enhancing the overall deployment process. Add detailed README.md documentation for each component to guide users through the setup and management of the noble lab environment.
This commit is contained in:
31
clusters/noble/bootstrap/kyverno/README.md
Normal file
31
clusters/noble/bootstrap/kyverno/README.md
Normal file
@@ -0,0 +1,31 @@
|
||||
# Kyverno (noble)
|
||||
|
||||
Admission policies using [Kyverno](https://kyverno.io/). The main chart installs controllers and CRDs; **`kyverno-policies`** installs **Pod Security Standard** rules matching the **`baseline`** profile in **`Audit`** mode (violations are visible in policy reports; workloads are not denied).
|
||||
|
||||
- **Charts:** `kyverno/kyverno` **3.7.1** (app **v1.17.1**), `kyverno/kyverno-policies` **3.7.1**
|
||||
- **Namespace:** `kyverno`
|
||||
|
||||
## Install
|
||||
|
||||
```bash
|
||||
helm repo add kyverno https://kyverno.github.io/kyverno/
|
||||
helm repo update
|
||||
kubectl apply -f clusters/noble/apps/kyverno/namespace.yaml
|
||||
helm upgrade --install kyverno kyverno/kyverno -n kyverno \
|
||||
--version 3.7.1 -f clusters/noble/apps/kyverno/values.yaml --wait --timeout 15m
|
||||
helm upgrade --install kyverno-policies kyverno/kyverno-policies -n kyverno \
|
||||
--version 3.7.1 -f clusters/noble/apps/kyverno/policies-values.yaml --wait --timeout 10m
|
||||
```
|
||||
|
||||
Verify:
|
||||
|
||||
```bash
|
||||
kubectl -n kyverno get pods
|
||||
kubectl get clusterpolicy | head
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- **`validationFailureAction: Audit`** in `policies-values.yaml` avoids breaking namespaces that need **privileged** behavior (Longhorn, monitoring node-exporter, etc.). Switch specific policies or namespaces to **`Enforce`** when you are ready.
|
||||
- To use **`restricted`** instead of **`baseline`**, change **`podSecurityStandard`** in `policies-values.yaml` and reconcile expectations for host mounts and capabilities.
|
||||
- Upgrade: bump **`--version`** on both charts together; read [Kyverno release notes](https://github.com/kyverno/kyverno/releases) for breaking changes.
|
||||
5
clusters/noble/bootstrap/kyverno/namespace.yaml
Normal file
5
clusters/noble/bootstrap/kyverno/namespace.yaml
Normal file
@@ -0,0 +1,5 @@
|
||||
# Kyverno — apply before Helm.
|
||||
apiVersion: v1
|
||||
kind: Namespace
|
||||
metadata:
|
||||
name: kyverno
|
||||
62
clusters/noble/bootstrap/kyverno/policies-values.yaml
Normal file
62
clusters/noble/bootstrap/kyverno/policies-values.yaml
Normal file
@@ -0,0 +1,62 @@
|
||||
# kyverno/kyverno-policies — Pod Security Standards as Kyverno ClusterPolicies
|
||||
#
|
||||
# helm upgrade --install kyverno-policies kyverno/kyverno-policies -n kyverno \
|
||||
# --version 3.7.1 -f clusters/noble/apps/kyverno/policies-values.yaml --wait --timeout 10m
|
||||
#
|
||||
# Default profile is baseline; validationFailureAction is Audit so existing privileged
|
||||
# workloads are not blocked. Kyverno still emits PolicyReports for matches — Headlamp
|
||||
# surfaces those as “policy violations”. Exclude namespaces that intentionally run
|
||||
# outside baseline (see namespace PSA labels under clusters/noble/apps/*/namespace.yaml)
|
||||
# plus core Kubernetes namespaces and every Ansible-managed app namespace on noble.
|
||||
#
|
||||
# After widening excludes, Kyverno does not always prune old PolicyReport rows; refresh:
|
||||
# kubectl delete clusterpolicyreport --all
|
||||
# kubectl delete policyreport -A --all
|
||||
# (Reports are recreated on the next background scan.)
|
||||
#
|
||||
# Exclude blocks omit `kinds` so the same namespace skip applies to autogen rules for
|
||||
# Deployments, DaemonSets, etc. (see kyverno/kyverno#4306).
|
||||
#
|
||||
policyKind: ClusterPolicy
|
||||
policyType: ClusterPolicy
|
||||
podSecurityStandard: baseline
|
||||
podSecuritySeverity: medium
|
||||
validationFailureAction: Audit
|
||||
failurePolicy: Fail
|
||||
validationAllowExistingViolations: true
|
||||
|
||||
# All platform namespaces on noble (ansible/playbooks/noble.yml + clusters/noble/apps).
|
||||
x-kyverno-exclude-infra: &kyverno_exclude_infra
|
||||
any:
|
||||
- resources:
|
||||
namespaces:
|
||||
- kube-system
|
||||
- kube-public
|
||||
- kube-node-lease
|
||||
- argocd
|
||||
- cert-manager
|
||||
- external-secrets
|
||||
- headlamp
|
||||
- kyverno
|
||||
- logging
|
||||
- loki
|
||||
- longhorn-system
|
||||
- metallb-system
|
||||
- monitoring
|
||||
- newt
|
||||
- sealed-secrets
|
||||
- traefik
|
||||
- vault
|
||||
|
||||
policyExclude:
|
||||
disallow-capabilities: *kyverno_exclude_infra
|
||||
disallow-host-namespaces: *kyverno_exclude_infra
|
||||
disallow-host-path: *kyverno_exclude_infra
|
||||
disallow-host-ports: *kyverno_exclude_infra
|
||||
disallow-host-process: *kyverno_exclude_infra
|
||||
disallow-privileged-containers: *kyverno_exclude_infra
|
||||
disallow-proc-mount: *kyverno_exclude_infra
|
||||
disallow-selinux: *kyverno_exclude_infra
|
||||
restrict-apparmor-profiles: *kyverno_exclude_infra
|
||||
restrict-seccomp: *kyverno_exclude_infra
|
||||
restrict-sysctls: *kyverno_exclude_infra
|
||||
22
clusters/noble/bootstrap/kyverno/values.yaml
Normal file
22
clusters/noble/bootstrap/kyverno/values.yaml
Normal file
@@ -0,0 +1,22 @@
|
||||
# Kyverno — noble (policy engine)
|
||||
#
|
||||
# helm repo add kyverno https://kyverno.github.io/kyverno/
|
||||
# helm repo update
|
||||
# kubectl apply -f clusters/noble/apps/kyverno/namespace.yaml
|
||||
# helm upgrade --install kyverno kyverno/kyverno -n kyverno \
|
||||
# --version 3.7.1 -f clusters/noble/apps/kyverno/values.yaml --wait --timeout 15m
|
||||
#
|
||||
# Baseline Pod Security policies (separate chart): see policies-values.yaml + README.md
|
||||
#
|
||||
# Raise Kubernetes client QPS/burst so under API/etcd load Kyverno does not hit
|
||||
# "client rate limiter Wait" / flaky kyverno-health lease (defaults are very low).
|
||||
# Two replicas: webhook Service keeps endpoints during rolling restarts (avoids
|
||||
# apiserver "connection refused" to kyverno-svc:443 while a single pod cycles).
|
||||
admissionController:
|
||||
replicas: 2
|
||||
# Insulate Kyverno API traffic via APF (helps when etcd/apiserver are busy).
|
||||
apiPriorityAndFairness: true
|
||||
container:
|
||||
extraArgs:
|
||||
clientRateLimitQPS: 30
|
||||
clientRateLimitBurst: 60
|
||||
Reference in New Issue
Block a user