Refine Argo CD configuration for kube-prometheus-stack to clarify CRD management and server-side apply limitations, ensuring compatibility and preventing last-applied-configuration overflow during deployments.

clusters/noble/bootstrap/argocd/app-of-apps/kube-prometheus-application.yaml

@@ -24,8 +24,9 @@ spec:
     server: https://kubernetes.default.svc
     namespace: monitoring
   # Manual sync: Ansible helm runs first; enable automation after cutover (see ../README.md §5).
-  # ServerSideApply: helps large manifests Argo applies directly (CRDs are handled via **helm.skipCrds** +
-  # chart **crds.upgradeJob** server-side apply — see values.yaml).
+  # ServerSideApply: avoids oversized **last-applied-configuration** on large chart objects where supported.
+  # CRDs: **helm.skipCrds** (Argo must not `--include-crds`). Do not enable chart **crds.upgradeJob** under Argo —
+  # its hook ConfigMap exceeds the same annotation limit; CRD bumps via Ansible **helm upgrade** (see values.yaml).
   syncPolicy:
     syncOptions:
       - CreateNamespace=true

clusters/noble/bootstrap/kube-prometheus-stack/values.yaml

@@ -24,14 +24,17 @@ prometheusOperator:
     certManager:
       enabled: true
 
-# CRDs: Argo CD applies chart CRDs with client-side apply → **last-applied-configuration** can exceed 256KiB.
-# **`crds.upgradeJob`** runs a pre-sync Helm hook that applies CRDs with **`kubectl apply --server-side`**.
-# Pair with **`helm.skipCrds: true`** on the Argo Application so Argo does not patch those CRDs itself.
+# CRDs + Argo CD: **`helm.skipCrds: true`** on the Argo Application avoids Argo rendering chart CRDs with
+# **`--include-crds`** (client-side apply would overflow **last-applied-configuration** on huge CRDs).
 # Ref: https://github.com/argoproj/argo-cd/issues/11269
+#
+# Do **not** enable **`crds.upgradeJob`** while this release is Argo-managed: the hook creates ConfigMap
+# **`kube-prometheus-crds-upgrade`** whose **binaryData** is enormous; Argo client-side apply repeats the same
+# annotation size limit on that object. Keep the job **off**; upgrade Prometheus Operator CRDs when you bump
+# the chart via **Ansible `helm upgrade`** (or the chart’s manual CRD steps), not via Argo sync.
 crds:
   upgradeJob:
-    enabled: true
-    forceConflicts: true
+    enabled: false
 
 # --- Longhorn-backed persistence (default chart storage is emptyDir) ---
 alertmanager: