Skip to content

v3.5 to 3.6

Breaking Changes

Repo-server OTLP headers environment variable renamed

The repo-server now reads OpenTelemetry collector headers from ARGOCD_REPO_SERVER_OTLP_HEADERS instead of the inconsistently-named ARGOCD_REPO_OTLP_HEADERS. The new name follows the ARGOCD_<COMPONENT>_OTLP_HEADERS convention used by every other Argo CD component (e.g. ARGOCD_SERVER_OTLP_HEADERS, ARGOCD_APPLICATION_CONTROLLER_OTLP_HEADERS) and matches the name the shipped repo-server Deployment already injects from the otlp.headers config map key.

Impact:

  • The default manifests already set ARGOCD_REPO_SERVER_OTLP_HEADERS, so most operators are unaffected.
  • If you set ARGOCD_REPO_OTLP_HEADERS directly (for example via a custom Deployment patch or Helm values), rename it to ARGOCD_REPO_SERVER_OTLP_HEADERS. The old variable is no longer read.

Behavioral Improvements / Fixes

Health transition events now identify the causing resource(s)

The Updated health status event emitted when an Application's aggregated health changes now includes the resource(s) that caused the new status.

New behavior:

The event message gains a (Caused by ...) suffix listing the offending resource(s), for example:

Updated health status: Healthy -> Degraded (Caused by apps/Deployment:default/api)

Up to three resources are listed. This makes it easier to investigate flapping application health by pointing directly at the resource(s) driving the change.

Impact:

  • If you consume this event message and parse it programmatically, update your parser to tolerate the new (Caused by ...) suffix.

Tracing sampler is now parent-based and configurable

Previously, when OpenTelemetry tracing was enabled (--otlp-address set), Argo CD sampled every span unconditionally (AlwaysSample). Argo CD now uses a parent-based head sampler whose root sampling rate is configurable per component via the --otlp-sample-ratio flag (or the ARGOCD_<COMPONENT>_OTLP_SAMPLE_RATIO environment variable). The flag defaults to 1.0 (sample every trace), so for most deployments behaviour is unchanged.

There are two behavioural differences to be aware of:

Note

Sampling is now propagated across services. A sampling decision made by the service that starts a trace (for example the application controller) is honored by every downstream service the trace context flows into (repo-server, commit-server). This keeps each trace whole instead of being partially sampled across process boundaries. Set the same --otlp-sample-ratio on all components to get a consistent rate.

Warning

An inbound "not sampled" decision is now honored. Because the sampler is parent-based, a request that already carries a W3C traceparent marked not sampled is no longer recorded, even with --otlp-sample-ratio=1.0. Under the old AlwaysSample behaviour such requests were recorded regardless. This only affects deployments where an upstream proxy, gateway, or client injects trace context into requests to argocd-server; if nothing upstream propagates traceparent, every trace is still rooted in Argo CD and sampled at the configured ratio.

Values supplied to --otlp-sample-ratio outside the [0.0, 1.0] range (or NaN) are rejected at startup with an error, so a misconfigured flag fails fast rather than silently disabling or exceeding sampling. When the ratio is set via the ARGOCD_<COMPONENT>_OTLP_SAMPLE_RATIO environment variable instead, an invalid or out-of-range value is ignored and the default (1.0) is used (a warning is logged).

Open Telemetry Upgraded

Argo CD v3.6 upgrades the otelgrpc and otelhttp tracing instrumentation to v0.70.0, which bundles semantic conventions v1.42.0 and v1.43.0. Please see those release notes for a list of changed/dropped span attribute names, which may require changes to Grafana dashboards/alerts based on Argo CD OTEL traces.

One specific change to be aware of: otelgrpc now sets the server.address/server.port span attributes from the gRPC dial target rather than the resolved peer IP. If a dashboard or alert keys on server.address expecting a literal pod/service IP for internal Argo CD gRPC calls (argocd-server ↔ repo-server ↔ cmp-server), it may need updating.