Reconcile Optimization¶
By default, an Argo CD Application is refreshed every time a resource that belongs to it changes.
Kubernetes controllers often update the resources they watch periodically, causing continuous reconcile operation on the Application
and a high CPU usage on the argocd-application-controller
. Argo CD allows you to optionally ignore resource updates on specific fields
for tracked resources.
When a resource update is ignored, if the resource's health status does not change, the Application that this resource belongs to will not be reconciled.
System-Level Configuration¶
Argo CD allows ignoring resource updates at a specific JSON path, using RFC6902 JSON patches and JQ path expressions. It can be configured for a specified group and kind
in resource.customizations
key of the argocd-cm
ConfigMap.
Enabling the feature
The feature is behind a flag. To enable it, set resource.ignoreResourceUpdatesEnabled
to "true"
in the argocd-cm
ConfigMap.
Following is an example of a customization which ignores the refreshTime
status field of an ExternalSecret
resource:
data:
resource.customizations.ignoreResourceUpdates.external-secrets.io_ExternalSecret: |
jsonPointers:
- /status/refreshTime
# JQ equivalent of the above:
# jqPathExpressions:
# - .status.refreshTime
It is possible to configure ignoreResourceUpdates
to be applied to all tracked resources in every Application managed by an Argo CD instance. In order to do so, resource customizations can be configured like in the example below:
data:
resource.customizations.ignoreResourceUpdates.all: |
jsonPointers:
- /status
Using ignoreDifferences to ignore reconcile¶
It is possible to use existing system-level ignoreDifferences
customizations to ignore resource updates as well. Instead of copying all configurations,
the ignoreDifferencesOnResourceUpdates
setting can be used to add all ignored differences as ignored resource updates:
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
data:
resource.compareoptions: |
ignoreDifferencesOnResourceUpdates: true
Default Configuration¶
By default, the metadata fields generation
, resourceVersion
and managedFields
are always ignored for all resources.
Finding Resources to Ignore¶
The application controller logs when a resource change triggers a refresh. You can use these logs to find high-churn resource kinds and then inspect those resources to find which fields to ignore.
To find these logs, search for "Requesting app refresh caused by object update"
. The logs include structured
fields for api-version
and kind
. Counting the number of refreshes triggered, by api-version/kind should
reveal the high-churn resource kinds.
Note
These logs are at the debug
level. Configure the application-controller's log level to debug
.
Once you have identified some resources which change often, you can try to determine which fields are changing. Here is one approach:
kubectl get <resource> -o yaml > /tmp/before.yaml
# Wait a minute or two.
kubectl get <resource> -o yaml > /tmp/after.yaml
diff /tmp/before.yaml /tmp/after
The diff can give you a sense for which fields are changing and should perhaps be ignored.
Checking Whether Resource Updates are Ignored¶
Whenever Argo CD skips a refresh due to an ignored resource update, the controller logs the following line: "Ignoring change of object because none of the watched resource fields have changed".
Search the application-controller logs for this line to confirm that your resource ignore rules are being applied.
Note
These logs are at the debug
level. Configure the application-controller's log level to debug
.
Examples¶
argoproj.io/Application¶
apiVersion: v1
kind: ConfigMap
metadata:
name: argocd-cm
data:
resource.customizations.ignoreResourceUpdates.argoproj.io_Application: |
jsonPointers:
# Ignore when ownerReferences change, for example when a parent ApplicationSet changes often.
- /metadata/ownerReferences
# Ignore reconciledAt, since by itself it doesn't indicate any important change.
- /status/reconciledAt
jqPathExpressions:
# Ignore lastTransitionTime for conditions; helpful when SharedResourceWarnings are being regularly updated but not
# actually changing in content.
- .status.conditions[].lastTransitionTime