Skip to main content

๐Ÿ”Ž What kwatch checks for you Pro

Pro

Automatic evidence is part of kwatch Pro, which is free during early access. Community is unchanged. Install and manage licenses with the installer.

When something breaks, kwatch runs the read-only checks you would run with kubectl yourself and shows the result. Only commands that change the cluster stay as commands for you to run. A Pro message therefore carries the confirmed cause, the proof, a log quote and the fix commands. The full picture is in the Evidence tab of the dashboard.

kwatch is source available under ELv2. Every check only reads. Secret and ConfigMap values are never shown, stored or sent; only key names appear. Credentials are hidden in log lines.

What kwatch checks, by scenarioโ€‹

ScenarioChecks
Crash loop or failing containerPod state, previous and current logs, events, probes
Out of memoryPod state, previous logs, current usage, node
Image will not pullImage, pull secrets, events
Pending podPod state, events, node, PVC, quota
Missing Secret or ConfigMap keySecret and ConfigMap key names (never values), pod state
Service or Ingress not answeringIngress, Service, endpoints, backing pods
Admission webhook rejects a changeWebhooks that touch the object, events
Bad rolloutRollout status, the diff against the last healthy revision
Expiring or invalid certificateCertificates
Autoscaling or Job stuckHPA, Job, quota
Node or control plane troubleNode, pods on the node, control plane /readyz

How kwatch correlatesโ€‹

  • Log signature to cluster object. A line such as a refused connection is matched to the Service, Secret or volume it points at.
  • Nearest prior change. The closest earlier change to that object.
  • Spread. How many pods and nodes show the same failure.
  • Rollout diff. What differs from the last healthy revision.

When the evidence proves the cause, the message says it is confirmed and why.

Settingsโ€‹

SettingDefaultWhat it does
investigation.logs (Helm config.investigation.logs)trueReads container logs for evidence. Set false to read none; the dashboard shows the Logs section as turned off.
rbac.readyz (Helm)trueGrants get on the API server's /readyz, for the control plane check.
# Helm values
rbac:
readyz: true
config:
investigation:
logs: true

The Secret and certificate checks use watch.secrets: true, which is the default.

When a read is not allowed, the dashboard says which value to set and shows every other check.

Dashboard API Proโ€‹

GET /api/incidents/{id} (see Dashboard API) adds:

FieldContents
evidenceOne section per check: check, title, status (ok, denied, timeout, absent), optional reason (rbac, off), subject, run (previous or current for logs), facts, lines and matched (indexes of the lines carrying the failure's signature)
correlationconfirmed (why the cause is proven, when it is) and facts labelled Signature, Matches, Change, Spread and Before and after

Both fields are absent in Community.

Audit log Proโ€‹

Each audit entry adds evidence, a list of check:status strings for the checks that ran, and readSteps, the number of read-only commands kwatch ran for you instead of listing them.