Silencing a kernel that never stops talking

The session began as a container chore: reboot one DNS resolver, stop the other, bring both back. Nothing was broken. What makes the session worth reading is how it felt: every state-changing command queued at an approval gate and waited for a tap before it ran.

One approval attempt even expired, because the agent's own word does not count as the operator's. The system wanted the formal confirmation, and it got it.

With DNS redundancy restored, the operator asked for a performance review of the host logs. The answer came back almost entirely green on a 56-core box: no memory pressure, no throttling, no disk errors, load barely awake.

One yellow flag stood out: the kernel audit subsystem was flooding its own log drain every hour, with around thirty containers generating syscall audit events nobody reads. The first suggested fix, a sysctl rate limit, survives only until reboot. The second, disabling the audit daemon, hit a wall: there is no audit daemon on this host. The unit does not exist.

That wall was the real finding. The kernel config says CONFIG_AUDIT=y: audit is compiled in, not loaded as a module and not run by any service. A userspace tool cannot switch off something the kernel does at boot.

The kernel was built with audit support compiled in and no userspace daemon consuming the events. Every syscall audit record went to the kernel log buffer faster than it could drain, so the kernel suppressed batches and complained about it hourly.

On a compliance-driven machine that trade can be correct. On a self-hosted hypervisor, it is CPU cycles and log volume spent on data nobody will ever read.

The only off switch for compiled-in audit is the boot command line. The essential fragment of the change: GRUB_CMDLINE_LINUX_DEFAULT="… audit=0". Nothing else in the line conflicts; the existing parameters are untouched.

The agent cannot edit system boot config from the session, so the exact edit, update-grub, and the reboot caveat were handed to the operator, medium risk because a reboot is needed.

A two-line edit plus a scheduled reboot retires the hourly flood, and the host loses nothing it was using. The kernel boot parameters, old and new, were documented in the wiki for the next person.

The approval gates did their job across the whole session: four state changes, four confirmations, zero surprises, and one honest thread where the agent owned a repeated tool mistake instead of blaming the API.