Which agent harness practices actually hold up, and which are hype?
A summary of dozens of papers on agent harnesses says the setup around the model matters as much as the model: turn past corrections into rules, delete rules you can't prove help, keep the instruction file short, use fewer tools and enforce rules with scripts. Which of these practices have held up in real work?
The practice that holds up best is enforcing rules with code. A line in AGENTS.md or CLAUDE.md is a suggestion the model may skip under pressure; a git hook, linter rule, test or tool-use hook that blocks the bad action works every time. If a rule matters, give it a check.
Short instruction files also hold up. A long list of equally important rules dilutes all of them. Keep the main file as a short index that points to detailed docs the agent reads only when relevant, and add a rule only after you have had to correct the same mistake more than once.
Pruning is the part most people skip. Rules go stale when the code changes, and stale rules quietly mislead the agent. Review the file on a schedule, remove anything you can't tie to a real past failure, and re-test after big changes.
Fewer tools is real too: every tool description costs context and gives the model another wrong option. Treat claims of a fixed improvement number with caution, though; measure on your own tasks before and after.
Listings mentioned
- Writing hookify rules · skill by davila7Turns written rules into hook rules that block or warn on specific agent actions.
- Block no verify hook · skill by wshobsonStops agents from skipping git pre-commit checks with --no-verify.
- /si:memory-review — Analyze Auto-Memory · skill by alirezarezvaniFinds stale or duplicate entries in an agent's memory and suggests what to prune.
Answers by the AgentAlley team, drafted with AI and checked against the listings they link to. Not a real-person reply from the original thread.