Is Claude Getting Worse? How to Measure It Yourself
Every few weeks the same thread shows up on r/ClaudeAI. The model feels dumber, and nobody can prove it. This week someone asked the better question: how do you actually measure it, and how do you spot a quiet downgrade instead of guessing?
Here is the short answer. You cannot prove a change by feel. You need a fixed set of tasks, a saved baseline, and a rerun on a schedule. Then you rule out the boring causes first: a full context window, a different model in the picker, a changed setting. Whatever survives that is a signal worth posting.
This post does not claim Claude got worse, or that it did not. We have not measured it. It shows you how to measure it on your own work, with five free skills.
Why "it feels worse" is hard to prove
Three things look like a model downgrade but are not one.
- Your session. A long chat fills the context window. Early instructions get less attention, and the output drifts.
- The product around the model. In its April 23, 2026 postmortem, Anthropic traced weeks of Claude Code complaints to three product changes: a lower default reasoning effort, a bug that dropped earlier thinking on every turn, and a system prompt line asking for shorter text. Anthropic says the API and inference layer were not affected.
- Your memory. You remember the great answers and forget the bad ones.
| What you notice | Likely cause | Quick check |
|---|---|---|
| Worse late in a long chat | Context fill | Fresh session, same task |
| Worse overnight, for many people | Product or setting change | Model name, effort setting, release notes |
| Worse on one kind of task, every time | Possible real change | Rerun your fixed set against the baseline |
How we picked
The top results for this search either collect complaints or explain context fill. None give you a check you can repeat next month. We picked MIT-licensed skills on our shelf for each step: define good, compare runs, watch over time, keep evidence, get a second opinion. Every claim below comes from the skill's own file.
Step 1: define "good" before you test
AI Product Canvas is a product planning skill, and its evaluation section is exactly what you need. It asks you to name the model and version, pick a primary metric, and set a minimum acceptable threshold. Under monitoring it asks one blunt question: how will you know when model performance degrades? Its anti-patterns list calls out "we'll know it when we see it".
So write it down. Pick a small set of tasks you run every week. Save the prompt, the input, and what a good answer must contain. That set is your baseline.
Step 2: compare before and after like an experiment
A/B Test Planner is built for product tests, but the discipline carries over. It wants a hypothesis with a predicted direction, one primary metric, and up to two guardrail metrics. It also flags when your volume is too low to reach significance, and suggests another approach.
Take that warning seriously. A handful of personal tasks will not give you a significant result. Treat your check as structured evidence, not proof. Primary metric: share of tasks that pass your checklist. Guardrails: answer length and time taken.
Step 3: rerun on a schedule, alert on change from baseline
Monitoring and alerting is written for websites. Its second layer, Correctness, uses synthetic checks: a script runs a known journey and validates the expected text. That is your task set in another form. Its performance rule fits too: alert on regressions from baseline, not on absolute thresholds without one.
Run your set on the same day each week. Same model, same settings, fresh session. Log the date and the exact model name next to each result.
Step 4: rule out your own session first
Context Mode writes a session.log at your project root after every significant action. It also trims long command output to the last 10 lines plus the exit code, so the window fills more slowly.
When an answer looks bad, open the log. If it came late in an overloaded session, rerun the task fresh before you blame the model.
Step 5: get a second opinion from another model
Codex runs the Codex CLI from inside Claude Code. It defaults to the gpt-5.2 model, asks which reasoning effort to use, and starts in a read-only sandbox. You need the Codex CLI installed.
Give it the same failing task. If both models stumble, your prompt or the task is the likely problem. If only one does, your case is much stronger.
How to start this week
- Write five to ten real tasks and what a good answer must include.
- Fill in the evaluation part of AI Product Canvas for them.
- Run them in a fresh session today. Save the outputs as your baseline.
- Rerun weekly and log the model name. Compare pass rates, not feelings.
- New to skills? Read how to install a skill.
FAQ
Is Claude getting worse? Not measured here. Your own baseline is the only answer that fits your work.
Does Claude get worse over time in one chat? It can. A full context window weakens attention to early instructions. Start fresh and compare.
Why does Claude Code feel worse some weeks? Settings and product changes around the model can shift output, as the April 2026 postmortem showed.
Do these skills cost anything? No. Every skill above is free to read and copy on AgentAlley, no account needed.