Did the Grok models in Cursor quietly get better, or is it just a feeling?

From the outside you usually cannot know. Providers and editors can change the model version, the system prompt, the context the editor sends, or the reasoning settings without a public note, and any of those can make the same model name behave differently.

A feeling based on a few sessions is weak evidence, because the tasks themselves differ day to day. An easy week makes any model look sharper. The only way to answer the question for your own work is to rerun the same tasks and compare.

Keep a small fixed set of 10 to 20 tasks from your real repo, each with a clear pass condition: tests pass, the bug is gone, the diff stays inside the files it should touch. Run the set when you want to judge a model, and run each task more than once, since agent results vary between runs.

Write down the date, model name and pass rate each time. After a few rounds you have your own history, and the next "did they buff it?" question takes ten minutes instead of a debate.

Listings mentioned

• Evals · skill by danielmiessler
  Builds an input-to-assert test set with deterministic checks and reports pass rates across repeated runs, the fixed task set described above.
• LLM evaluation · skill by wshobson
  Covers automated metrics, benchmarking and A/B comparison for judging whether one model or version is actually better.

Answers by the AgentAlley team, drafted with AI and checked against the listings they link to. Not a real-person reply from the original thread.