AI Agent Decision Making: When a Model Like JEV Helps
A thread on r/AI_Agents this week asked a fair question. Is a decision model like JEV actually useful inside an agent setup, or is it one more dependency?
Here is the short answer. It helps when your agent makes the same small choice many times, the possible answers are known up front, and nobody needs to read the reasoning. Should I call a tool? Which one? Retry or stop? If your agent makes a few calls a day, it adds a moving part and buys you little.
This post does not promise speed or savings. We have not measured JEV on your workload, and nobody else has either. It shows how to find out with your own data, using five free skills.
How AI agent decision making works
Most agents run the same loop. Read the goal. Gather context. Pick the next step. Act. Look at the result, then go again.
The "pick the next step" part hides many tiny decisions. Most setups send each one to the same large model that writes the answers. That works. It is also the slowest and least predictable part of the loop, because a free-text reply has to be parsed back into a choice.
A typed decision model takes a different route. Per its launch coverage, JEV comes from TypeSafe AI. It returns a fixed answer type, such as yes/no, a choice from a list, or a score, with a probability attached. It is not meant for writing, code or open chat.
| Large model call | Typed decision model | |
|---|---|---|
| Output | Free text you parse | A label or score, plus a probability |
| Good for | Plans, explanations, code | Narrow choices that repeat |
| Needs the answers known up front | No | Yes |
| Worth it at low volume | Yes | Usually not |
How we picked
The top guides for this search explain the decision loop well. None show how to decide whether a new decision layer belongs in your loop. We kept skills with a public file and a license on our shelf, picked for that job: question the idea, compare it, gate it, log it, write it down. Every claim below comes from that file.
Step 1: write down what must be true
Assumption Mapper pulls hidden assumptions out of a brief. It looks through four lenses: desirability, feasibility, viability and usability. Each assumption gets two scores from 1 to 5: how load-bearing it is, and how confident you are.
Feed it a short note: "Replace our tool-choice LLM call with a typed decision model." It will surface the guesses you skipped. Are the answers really fixed? Does anyone downstream need the reasoning?
Step 2: compare it against your current call
A/B Test Planner is built for product experiments, but its checklist fits here. It asks for a hypothesis with a direction, one primary metric, up to two guardrail metrics and a rollback plan. Its rule is blunt: no test without a directional hypothesis.
Make the primary metric "correct decisions on labelled records". Make latency and cost the guardrails. Then you are comparing on your own data, not on a launch post.
Step 3: put the decision at the tool boundary
Most loop decisions are about tools. Docker MCP gateway gives you one place to see them. It runs each MCP server in its own container behind a single gateway. It keeps secrets out of environment variables and has built-in logging and call tracing.
That trace is where a "should this call happen?" check can sit, and where you can see what it decided. You need Docker Desktop 4.59 or later with the MCP Toolkit turned on.
Step 4: collect real records from your sessions
A decision model is only as good as the examples you check it on. Context Mode writes a session.log at your project root with everything that happened in a Claude Code session. It also trims long command output to the last 10 lines, the error lines and the exit code.
Those logs are the raw stream. Pull real tool choices from them, label the right answer by hand, and you have the set for Step 2.
Step 5: record the call either way
Architecture Decision Record writes up a technical choice in the Nygard format. It asks for at least two options, the reason for the choice and a status: proposed, accepted, deprecated or superseded.
Write one whether you add the decision model or not. When the next tool like this launches, you will know why you said yes or no, and what would change your mind.
How to start this week
- Pick one decision your agent makes over and over.
- Run Assumption Mapper on the idea of moving it to a typed model.
- Log a week of sessions and hand-label the right answers.
- Plan the comparison with A/B Test Planner. Keep the old call as the baseline.
- Write the result down as an ADR. New to skills? Read how to install a skill.
FAQ
Can AI agents make decisions on their own? Yes, inside the loop you build. You still choose which decisions need a person to approve.
Does a decision model replace the main LLM? No. It takes the small, repeated choices. Planning and writing stay with the large model.
How many examples do I need? Not measured here. Start with real records from your own sessions, not made-up ones, and add more before you trust the result.
Is JEV free? Check the maker's current terms. Every skill above is free to read and copy on AgentAlley, no account needed.