Run a single experiment iteration

/ar:run — Single Experiment Iteration

by alirezarezvani · MIT license · ★ 26,349 on GitHub · updated Aug 30

Run a single experiment iteration. Edit the target file, evaluate, keep or discard.

---
name: "run"
description: "Run a single experiment iteration. Edit the target file, evaluate, keep or discard. Use when the user runs /ar:run or asks for one manual autoresearch iteration."
command: /ar:run
---

# /ar:run — Single Experiment Iteration

Run exactly ONE experiment iteration: review history, decide a change, edit, commit, evaluate.

## Usage

```
/ar:run engineering/api-speed              # Run one iteration
/ar:run                                     # List experiments, let user pick
```

## What It Does

Overview

Run exactly ONE experiment iteration: review history, decide a change, edit, commit, evaluate.

Manual install (npx, ZIP)

Claude Codenpx skills add alirezarezvani/claude-skills --skill run -g -a claude-code

Claude Code (manual)

mkdir -p ~/.claude/skills && curl -sL https://codeload.github.com/alirezarezvani/claude-skills/tar.gz/19392f7a0826 \
  | tar -xz -C ~/.claude/skills --strip-components=4 claude-skills-19392f7a0826/engineering/autoresearch-agent/skills/run

Windows (PowerShell)

New-Item -ItemType Directory -Force "$HOME\.claude\skills" | Out-Null; Invoke-WebRequest "https://m.agentalley.io/zip/cc3b96577bb9c9a03f03/run.zip" -OutFile "$env:TEMP\run.zip"; Expand-Archive "$env:TEMP\run.zip" -DestinationPath "$HOME\.claude\skills" -Force

Claude.ai / desktopDownload the ZIP and upload it under Settings → Capabilities → Skills.

Codex, Cursor, other agentsPoint them at SKILL.md. The other files are plain text.

Connects with Agent Alley MCP · agentalley.io/mcp

Files included

SKILL.md2.5 KB

Read the files here

alirezarezvani/19392f71 file

SKILL.md2.5 KB

Shared references2SKILL.md reads these from elsewhere in the repo. Not in the ZIP.

Get them on GitHub ↗

SKILL.md

Show the full text85 lines

run-2/SKILL.md85 lines · 2.5 KB

PreviewCode

Outline

RawView on GitHub

/ar:run — Single Experiment Iteration

Run exactly ONE experiment iteration: review history, decide a change, edit, commit, evaluate.

Usage

/ar:run engineering/api-speed              # Run one iteration
/ar:run                                     # List experiments, let user pick

What It Does

Step 1: Resolve experiment

If no experiment specified, run python {skill_path}/scripts/setup_experiment.py --list and ask the user to pick.

Step 2: Load context

# Read experiment config
cat .autoresearch/{domain}/{name}/config.cfg

# Read strategy and constraints
cat .autoresearch/{domain}/{name}/program.md

# Read experiment history
cat .autoresearch/{domain}/{name}/results.tsv

# Checkout the experiment branch
git checkout autoresearch/{domain}/{name}

Step 3: Decide what to try

Review results.tsv:

What changes were kept? What pattern do they share?
What was discarded? Avoid repeating those approaches.
What crashed? Understand why.
How many runs so far? (Escalate strategy accordingly)

Strategy escalation:

Runs 1-5: Low-hanging fruit (obvious improvements)
Runs 6-15: Systematic exploration (vary one parameter)
Runs 16-30: Structural changes (algorithm swaps)
Runs 30+: Radical experiments (completely different approaches)

Step 4: Make ONE change

Edit only the target file specified in config.cfg. Change one thing. Keep it simple.

Step 5: Commit and evaluate

git add {target}
git commit -m "experiment: {short description of what changed}"

python {skill_path}/scripts/run_experiment.py \
  --experiment {domain}/{name} --single

Step 6: Report result

Read the script output. Tell the user:

KEEP: "Improvement! {metric}: {value} ({delta} from previous best)"
DISCARD: "No improvement. {metric}: {value} vs best {best}. Reverted."
CRASH: "Evaluation failed: {reason}. Reverted."

Step 7: Self-improvement check

After every 10th experiment (check results.tsv line count), update the Strategy section of program.md with patterns learned.

Rules

ONE change per iteration. Don't change 5 things at once.
NEVER modify the evaluator (evaluate.py). It's ground truth.
Simplicity wins. Equal performance with simpler code is an improvement.
No new dependencies.

1	---
2	name: "run"
3	description: "Run a single experiment iteration. Edit the target file, evaluate, keep or discard. Use when the user runs /ar:run or asks for one manual autoresearch iteration."
4	command: /ar:run
5	---
6
7	# /ar:run — Single Experiment Iteration
8
9	Run exactly ONE experiment iteration: review history, decide a change, edit, commit, evaluate.
10
11	## Usage
12
13	```
14	/ar:run engineering/api-speed # Run one iteration
15	/ar:run # List experiments, let user pick
16	```
17
18	## What It Does
19
20	### Step 1: Resolve experiment
21
22	If no experiment specified, run `python {skill_path}/scripts/setup_experiment.py --list` and ask the user to pick.
23
24	### Step 2: Load context
25
26	```bash
27	# Read experiment config
28	cat .autoresearch/{domain}/{name}/config.cfg
29
30	# Read strategy and constraints
31	cat .autoresearch/{domain}/{name}/program.md
32
33	# Read experiment history
34	cat .autoresearch/{domain}/{name}/results.tsv
35
36	# Checkout the experiment branch
37	git checkout autoresearch/{domain}/{name}
38	```
39
40	### Step 3: Decide what to try
41
42	Review results.tsv:
43	- What changes were kept? What pattern do they share?
44	- What was discarded? Avoid repeating those approaches.
45	- What crashed? Understand why.
46	- How many runs so far? (Escalate strategy accordingly)
47
48	Strategy escalation:
49	- Runs 1-5: Low-hanging fruit (obvious improvements)
50	- Runs 6-15: Systematic exploration (vary one parameter)
51	- Runs 16-30: Structural changes (algorithm swaps)
52	- Runs 30+: Radical experiments (completely different approaches)
53
54	### Step 4: Make ONE change
55
56	Edit only the target file specified in config.cfg. Change one thing. Keep it simple.
57
58	### Step 5: Commit and evaluate
59
60	```bash
61	git add {target}
62	git commit -m "experiment: {short description of what changed}"
63
64	python {skill_path}/scripts/run_experiment.py \
65	--experiment {domain}/{name} --single
66	```
67
68	### Step 6: Report result
69
70	Read the script output. Tell the user:
71	- KEEP: "Improvement! {metric}: {value} ({delta} from previous best)"
72	- DISCARD: "No improvement. {metric}: {value} vs best {best}. Reverted."
73	- CRASH: "Evaluation failed: {reason}. Reverted."
74
75	### Step 7: Self-improvement check
76
77	After every 10th experiment (check results.tsv line count), update the Strategy section of program.md with patterns learned.
78
79	## Rules
80
81	- ONE change per iteration. Don't change 5 things at once.
82	- NEVER modify the evaluator (evaluate.py). It's ground truth.
83	- Simplicity wins. Equal performance with simpler code is an improvement.
84	- No new dependencies.
85