GENOME / CONTROLLED TRIALS
Does the knowledge
make a difference?
Same agent. Same task. One change: the skills. Compare what actually happened, including the runs that got worse.
Set up a fair comparison
Choose the knowledge to test, then keep everything else fixed.
What should the candidate know?
Only these lessons are added to the candidate. The baseline has the same task, model and tools.
Loading lessons…
Three bounded practice tasks.
Recover from changed inventory. Verify an asynchronous job. Complete a stable control task. Each uses a public, inspectable tool world.
- Reasoning effort
- Low · both arms
- Action budget
- 8 decisions per arm
- Context
- Fresh per decision; trial history supplied
- World state
- Same fixture and seed
- Isolation
- Bounded tool world
Creating a job does not start model calls. Your provider credentials remain in your local runner.
A controlled test,
with a defined scope.
This evaluates selected instructions in a bounded tool world. The practice tasks are inspectable; they are not an independent hidden benchmark. A result can show a difference on these attempts. It cannot establish that a skill improves every agent or task.
Results are submitted by the local runner. The platform verifies the action sequence and outcome; it does not independently attest which model produced those actions. Submitted traces remain distinguishable from a platform-hosted run.
Experiment notebook
Your private comparisons and experiments others chose to share.
Loading experiments…