ADAS
Automated Design of Agentic Systems
Meta-agent search evolves Python agent workflows in code space, scoring each candidate on validation data and reporting on held-out test data.
The loop
- Proposes
A meta-LLM writes complete multi-step agent architectures as executable Python forward() functions.
- Judged by
An automated benchmark harness executes each candidate workflow on validation tasks (ARC, DROP, GSM8K, MMLU) and computes exact accuracy.
- Steers
Winning architectures and their validation scores are appended to an growing archive that conditions the next meta-generation step.
Headline result
Discovered agent architectures that outperform hand-designed baselines by up to 13.6 percentage points on ARC logic tasks and transfer across foundation models.
What this does not prove
Evaluated on validation splits of static benchmarks; executing arbitrary LLM-generated code requires sandboxing and incurs high API token cost.
How it generalizes the harness
The evolutionary archive loop calls the meta-model to synthesize new workflow programs and retains the discovered code artifacts.
Classification
| What it proposes | Program |
|---|---|
| Generative model | GPT-4 meta-agent |
| Algorithmic authority | Sandboxed Python execution and validation benchmark evaluator |
| Feedback | Score |
| What gets updated | Population of candidates and prompt context |
| When it runs | Offline, at design time |
| Validity | Empirical (measured) |
| Evidence | Code released |
| Lineage | Original system |