- author = "Paul Brookes and Vardan Voskanyan and Rafail Giavrimis and Matthew Truscott and Mina Ilieva and Chrystalla Pavlou and Alexandru Staicu and Manal Adham and Will Evers-Hood and Jingzhi Gong and Kejia Zhang and Matvey Fedoseev and Vishal Sharma and Roman Bauer and Zheng Wang and Hema Nair and Wei Jie and Tianhua Xu and Aurora Constantin and Leslie Kanthan and Michail Basios",
- title = "Evolving Excellence: Automated Optimization of {LLM}-based Agents",
- howpublished = "arXiv 2512.09108",
- year = "2025",
- month = "9 " # dec,
- keywords = "genetic algorithms, genetic programming, ARTEMIS",
- primaryclass = "cs.SE",
-
URL = "
https://arxiv.org/abs/2512.09108",
- size = "27 pages",
-
abstract = "Agentic AI systems built on large language models
(LLMs) offer significant potential for automating
complex workflows, from software development to
customer support. However, LLM agents often under
perform due to suboptimal configurations; poorly tuned
prompts, tool descriptions, and parameters that
typically require weeks of manual refinement. Existing
optimization methods either are too complex for general
use or treat components in isolation, missing critical
interdependencies.
We present ARTEMIS, a no-code evolutionary optimization platform that jointly optimizes agent configurations through semantically-aware genetic operators. Given only a benchmark script and natural language goals, ARTEMIS automatically discovers configurable components, extracts performance signals from execution logs, and evolves configurations without requiring architectural modifications.
We evaluate ARTEMIS on four representative agent systems: the ALE Agent for competitive programming on AtCoder Heuristic Contest, achieving a improvement in acceptance rate; the Mini-SWE Agent for code optimization on SWE-Perf, with a statistically significant 10.1 percent performance gain; and the CrewAI Agent for cost and mathematical reasoning on Math Odyssey, achieving a statistically significant reduction in the number of tokens required for evaluation. We also evaluate the MathTales-Teacher Agent powered by a smaller open-source model (Qwen2.5-7B) on GSM8K primary-level mathematics problems, achieving a 22 percent accuracy improvement and demonstrating that ARTEMIS can optimize agents based on both commercial and local models.",
