[
 {
  "id": "recommend",
  "order": 1,
  "text": "What we recommend: the OpenEvolve tool for well-defined tasks with a reward it can climb; the Microsoft Agent Framework harness, the best overall but with known cases where it fails completely (0 on its retrieval and vulnerability runs); and DeepSeek Harness as the safer alternative. OpenResearch is still in development."
 },
 {
  "id": "research",
  "order": 2,
  "text": "Hard research is where a swarm pays off. On the legislation questions the Microsoft Agent Framework was right 9 of 14 times and OpenResearch with the Sonnet decider 7 of 14. One plain GLM agent was right 1 of 15 times and GPT-6 Astra alone 1 of 5, though both answered in minutes."
 },
 {
  "id": "short",
  "order": 3,
  "text": "On short, well-specified tasks one GLM agent did about as well as any swarm, faster and for much less: 3 or 4 of 4 right on tax law and data analysis in 1 to 6 minutes for under $2 a run. The swarms were also right on the items they reached, but took 4 to 40 minutes and $1.50 to $13 a run."
 },
 {
  "id": "few",
  "order": 4,
  "text": "The box was torn down while the swarms were on their first item or two of each benchmark, so their short-task points rest on one or two runs each."
 },
 {
  "id": "hard",
  "order": 5,
  "text": "Document retrieval and finding vulnerable files stayed hard for every harness: the best averages were 22 of 100 on retrieval and 0.36 File F1 on vulnerabilities, the latter from a single run."
 },
 {
  "id": "numeric",
  "order": 6,
  "text": "OpenEvolve steered only by the OpenResearch add-on\u2019s judgment, with no cycle count in the search, made the Anthropic kernel 55.7 times faster for about $60 of box time. OpenEvolve with a numeric evaluator was not measured: its one ALE-Bench run, at high reasoning effort, produced no programs, and the low-effort runs never started."
 }
]