Zero-shot homogeneous LLM teams are baselined against a trained MARL reference policy.
Alem leaderboard
Model leaderboard
Zero-shot 3-agent LLM teams and trained MARL baselines on the Alem benchmark, separating base-task competence from explicit coordination. ← Back to the full overview
Coordination reward is reported independently from base reward and total return.
17.5% Coord.% on Hard difficulty, close to our 1B-step MARL baseline on this metric (different interface, same underlying environment).
| # | Model | Type / date | Harness | Base% | Coord.% | Total% |
|---|
Click any LLM score for the run config and how-to-run command. Bars show 95% bootstrap confidence intervals; dots show means. Base%, Coord.% and Total% are normalised independently, so Total% is not the sum of the other columns. MARL 100M / 1B are environment training steps, not model parameters. On Hard, mixed-model teams (marked mix) join the table unranked; click one for the member breakdown. Submit a result →
Show the MARL baseline table
| # | Algorithm | Base% | Coord.% | Total% |
|---|
Trained MARL baselines (IPPO, HyperMARL-IPPO, MAPPO, PQN-VDN), ~25M-parameter recurrent policies. Means with 95% bootstrap CIs across 5 seeds.