Terra 6.1 is practically dead on arrival; Luna often beats it at a fraction of the cost in many cases, while being faster

Since the labs like to cherry pick benchmarks to showcase their new models in a better light (as well as mislead and confuse by inverting the X axis, not starting the Y axis from 0, selectively using linear/log for cost axis depending on which fits their narrative better, and so on), I thought it'd be a fun exercise to do the opposite: cherry pick the worst results to make 6.1 Terra look as terrible as possible. Thankfully, the model's questionable cost efficiency makes this quite an easy job.

(this isn't meant to be an accurate assesment of 6.1 Terra, for example it very clearly runs circles around Luna in Terminal Bench 4.0)

[FrontierCode 1.1](https://artificialanalysis.ai/evaluations/frontiercode?models=gpt-6-1-terra%2Cgpt-6-luna) - FYI this benchmark replaced DeepSWE 1.1 in Artificial Analysis' benchmarks 4 days ago. - Luna 6 max beats Terra 6.1 max while being 91% cheaper and using less tokens. Time per task is 42% lower for Luna.

Humanity's last exam: 6.1 Terra medium 40%, 6 Luna max 39%

AutomationBench-AA: 6.1 Terra max 60%, 6 Luna max 53%

AA-Briefcase 1.1: 6.1 Terra max 1402, 6 Luna max 1336

GDPval-AA v2.1: 6.1 Terra max 1476, 6 Luna max 1445

Long context reasoning, AA-LCR v1.1: Luna 6 max at the same level as 6.1 Terra max.

[SciCode Bench](https://artificialanalysis.ai/evaluations/scicode?models=gpt-6-luna%2Cgpt-6-1-terra) is currently flagged as having possible errors so the results aren't fully complete, but even then: Luna only loses by 0.5%, but uses half the output tokens, while being 95% cheaper.

In short, in some workflows you may be using significantly more tokens, billed at a higher price/usage, all while achieving the same or worse results you would have gotten with 6 Luna. Taking 6.1 Sol into consideration too, Terra sticks out as a cost efficiency misfit. Case in point: on TBench 4.0: - 6.1 Sol (xhigh) - 54% - $1.03 - 57K tokens - 6.1 Terra (max) - 55.6% - $1.49 - 101K tokens

For coding, even if Terra has 2x faster tok/s, why would you ever pay almost 50% more to perform at the same level, at the same time per task?