RedditPP Mascot

RedditPP

r/ClaudeAI • … • u/TheOnlyVibemaster

Is Opus 5.5 nerfed? New benchmark called LiveNerf measures this live

I’ve been tracking Opus 5.5 since day one to see if Anthropic is nerfing models. Everyday, I re-run its independent benchmarks like GPQA or SWE-bench and then keep track of it in a graph on the repo. As of now, no nerf is detected. I’ll continue monitoring in the coming months and if a nerf is detected, you’ll see it on the graph of the repository. If you’re interested in tracking Opus 5.5 performance, here’s the repo:

https://github.com/ninjahawk/livenerf

I used Claude to choose which questions Opus 5.5 struggles the most on and my control benchmark is against Opus 5. A deviation of more than 7.5 points on LiveNerf would constitute a potential decrease in capability. If everyone starts saying the model is nerfed in the next week or two, I’ll likely be sharing the results again to see if it’s hysteria or if we have some data to back it up.

Attached is the current day to day capability Opus 5.5 has shown. We should have a baseline by day 10, and I won’t be trying to make any real judgments until day 20.

— 0