Did they nerf Opus 5.5? Doesn’t look like it. LiveNerf day 29 update

About a month ago we started LiveNerf as a way to track model capabilities over time. Since then it’s gotten bigger than I’d expected. We’ve accumulated around 1.5k stars on Github and now we’ve collected 29 consecutive days worth of data on the model capabilities of Opus 5.5. If anyone is interested in tracking model performance of Opus 5.5, here’s the repo:

[https://github.com/ninjahawk/livenerf](https://github.com/ninjahawk/livenerf)

As you can see from the 29 collected days of data, no nerf was detected. This weakens the idea that Anthropic nerfs models, or at least that they did not nerf Opus 5.5 within a range that LiveNerf can detect which would be by 6.6 points or more.

We set out to produce unbiased data and determine if Anthropic nerfs their models, I believe we’ve done that.

But the job is not finished.

While Anthropic may not decrease the capabilities of their models over time, other companies might.

I plan to expand LiveNerf to independently track models and APIs through this same scientific approach in the future. Currently we’re in talks for potential funding but nothing is definite yet. If funding is secured, I plan to make this much bigger. We should know in the next few days.

I want to thank everyone for your community involvement so far, I’ll definitely be tracking Fable 5.5 when it’s released in the same way, and probably other models in due time. It’s meant the world to me seeing the community rally behind this in an effort to give independent transparency. I can’t thank you all enough.

Give me some feedback in the comments, do you think that we could’ve done things differently? Maybe you had a problem with the way certain things were performed?