Somehow my RTX 4090 has low Clock speed

Yea, as I was trying to figure out why Monster Hunter Wilds was running way worse than it should be, I stumbled on something quite mysterious. If I reinstall my driver (just by clicking it in the NVIDIA App - used to be called NVIDIA Experience), and start up the game my frames essentially double, if not tripple. But this only lasts until I shut down my PC, then I need to reinstall my driver again.

I noticed that these performance issues are not just when playing Monster Hunter, but all the time it seems! This must have happend recently, as I don't remember having such a bad performance before, so I dug deeper using nvidia-smi -a in cmd and found something ...

These are my Max Clock values after a fresh start of my PC:

Clocks

Graphics : 210 MHz

SM : 210 MHz

Memory : 405 MHz

Video : 1185 MHz

And \*\*these\*\* are my Max Clock values after I reinstall my GPU drivers:

Max Clocks

Graphics : 3120 MHz

SM : 3120 MHz

Memory : 10501 MHz

Video : 2415 MHz

What the actual fuck!?

I've been digging around how this is possible or if I am able to set these manually but no luck. Has anyone else seen this behavior before? Please help.