Power related issues on my desktop.

I've been having this strange issue on my PC where when I power it on, it sometimes won't power up properly. The towers seems to power on, the fans all kick in etc, but none of my peripherals will power on. It usually takes a few tries before the computer will power on properly. If I turn the PC off by the power button while this problem is occurring, it will instantly power down.

I've replaced the PSU, and this hasn't helped. I've reseated all of the power cables from the PSU. I've tried using a new power cable. I've cleaned my tower with a spray duster.

I can't remember how long I've had this problem as I mostly ignored it, but it seems to be getting worse. I thought it might be related to the power button, as I was having a problem where slight nudges to the desk would cause the PC to restart, so I pulled the wire out from the reset button, and this fixed my issue. I thought maybe I'd damaged the power button, but I'm also having this issue if I restart from within Windows, so that seems unlikely?

I may try reseating the RAM next but I have a big CPU cooler blocking the RAM so it's going to be a huge pain in the ass.

System Specs:

Intel i5 4690k

16GB RAM

GTX 980

ASUS Z97 PRO-GAMER Motherboard

120GB SSD

2TB HDD

Noctua DH-14 CPU Cooler

Windows 10 64-bit

I think that's everything, just let me know if there's anything else you need to know. I appreciate any and all help!