Weird Crossplay Server Delay.

So this is a weird one. Different actions, like opening a chest, hitting an enemy or hitting/mining an object is delayed. Like, the hit registers almost half a second late. HOWEVER, it goes away at times AND it can be the case for some objects and not for others.

Like I built a wall surrounding our base. Some of the built walls are delayed when I hit it with my first, and the one right next to it isn't. My ping is around 200, but steady.

I can mine a small rock and have it be with the delayed hits, and another small rock be perfectly fine. Some enemies delayed, others, the same type, perfectly registers my hits.

I can swap between hitting the different walls and one of delayed, the other isn't. Does anyone know what this can be? And it's not coincidental timing.