Ray Tracing pop-in flickering

I recently upgraded from a gtx 1070 to an RTX 3080 ti. I am experimenting with the new features offered and I already made a post on DLSS. So in multiple games now, when I turn on Ray Tracing, I notice lights flickering in unnatural ways and a lot of really close shadow pop-in. So in The Witcher 3, RT off, the image is smooth while riding Roach. When I turn on RT, performance or quality mode doesnt matter, I am getting a ton of foliage shadow pop-in all over the place from 20 feet in front of me. While the RT lighting adds an amazing amount of depth to the image, it seems to be very range limited, and is very distracting. I'm also seeing certain objects change color depending on what angle I look at them, such as cobwebs flickering between purple and not purple.

So is Ray Tracing just an incomplete feature? Does anyone actually turn this setting on for more than just taking screenshots? Do I need to disable other lighting effects settings to prevent conflicts?