Why would a society allow a body stealing ancestry?

So I'm making my first ever sci-fantasy setting for a GURPS game I want to run, and I have had the idea of Elves being based off Cordyceps

This means that Elves are fungal in nature, a product of a host, and the main thing that they take your body. The host's features become more pointed and more typically elf like as the host ages and mushrooms grow, until eventually the body is completely fungal

Now the issue I'm dealing with: I kind of want Elves to still be accepted members of an intergalactic society in spite of the fact that they end up taking over and killing people to make more of them.

They don't have to be liked, I'm more than happy to have a bit hatred for them, but I want them to still be able to function among humans and others. This is why I ask for ideas or potential solutions of you all

A few ideas I had to help this:

- Not all Elves go full take over route, some act more as symbiote to their host. This works but kind of further implies the ones taking people over are evil and shouldn't live at all - There have full elven heroes in the past, in spite of their origin they still chose to be good. This is better but I feel it doesn't fully justify their continued existence