Can racial allegory be done well?

I am trying to write a fantasy novel and I have been interested in the idea of using alllegory as a way to explore descrimination. However I have heard that often racial allegory can be racist and can in a way justify descrimination. Obvious examples being Orcs goblins ect. I felt kind of discouraged by the idea that this is a trope that should never be explored or is inherently racist. Keep in mind my characters are not trying to be a allegory for any specific race but as a woman of color there are some things I have gone through that I wanted to explore in my writing. Is there a right or wring way to do it? What pitfalls should I watch out for?