How would I track food from real life into VR?
I'm looking into creating a mixed reality sensory experience where the "player" would be eating something in real life, say sushi, but the headset or another camera would be able to track the food and "replace" it with a CGI model/texture so that it looks like they're eating something different like an alien tentacle or whatever.
I'm unsure how to go about this. I would need some way to track the food that wouldn't break the illusion when someone took a bite out of the food. I think It'd be easy enough to make everything bite sized but I'd like to be able to have larger foods available that might take more than one bite.
I have a Quest 2 to work with and am open to getting other hardware and/or software that I might need to be able to make this work.