Meta says partner and I moving in together is a dealbreaker

My (42f) partner (38m) and I have been together about a year and a half, ENM/poly from the outset. We both have other partners and don’t have a formal hierarchy, but have become de facto primaries based on time spent together, integration into each other’s family, etc. We are fully committed to each other and both in it for the long haul.

We live on opposite sides of a large city and have both been feeling the strain of that commute and the desire to spend more unstructured time together, and so started discussing nesting and have been casually looking at condos.

This morning I got a text from my partner saying “You and me moving in together is a deal breaker for [Meta] :( She doesn't feel comfortable getting sexy with me in a space shared by another.”

I am not sure of her living situation, but I believe he always hosts her at his place currently. They typically see each other about 3x per month.

I just texted back “well, that is unfortunate” and left it there, so not sure what he’s feeling about her statement yet (we are both at work, so not a good time for an in-depth convo) but my first reaction is to be annoyed and I guess I’m just looking for a sanity check.

Am I unreasonable in feeling like she shouldn’t be able to dictate his living arrangements? Should his desire to keep Meta happy trump our plans to keep building a life together? I know it’s ultimately his decision, but am I way off base in feeling peeved here? Could use some perspective, thanks.