NSX-T 3.0.0 with VDS 7 filtering/dropping return geneve traffic?

**edit** I've nailed it down to the VDS dropping the return packets as suspected, capturing with `pktcap-uw` and the `--capture Drop` switch is showing the relevant return traffic being dropped because `Drop Reason 'VXLAN Module Drop'`. Digging around it seems that an NSX enabled DVS will drop any encapsulation traffic passed to it if the destination IP isn't one of the TEPs on the host. Hence traffic being allowed *from* the Edge but now back *to* it, any why a `vmkping` will work.

The 'solutions' seem to be either move back to an N-VDS setup which I'd like to avoid, move the routers off to non-NSX enabled hosts or switches or change the port used by geneve traffic. Not sure if the last one is possible with NSX-T, it was with NSX-V, currently looking into this.

**edit2**

You can't change the port used for geneve tunnels anywhere in the API I can find, so my only option is to move the VyOS VMs to a non-NSX enabled vSwitch.

**---**

I've got a little homelab setup with NSX-T which i've rolled from 2.3 -> 2.4 -> 2.5 and now to 3.0.0, previously always using the only v2 choice of a seperate N-VDS and pNICs.

When I was originally setting this up the Cisco 3750x switches I have will do L2 MTU9000 but unfortunately not L3 9000 MTU, no problem I thought and stood a pair of VyOS router VMs up which I configured to be the edge and host TEP subnet gateways and also the edge uplink subnet gateways. All connects into my 'core' in an iBGP mesh all working beautifully. These VMs are attached to VLAN portgroups on the same DVS that is being turned into a v7 NSX switch.

With v3 and the support for dropping the N-VDS and going all in on a VDS 7 I decided to try out the migration so I could pull a couple of extra NICs out that I had added to provide the seperate pNICs before. The migration went fine as far as I could tell until I tried to start up an VMs connected to the overlay, the hosts would turn to Degraded and the Host <--> Edge tunnel wouldn't come up.

Working through it all with PCAPs at the various points I could see:

1. Geneve traffic leaving the Edge destined for host TEP IP 2. Reaching the VyOS router gateway on the Edge VLAN, being routed to the Host VLAN 3. Received by the host TEPs 4. Host returning the traffic back to the Edge TEP IP 5. Nothing being receieved by the VyOS VM on the Host VLAN interface

The packets all looked good, correct MACs, correct VLANs etc.

I have the VyOS VMs running on seperate clusters with their respective edge VM to provide HA when powering down one or the other. It didn't matter whether I tried with either router or from different hosts, the return traffic to init the tunnel never made it back to the router VM interface. Using `pktcap-uw` capturing directly on the relevant vmnic I could see the return traffic making it to the host before vanishing.

Now I was suspecting the NSX enabled VDS' so I stood up a quick and dirty ESXi host on another PC I have and migrated one of the VyOS VMs to it and it started working.

So it seems that there is some sort of reverse traffic flow filtering going on or it's a bug, I appreciate this is probably a bit of a edge case.

Anyone ran into this or similar or has anything I they could point me in the direction of looking at? I can't find anything in the NSX logs on the hosts and there's nothing obvious in the documentation that I can see.

Otherwise I guess i'm about to add a seperate ESXi host for running the VyOS VMs or convert them to bare metal.