You cannot pass the nic through and still use it in proxmox - it loses all access to the nic. So if you want to go that route I recommend you to use eth3 as dedicated port for proxmox.

For (2) - affected in which way? Compared to which baseline? Are you concerned that your machine cannot keep up with 320Mbps? I doubt that.

For (3) - depends on your local network setup. Do you use vlans that need routing? Then it goes via OPNsense. Otherwise if all devices are in the same subnet, it likely will not.

For (5) - no loops. You’re not routing traffic between eth3 and eth2.

I personally would not recommend this setup as any issue with your proxmox cluster will turn into a network issue.

Instead, I would purchase a cheapish router that can run OpenWRT. If you are dead set on OPNsense you can find x86 boards from various vendors or you can make a dedicated router out of a network card and a small form factor computer

Enable pci passthrough for the card, use the onboard interface for proxmox management. It will only cause a loop if you bridge the interfaces or enable routing in proxmox, which is really hard to do accidentally. Don’t worry about it too much. Even if it happens, just unplug one of the connections and reboot the switch, then go back in and fix it.

The way I would do it is this (assuming this VM is going to act as your main router):

Connect eth3 to your switch, set up proxmox with that as your management interface. Create the OPNSense VM and pass through the NIC, make sure you remove the one it automatically creates. It will either prompt you to set one port as WAN and one as LAN, or if you connect one port to your upstream connection it should autodetect that as WAN and assign the other as LAN. Finally, connect the LAN port to the switch as well; it won’t cause a network loop because eth3 isn’t bridged with either of the others (it can’t be, because the host can’t see a PCIe device being passed through to a VM).

With this, you can always access the proxmox host via eth3, so no matter what happens to the OPNSense VM you can still access the host. Just make sure that the OPNSense LAN subnet overlaps with the IP you set in Proxmox - since it’ll probably be statically set, not DHCP, it won’t automatically pick up an IP in the LAN subnet.

Don’t dedicate your nics. Use vlans and trunk ports.

Create a wan vlan. Create a lan vlan.

There’s a few things we don’t know here.

• Are you hosting more VMs on Proxmox that need network via a virtual switch?
• Are you providing network to other physical devices as well via a switch, so you need to output to that?
• Do you want OPNsense to be your gateway and assign IPs, or do you have a router?
• As you have 3 NICs and sound like you want to use them, let’s assume you aren’t doing RoaS, but this could all be done on one (very busy) NIC.

There’s nothing wrong with your plan, but that’s not how I would do it.
I don’t pass through NICs. I bond them or I bridge them.
In a virtual world, this sort of task is done with virtual switches. OVS switches at my job.
OVS is a lot easier to use than oldskool linux bridges that come installed with Proxmox. There’s already a dropdown in Network where you can build with OVS objects, but you need to add the package.
apt install openvswitch-switch

• MGT. For your setup, I might consider (the onboard!) eth3 as my mgt NIC. That might be handy some day if you have to remove that card. Your server will still be online.

• DMZ WAN. I would run the WAN line straight to eth1. Add eth1 to a ‘dmz’ virtual switch. Add the OPNsense WAN leg to this dmz virtual switch, so the OPNsense (and nothing else) can directly talk to the upstream router.

• LAN Virtual. Create a ‘protected network’ virtual switch. Add the OPNsense LAN leg to this virtual switch. VMs can be a member of this downstream protected network and access any services provided by the OPNsense.

• LAN Local. If you need to share the OPNsense protected network back out to other devices, add eth2 to to the protected network switch, and ethernet cable out from eth2 to a dumb switch. Plug other external devices into the dumb switch, and they will be downstream from and protected by your OPNsense, accessing its services.

Feel free to ignore me here. I build a lot of big things, so I use enterprise-scale techniques. There’s nothing wrong with your pass-through plan.
And … you can do this! I have a somewhat similar setup on my laptop with HyperV, so I can distribute wired (work VPN) and wireless (everything else) internet to guest VMs and the main OS. I made two virtual switches in HyperV.

• The first switch gets exclusive access to my NIC attached to my VPN device. This is the OPNsense WAN leg.
• The second switch is the OPNsense LAN leg and VMs are members.

Good luck!

The way you plug a VM into proxmox is with a virtual switch, bridge devices do they or the SDN stuff. I’ve played SDN yet so not sure how that works, but I do run bridges.

I would make 2 bridges, br0 and br1.

Br0 is the wan side of opensense, br1 is the Lan side.

Don’t add any IP to either, add an IP to the last nic as a management port.

Now plug your VM into the bridge devices.