from muusemuuse@sh.itjust.works to selfhosted@lemmy.world on 25 Mar 14:10
https://sh.itjust.works/post/57393940
I have a firewalla purple. it’s idiot mode networking and I love it, but I have never been too thrilled with it’s cloud shit and really don’t to rely on it as my only option right now.
A while back I tried spinning up a VM with opnsense and never got good performance off my home ryzen server. I tried multiple NICs and even bare metal installs and while bare metal was a little more performant, it was never able to reach gigabit on WAN. the firewalla falls just a hair short of gigabit WAN but its still way ahead of my more muscular server. I notice the CPU load spikes high. it seems nothing I do can bring down that CPU load for opnsense. openwrt performed a bit better but still never hit gigabit speeds and was still below the firewalla’s performance. bare metal was again a bit better but still not matching the firewalla.
The firewalla is a heavily optimized amlogic based pi. it’s not special. but it works right and my crap doesnt. I have other SBCs I can use if folding into the home server as a VM just isnt practical but the server is always on anyway and already has extra resources I can throw into this so I’d like to just throw it all in there, snapshot a working config and be done with it if I can.
I walked away from this a while back thinking I would have a fix if I took a break and came back to it later but I’m still stumped. How are other people doing this?
#selfhosted
threaded - newest
What CPU? If it was hitting 100% then that was probably your bottleneck. It just couldn’t handle the packets that fast.
Also note that the more features you turn on (firewalling, routing, inspection, etc.) the more processing has to be done on each packet.
Also also note that due to network overhead, gigabit speed for a real-world download is about 800 Mbps.
ryzen 5800xt. it didnt matter if it was booted bare metal either, it would max out 1 or 2 cores and never hit gigabit speeds
I can easily push gigabit speeds out of a Pentium G3220 running OPNSense so that sounds like a virtualization performance issue.
it happened bare metal too, booted off USB and with opnsense hitting the hardware directly
Running it as a VM or even on a server that is running other services and potentially competing for I/O or memory bandwidth also introduces many other potential sources of inefficiency. I always recommend running a firewall on dedicated bare metal hardware, it is a very specialized task with very particular requirements on behalf of both the hardware and the software and it has very little tolerance for other sources of latency or delays. That doesn’t mean you need to use a pre-built appliance, but it does explain why it’s so common, and running it on a VM on a server that is doing other stuff is likely contributing to your issues significantly.
Personally, I run my firewall/router on a very stripped-down Debian with almost no non-essential services and a custom built kernel. I hand-picked a multi-port PCIe x4 Intel NIC with good Linux compatibility and drivers, and I’m using foomuuri to handle the routing and kea to handle DHCP/DNS for my internal network. This is a very minimal, bare-bones configuration and I wouldn’t really recommend it unless you really know what you’re doing, and it’s absolutely not “idiot mode networking” and if that’s what you want you’re going to have a real bad time if you try to follow in my footsteps, because I am a very different kind of idiot. But it works for me, so it’s proof that it is possible.
Damn sure seems special. WOW! What features are/were you running on Opnsense?
I looked for specs on the Firewalla Purple. However, to compare, I’m running pFsense on an Intel Celeron CPU J3160 @ 1.60GHz/4 core/32gb RAM with pfblockerng, suricata, ntopng, and Tailscale, unbound, with customized and publicly available DNSBL lists.
Load average 0.80, 0.51, 0.45
As @frongt@lemmy.zip said, the more ‘things’ you have running, the more load, and 800 Mbps is about what I can do even with a gigabit connection and CAT6 pulled for every connection. If I were try to run huge generic block lists, I will start peeking, which is why I run mostly slimmed down, targeted, custom lists. When you stop and think about it, the amount of list checking, resolving, etc, it’s really pretty amazing.
I tried a while back to see if I could better the 800 Mbps, but nothing produced any thing much higher than the standard 800 Mbps which frustrated me. I just finally accepted the fact that getting as close to a gigabit connection would be the best I could do with what I’ve got. Being the type of person I am, I was rather verklempt I couldn’t squeeze that extra 200 Mbps.
Acronyms, initialisms, abbreviations, contractions, and other phrases which expand to something larger, that I’ve seen in this thread:
[Thread #193 for this comm, first seen 25th Mar 2026, 23:20] [FAQ] [Full list] [Contact] [Source code]
Wow, I run opnsense in proxmox along with a pihole and a couple of other small services and never hit 100% CPU on an Intel N100. My miniPC box has 4 2.5 gigabit network ports though I only use 2 of them, one for LAN and one to the modem. I do also have a managed switch, though, that has a couple of 10 gigabit ports a couple of 2.5 and the rest 1. Likely the switch is taking some of load off of the router I suppose. Might try getting a low-end managed switch. If you’re in the US do it quick, though as a lot of networking equipment is about to spike in price since the administration banned all new foreign made equipment and none is made I’m he US.
I have a smart switched. It’s kind of got man managed features like villains and stuff like that but it’s not a full managed switch
If configured properly, it can usually bypass the router altogether. In my setup I have several VLANs for different traffic, so for me it’s important to have a Layer 3 switch that can handle the routing between VLANS. But if you don’t use VLANs, a layer 2 switch will build a mac address table and bypass the router once it knows where the traffic is going. That way only your DNS queries and similar get sent to the router for internal traffic on the LAN. Then the issue is just traffic going to the internet.
For the internet side you just need to configure the firewall to drop packets on ports (not reject, just drop/ignore) you don’t use and use something like fail2ban or crowdsec to make your router outright drop malicious and LLM bot kinds of traffic to ports you do use that otherwise have to be processed. That generally will reduce processing load unless you have self-hosted services that really generate a ton of traffic in which case you can move those to VPSs outside of your network.
Those are my general strategies at a very high level.
I run Opnsense in a VM in proxmox and I passthrough an Intel X540 (2x 10Gb NIC) card into it. I have 5Gb fibre and i get that full speed on Opnsense. The CPU is an Intel N100 and I never see any CPU spikes.
I’ve had it setup for a few years now and I’ve had no issues with it.
I think you have enough people here stating their pfSense / OPNSense works fine, so I’d guess you have something unique with your setup - maybe it’s a dodgy cable, or you’re running both In & Out traffic over vlans on the same NIC on your PC and getting problems with unmanaged switches dealing with that…
I had an issue with my pfSense box not negotiating to 1Gb on a Cat6 cable to a switch. I tried all sorts of diagnostics and it turned out to be a problem with the wall socket crimping, so hardware issues do need to be checked… I’m obviously assuming you didn’t use the exact same cables as your firewalla…
Just some different angles to think about…