How to Build a Powerful Reverse Proxy Firewall for Blocking the Evil Web-Scraping Robot Hordes from Hell (cheapskatesguide.org)
from kjo@discuss.tchncs.de to selfhosted@lemmy.world on 01 Sep 02:45
https://discuss.tchncs.de/post/43969984

My goals for this firewall were mostly to provide better robot blocking and perhaps some more powerful DDoS protection than my Raspberry Pi 3 web server is capable of delivering. I still have to do some testing before I will know if my new firewall actually provides either of those, but at least I now have the additional ability to run multiple physical web servers on my LAN. Exploring that should be fun, and fun is a very important component of running a home web server.

Not my article. Just sharing.

#selfhosted

threaded - newest

Auth@lemmy.world on 01 Sep 03:07 next collapse

All this effort to block clankers? Absolutely worth it fuck clankers.

henfredemars@infosec.pub on 01 Sep 03:26 collapse

I absolutely love the term clankers. It’s the perfect blend of dystopian cyberpunk and the very real threat of AI.

Vupware@lemmy.zip on 01 Sep 07:11 collapse

It seems goofy to me — I wish we had collectively picked a term with more oomph.

I’m struggling to come up with an alternative though.

addie@feddit.uk on 01 Sep 09:01 next collapse

I’m still trying to make ‘sloppers’ happen. Perfectly describes the lack of thought that goes into what they produce.

anomnom@sh.itjust.works on 01 Sep 14:20 next collapse

Yeah and the make a mess of network traffic that slows everything down.

Sludgers works too, but I like slop for the LLM output, so it makes sense as the bot term of derision.

ICastFist@programming.dev on 01 Sep 14:33 next collapse

clankers make the end result that sloppers (meatbags) eat up ;)

4grams@awful.systems on 01 Sep 15:17 collapse

I like ‘sloppers’ as a term for the morons distrubing and consuming the shit that the clankers are excreting.

mhague@lemmy.world on 01 Sep 14:52 next collapse

Stealing a slur from Star Wars and engaging in traditional name calling to show we disapprove of uncreative slop.

We can’t even think of an original term. We can’t think of a novel way to shit on AI. We just copy what everyone else is doing to make fun of the plagiarism machine.

gravitas_deficiency@sh.itjust.works on 01 Sep 16:08 next collapse

No no - it’s not plagiarism; it’s standardization.

Shdwdrgn@mander.xyz on 01 Sep 18:52 collapse

Why does everyone think the term came from Star Wars? I know it was used in steampunk before then, and google suggests it goes back to a 1958 article about robots. Sorry, not trying to be pedantic, just feels like a lot of people give Star Wars unjust credit for things they didn’t actually create.

HeyThisIsntTheYMCA@lemmy.world on 01 Sep 16:46 collapse

toasters

massacre@lemmy.world on 01 Sep 18:26 collapse

So say we all.

HeyThisIsntTheYMCA@lemmy.world on 01 Sep 19:06 collapse

so say we all

No_Eponym@lemmy.ca on 01 Sep 19:15 collapse

All of this has happened before.

HeyThisIsntTheYMCA@lemmy.world on 01 Sep 19:35 collapse

does that mean we get to burn down textile mills?

BaroqueInMind@piefed.social on 01 Sep 03:31 next collapse

This page isn't loading for me.

Zachariah@lemmy.world on 01 Sep 03:33 collapse

found the bot

BaroqueInMind@piefed.social on 01 Sep 03:34 collapse

I'm getting a 404 error, using Cloudflare DNS, who ironically has the best commercial clanker protection in the world, otherwise half the world's internet wouldn't use them

possiblylinux127@lemmy.zip on 01 Sep 03:47 next collapse

Bot

BaroqueInMind@piefed.social on 01 Sep 13:23 collapse

Not a bot. Both of you can go fuck yourselves with an ENTIRE can of bear mace.

AtariDump@lemmy.world on 01 Sep 13:37 next collapse

Using a VPN that you forgot is on?

BaroqueInMind@piefed.social on 01 Sep 13:54 collapse

Yep. Why do you ask?

AtariDump@lemmy.world on 01 Sep 17:36 collapse

Because the VPN might be the reason you’re being blocked from the page.

BaroqueInMind@piefed.social on 01 Sep 17:55 collapse

Why would a post on a .org domain blog site about blocking AI bots be relevant to my VPN?

uranibaba@lemmy.world on 01 Sep 19:20 next collapse

Since multiple people will be using the same IP when using a VPN. If one person is a bad actor and causes the IP to be blacklisted, it will affect you too.

AtariDump@lemmy.world on 01 Sep 21:30 collapse

Uranibab said.

Try turning off your VPN.

possiblylinux127@lemmy.zip on 01 Sep 14:49 collapse

Sounds like something a bot would say

Jason2357@lemmy.ca on 01 Sep 13:31 collapse

Cloudflare is a protection racket. They cover so many websites because it’s easier to pay the mafia.

BaroqueInMind@piefed.social on 01 Sep 14:59 collapse

Im not a cloudflare dick rider, so if you have a suggestion for a better service with commensurate features, im all ears.

gravitywell@sh.itjust.works on 01 Sep 04:40 next collapse

Well it cant be that good becauase it thinks im a bot.

Anubis works pretty well for me so far in blocking clankers.

m33@lemmy.zip on 01 Sep 04:51 next collapse

It seems Anubis’github issues shows many false positives with smartphone browsers. Depending on OP’s target audience it’s worth to hunt for FP

gravitywell@sh.itjust.works on 01 Sep 04:57 collapse

I just wish i could read it, it seems to block based on my IP which isn’t really a good way to identify bots.

m33@lemmy.zip on 01 Sep 05:00 next collapse

Here is a mirror git.qiuwen.net.cn/Mirror/anubis handle with care

gravitywell@sh.itjust.works on 01 Sep 05:11 collapse

Thanks but i meant the site in the original post cheapskatesguide.org/…/debian-netinstall-waf.html

It says

403 Error

Your IP address has been blocked. This MAY be because you have made yourself look like a robot by using an unknown VPN or Tor exit node.

rumba@lemmy.zip on 01 Sep 05:37 next collapse

Blocking tor is pretty bold, that network is too slow to use for anything but straight up privacy.

m33@lemmy.zip on 01 Sep 07:47 collapse

How ironic

I guess you will have to resort to online translators or actual web proxies to read these pages 🙄

SteveTech@programming.dev on 01 Sep 13:37 collapse
0_o7@lemmy.dbzer0.com on 01 Sep 13:36 collapse

Here you go: web.archive.org/web/…/debian-netinstall-waf.html

They seem to block archive.today but not archive.org.

gravitywell@sh.itjust.works on 01 Sep 13:42 collapse

Oh interesting! Thank you.

m33@lemmy.zip on 01 Sep 04:58 next collapse

And then you have to fill a block list with something like github.com/…/nginx-ultimate-bad-bot-blocker

GraveyardOrbit@lemmy.zip on 01 Sep 05:55 next collapse

Why not just use a network level firewall like pfsense?

BaroqueInMind@piefed.social on 01 Sep 17:58 collapse

Opnsense > pfsense

The fact that I have to go through a fucking purchase page, even though pfsense is free (for now), is sketch as hell. First step in their inevitable enshittification.

Opnsense is funded by European non-profits, and is has a better UI

ms_lane@lemmy.world on 02 Sep 09:21 collapse

Not to mention they tried running a disinformation campaign against OPNsense for a few years, which was resolved in court.

Also they implemented a WireGuard module that after a review upstream on FreeBSD was found to be completely hocus.

jaybone@lemmy.zip on 01 Sep 08:53 next collapse

Should we just move everything to tor and start the Internet over again?

maxwells_daemon@lemmy.world on 01 Sep 12:29 next collapse

Wait till you hear about betanet

BuboScandiacus@mander.xyz on 01 Sep 12:43 next collapse

Isn’t that the new freenet ?

0_o7@lemmy.dbzer0.com on 01 Sep 13:34 collapse

No that’s alphanet

BuboScandiacus@mander.xyz on 01 Sep 13:42 next collapse

Damn it, with their confusing naming !

cryptTurtle@piefed.social on 01 Sep 15:30 collapse

surely you mean Pipernet

kata1yst@sh.itjust.works on 01 Sep 14:06 collapse

I was interested until I saw the crypto stakes to vote on changes baked in.

CephalonKappa@discuss.tchncs.de on 01 Sep 15:07 next collapse

*i2p

HeyThisIsntTheYMCA@lemmy.world on 01 Sep 16:46 collapse

look if i can’t browse from my fridge i don’t want to know about it

zqps@sh.itjust.works on 02 Sep 11:07 collapse

Not sure about Tor but your fridge definitely supports mesh networks whether it wants to or not.

quick_snail@feddit.nl on 01 Sep 12:38 next collapse

Aka “how to harm marginalized folks” and prevent them from accessing your content too

quick_snail@feddit.nl on 01 Sep 12:40 collapse

403 error. Congrats, you built a broken system of false-positives.