It’s relatively easy for Cloudflare to profile clients as being web scrapers. A concerning amount of internet traffic goes through their servers in plain text.

Yeah. Me choosing to use a vpn and a privacy respecting browser has earnt me a constant captcha

Yeah, it’s only anecdotal but I feel like hobbyists like us, who do slightly unusual things without nefarious intent, who are the ones who get hit with these sorts of issues the most. For example, I’ve noticed that some websites start throwing captchas at me or even just straight-up refuse to load with 403: unauthorized errors because I have my router set up to load-balance across two Internet connections. (At least, that’s my guess as to why it’s happening.)

I’ve seen captchas for years before the recent influx of AI. It’s the way I go about obfuscating network activities that the site security cannot determine if I am a bot on not. There is a Captcha Buster extension for Firefox. If the captcha is ‘Pick the three busses from these blurry, pixelated set of pictures’ then I can solve those easily. It’s when the captcha is a full page of a motorcycle and you have to check all the relevant pieces, then on to the next full picture, that chap me. So you click Captcha Buddy and it ‘listens’ to the audio portion of the captcha, then solves it. It’s not 100% on all types of captchas, but it 90% of the time it works every time. It’s interesting to me that after a while, you start to notice patterns in the captcha images. For instance if the directions are ‘Pick the fire hydrants’, there will be at least 5 you have to pick. Crosswalks are the same way too.

I’d much rather have to do captchas than have my jimmy out in the ether traffic. Anecdotal, but Stack Overflow doesn’t trigger a captcha for me. All I get is the cookie popup.

That uses proof of work rather than just detecting and blocking the bots.

do they just want everything to be crawled

Yes. Web crawling has been a normal and vital part of the web from day 1. We’d have no search engines without crawlers.

The web is user-centric by design. I’m sick of tech companies trying to flip the script and hoard information, most of which is not theirs to begin with (e.g. Google, Reddit, Twitter, Facebook, etc.).

Well, they have access to logs showing who connects to 24 million websites, how they use those websites, and for how long. So if there’s anyone who knows what traffic is crawlers, and which crawlers are AI, it’s Cloudflare. There’s no way they wouldn’t know, they have all the data they would ever need to figure it out. In fact, there’s nobody on the internet who is better positioned to be able to identify AI crawlers than Cloudflare.

Because it never was about security. You’re training LLMs for free.

I’m pretty sure some auto drive company is getting the advantage since a lot of captchas are spotting crosswalks, traffic lights, stairs, busses, mountains, motorcycles etc. Wonder if it’s fucking tesla

To punish you for trying to protect yourself. To extract micro-labour out of you (AI training) To discourage you from privacy best practices Oh BTW, the captcha will eventually contain unblockable ads

It was minding it’s own business and adding them… lol 😀