SMRTR ProgrammingSep 24, 2024Hacker Noon

FaceBook Bots, Crawlers And User Agents Causing Resource Drains On Websites And Hosting Accounts

SMRTR summary

Website owners are grappling with Facebook's overzealous crawling, as its "facebookexternalhit" bot bombards servers with up to 400 requests per second, prompting some to use rate limiting tools like Cloudflare to manage the load while preserving content indexing.

SMRTR provides this summary for quick context. The original article belongs to Hacker Noon.

Read the original article
SMRTR Programming

Get the next batch of curated stories in your inbox.

This archive is built from SMRTR newsletter stories. Subscribe for hand-picked stories without the extra noise.

Related Stories

Browse Programming
ProgrammingAug 24, 2026

Programming Paradigms

A structured roadmap for experienced developers to move beyond language familiarity and genuinely master the four core programming paradigms.