← All guides · Published November 14, 2026 · Reviewed November 14, 2026 · By the Ingenious AI team · 7 min read
What Is Meta-ExternalAgent and the October 2026 Crawl Clause?
The short answer: Meta-ExternalAgent is Meta's training crawler — its documented job is to crawl 'to train foundation AI models or index content directly'. It drew attention in October 2026 when reporting highlighted a crawl clause in Meta's updated Commercial Terms effective October 30. It is a verified training-class entry in AgentBlock's database: block it in the builder, express the decision in robots (Meta says changes can take up to 24 hours to take effect), and enforce at the edge for immediacy.
Build my block kit — free · Test a policy in the simulator first
What is Meta-ExternalAgent?
Meta's training and indexing crawler, a verified training-class entry in AgentBlock's database, documented in Meta's own web-crawler documentation updated in 2026. The database records Meta's robots guidance for it, including that robots changes can take up to 24 hours to take effect.
Why this crawler got a news cycle
Most training crawlers arrived quietly. Meta-ExternalAgent arrived with paperwork: in mid-October 2026, reporting by several outlets drew attention to Meta's crawler documentation alongside a crawl-related clause in Meta's Commercial Terms taking effect October 30, 2026. Site owners who had never heard the name found it in articles about their own rights — an unusual route to crawler awareness, and a useful one, because it sent them looking at their logs.
What they found there varied. Meta's crawler family is a suite — training (Meta-ExternalAgent), indexing (meta-webindexer), user-triggered fetching (meta-externalfetcher), ads (meta-externalads) and the utility link fetcher (facebookexternalhit) — and October's attention attached to the family name while the decisions belong to the individual agents.
The training job, in Meta's own words
The database quotes the documentation: Meta-ExternalAgent crawls 'to train foundation AI models or index content directly'. That phrasing puts it squarely in the training class for policy purposes — the indexing it mentions is in service of Meta's models and products, not a citation channel an owner can benefit from the way retrieval indexing works. Owners deciding the training class as a set will find this entry already at the class default of block.
Meta's documented control is robots.txt: disallow the agent, with the caveat the database preserves — changes can take up to 24 hours to take effect. Twenty-four hours is fine for a posture change and useless for an urgent one, which is why AgentBlock kits also enforce the same decision at the edge or server, where a refusal is effective on the next request.
The other Meta agents are separate decisions
Blocking Meta-ExternalAgent does nothing to meta-webindexer, Meta's AI search indexer, whose documentation argues the opposite trade: allowing it helps Meta AI cite and link to your content. It does nothing to meta-externalfetcher, the user-triggered fetcher that may bypass robots because a person initiated the request. And it must never sweep in facebookexternalhit, the link-preview fetcher whose blocking breaks how your pages render when shared on Facebook and WhatsApp.
Five Meta entries, five decisions — the builder lists them adjacently precisely so the family resemblance does not collapse them into one switch. The worked example below shows a business that wanted exactly two of the five blocked and three allowed, for reasons specific to each job.
Evidence and the 24-hour caveat
Because Meta's own guidance runs through robots, the evidence trail for this agent is double: the robots layer, checked by the enforcement-gap audit, and the edge layer, visible in the block log. If the two disagree — robots open, edge closed, or the reverse — the audit reports the drift and the owner decides which layer was wrong. A week after deployment, the block log should show Meta-ExternalAgent refusals at the edge regardless of when Meta's systems absorbed the robots change.
Keep the October clause reporting in proportion. The database entry cites it as context with its sources and dates; the operative facts for your policy are the documented user-agent, the documented job, and your own decision. Policy built on documentation survives news cycles; policy built on headlines needs rebuilding with each one.
Five actions to take today
- Decide Meta agent by agent. Five verified Meta entries exist; Meta-ExternalAgent is only the training decision.
- Block it in the builder. Let the kit express the decision in robots and enforce it at the edge or server at once.
- Respect the 24-hour note. Meta documents that robots changes can take up to a day to take effect — the edge refusal covers the gap.
- Protect facebookexternalhit. Keep the link-preview fetcher allowed or accept broken previews on Facebook and WhatsApp shares.
- Audit both layers after a week. Enforcement-gap audit for robots, block log for the edge; the pair should tell the same story.
Worked example: a tour operator splits the Meta family five ways
Riverbend Tours, a fictional Belize tour operator, arrived via the October reporting with a blunt instruction to its web helper: 'block Meta'. Working through the database changed the shape of the job. Facebook and WhatsApp shares drive a third of its bookings, so facebookexternalhit had to stay allowed; Meta AI citations were a channel it wanted, so meta-webindexer and meta-externalfetcher stayed allowed too.
The final Meta policy blocked exactly two: Meta-ExternalAgent (training) and meta-externalads (ads crawling). In the first month the kit logged 2,214 Meta-ExternalAgent refusals and 311 meta-externalads refusals, while link previews continued rendering — the operator verified by sharing its own tour pages in a WhatsApp group and watching the cards load.
The owner's note on the policy reads: 'We did not block Meta. We blocked two jobs Meta does, and kept three.' That sentence is the whole discipline of per-agent policy in miniature.
Frequently asked questions
What did Meta's October 30, 2026 terms change do?
Reporting in October 2026 highlighted a crawl-related clause in Meta's updated Commercial Terms effective October 30, alongside Meta's crawler documentation. AgentBlock's database cites that reporting as context for the Meta-ExternalAgent entry; the entry's operative facts are Meta's documented user-agent and its stated training purpose.
How do I block Meta-ExternalAgent in robots.txt?
Meta documents a robots disallow for the agent, with changes taking up to 24 hours to take effect. AgentBlock generates that robots rule as part of your kit and also enforces the block at the edge or server, so the refusal is immediate rather than next-day.
Does blocking Meta-ExternalAgent stop Meta AI from citing me?
No — citation flows through meta-webindexer (indexing) and meta-externalfetcher (user-triggered fetch), which are separate entries with their own decisions. Blocking the training crawler alone leaves both citation paths exactly as they were.
Why is facebookexternalhit in an AI database if it is not an AI bot?
Because owners meet it in the same logs and the same decisions. The database labels it utility — a link-preview fetcher — and defaults it to allow, precisely so that an AI-blocking exercise does not break link previews by accident.
Is Meta-ExternalAgent the same as Meta's Muse agent?
No. Muse is a separate, watch-class entry: Meta publishes no user-agent for it, so it cannot be blocked by name and always classifies UNKNOWN. Meta-ExternalAgent is fully documented and blockable — the two entries illustrate the database's verified/watch divide.
Related guides
- How to Block AI Scrapers and Bots from Your Website
- Which AI Crawlers Should You Allow?
- How Do You Block AI Bots on WordPress, Cloudflare, or Plain JS?
- Pre-deploy breakage simulator — test a policy before it ships
- Impact preview — your server log against a draft policy
- How Do You Test a Block Policy Before Deploying It?
- How Do You Preview a Block Policy Against Your Real Server Log?
- What Is an Enforcement Gap Audit for AI Crawler Policy?
- How Do You Prove What Your AI Block Kit Actually Blocked?
- How Do You Verify a Block Kit Is Genuine and Unchanged?
- Block evidence — parse a log, build an evidence pack
- The dated agent database — 25 entries, graded