← All guides · Published November 14, 2026 · Reviewed November 14, 2026 · By the Ingenious AI team · 6 min read

What Is a Canary Path and How Does It Catch Crawlers That Ignore Rules?

The short answer: a canary path is a URL planted in your kit that no legitimate visitor can reach by browsing — it exists only in places structural crawlers read. When something fetches it, the Worker logs a CANARY verdict with the path, user-agent and IP. Because the tripwire is behavioural rather than name-based, it catches crawlers your name rules cannot: spoofed agents, unnamed agents, and agents ignoring the policy they were given. Every AgentBlock policy generates a fresh canary path of its own.

Build my block kit — free · Test a policy in the simulator first

What is a canary path?

a unique, unlinked URL generated with each AgentBlock policy (the generator mints one when you do not supply your own). It is reachable only by something that enumerates or follows machine-readable structure, so a request for it is evidence of structural crawling and is logged with a CANARY verdict.

Why a tripwire beats another rule

Rules govern agents that identify themselves. The residual problem in crawler control is everything else: agents under browser names, agents with no name, agents whose operators ignore refusals. You cannot write a name rule for a visitor whose defining trait is a false name. What you can do is plant evidence — a path whose only plausible visitors are programs traversing structure — and let behaviour identify them.

Security teams have used the pattern for decades under names like honeypot and canary token. AgentBlock builds it into the kit rather than leaving it as an advanced craft: the policy carries a canary_path, freshly generated per kit when you do not set one, and the Worker watches it on every request at no extra configuration cost.

What a CANARY verdict actually tells you

A canary hit proves one thing precisely: something fetched a path that browsing cannot reach. Read with its user-agent and IP, it usually sorts itself. A hit from a named agent you have blocked is a crawler working from a stale map or ignoring its refusal — escalation-worthy. A hit from a browser-named agent is structural scraping under a spoof — the exact population your name rules cannot see. A hit from your own monitoring or a search engine's validator is a false friend, identified once and noted.

What it never proves is intent. The verdict is a detection, not an accusation, and the evidence tooling treats it that way: CANARY events aggregate like other verdicts, and a single hit is a curiosity while a pattern of hits across your path space is a finding. Owners who check the canary line in their monthly review learn about structural crawling months before it shows up as a business symptom.

Where canary paths sit in the stack

The canary complements each other control without duplicating any. Name rules handle the honest identified majority. The sender gate flags agents whose claimed name and IP range disagree. Rate caps bound volume behaviourally. The canary detects exploration itself — the mapping behaviour that precedes extraction — often before any content of value has been taken. Together they cover identity, honesty, volume and reconnaissance, which is most of the observable surface of crawler behaviour.

Because the path is generated per kit, it rotates when you regenerate: a fresh tripwire accompanies each new policy, and the old path's history remains in your logs as the record of the previous watch.

Acting on a hit without theatre

The proportionate responses, in order: note it; check whether the same IP or agent appears elsewhere in the log at volume; fence or cap the pattern if it recurs; and let a single unexplained hit stay a single unexplained hit. The failure mode to avoid is treating the canary as an alarm bell that demands a same-day policy overhaul — tripwires inform a monthly review, and their evidentiary value depends on the calm, dated log that accumulates between reviews.

For agencies and multi-site owners, canary evidence has a second use: it is the most client-legible proof that monitoring is real. 'Something probed a path no human can reach, on this date, and here is what we did' lands differently from any assurance about vigilance.

Five actions to take today

  1. Let the kit mint your canary. A policy generated without a custom path gets a fresh canary path automatically — confirm it is in the deployed Worker.
  2. Never link the canary. Its value depends on browsing being unable to reach it; it must exist only where structural crawlers look.
  3. Read CANARY verdicts monthly. Sort hits by agent and IP; patterns are findings, singles are curiosities.
  4. Correlate before acting. Check the same sender's wider footprint in the evidence view before fencing or capping anything.
  5. Rotate with each regeneration. A new kit means a new tripwire; keep the old one's log history as the previous watch's record.

Worked example: a directory site spots the mapper

TradeLedger, a fictional business directory with 22,000 listing pages, deployed its kit in January with training blocked and retrieval allowed. Its March review found 14 CANARY verdicts across six weeks — all from browser-named agents, all from a rotating set of IPs, each hit preceded in the logs by methodical category-page fetches at machine regularity.

No name rule could have caught the visitor: it never claimed an agent identity. But the pattern around the canary hits — 9,300 category fetches in the same window, at an even 40 requests a minute — justified a behavioural response: tighter rate behaviour on category paths and a fence on the directory's export-style endpoints.

By May, category fetches from the rotating IPs had fallen by 83%, and the canary went quiet. TradeLedger's review note credits the tripwire with the only thing that mattered: it saw the mapping while the mapping was still reconnaissance.

Frequently asked questions

Will real visitors ever hit the canary path?

Not by browsing — the path is unlinked and generated per kit. Hits come from structural traversal: crawlers following machine-readable structure, security scanners, or spoofed agents mapping the site. A hit from your own uptime or SEO tooling is possible once, easily recognised, and worth noting rather than chasing.

Is a canary path the same as a honeypot?

It is the same family — a planted artifact whose interaction is itself the signal. AgentBlock's version is deliberately narrow: one path, one verdict type, integrated into the kit's normal logging, so it needs no separate honeypot infrastructure to run or review.

What should I do after a single canary hit?

Record it and look at the sender's wider footprint in your logs. A lone hit with no surrounding pattern is a curiosity for the monthly review. Repeated hits, or hits alongside high-volume category fetching, justify rate or path responses — proportionate, dated, and logged like any other policy change.

Does the canary path block anything?

No. It detects. The Worker logs a CANARY verdict; blocking, capping or fencing decisions stay with your policy's explicit rules. Detection and enforcement are kept separate so the evidence is never muddied by the response.

Is the canary included on the free plan?

The canary path is part of the policy the generator builds — the Worker features that use the full policy (path matrix, sender gate, canary, rate caps) are the Pro tier of the generated Worker, while the free Worker carries the core name-based site rule.

Build my block kit — free · Browse the agent database

Related guides

HONEST SCOPE: this policy blocks KNOWN agent user-agents and behaviors listed below. Sophisticated spoofing can bypass any user-agent block (e.g. an agent sending a stock browser UA, or rotating residential IPs). This is a control, not a guarantee.
← Back