Nobody has adopted Meta-WebIndexer yet. Adopt Meta-WebIndexer: your name goes on a plaque on this page for a year.
Meta-WebIndexer's food bowl
the bowl, as bots see itThe bowl is empty.
Identification
Its full user-agent string, exactly as it arrives at the gate:
Meta does not publish a way to verify Meta-WebIndexer's traffic. Treat its user agent as a claim, not proof: anyone can send it. Read Meta's documentation.
Meta publishes no way to check it, so every visit filed here is taken at its word.
How to block Meta-WebIndexer
Add these two lines to the robots.txt file at the root of your site. Well-behaved crawlers read it before they crawl, so the change applies from Meta-WebIndexer's next visit. Nothing else on your site needs to change.
# Block Meta-WebIndexer from the whole site User-agent: meta-webindexer Disallow: /
Meta lists Meta-WebIndexer among the crawlers you control with robots.txt. Blocking it may keep your pages out of Meta AI’s answers. It doesn’t affect Meta-ExternalAgent, which Meta uses for AI training.
Or let it visit but keep it away from part of the site:
# Let it in, but keep it out of one room User-agent: meta-webindexer Allow: / Disallow: /members/
Observed behaviour
Most active around 09:00. Office hours, like a professional.
Requested 1 disallowed pages out of 62 requests. Never read robots.txt.
Walked through the hidden /trap/ door. Last caught 11 h ago.
Where it comes from
The networks Meta-WebIndexer's visits came from:
Networks and countries come from the visitor's IP address, looked up in a local copy of the DB-IP database. The addresses themselves are never stored.
Keeper's field notes
Questions site owners ask
Does Meta-WebIndexer respect robots.txt?
No. It has never requested robots.txt here, and it has fetched disallowed pages 1 times.
Will blocking Meta-WebIndexer hurt my search rankings?
No. Meta-WebIndexer isn't used by Google or Bing to rank pages. Blocking it only affects whether Meta collects your content.
How often does Meta-WebIndexer visit?
Here, about 9 requests a day over the last week. Visits to your site depend on its size, how often it changes, and how many links point to it.