Adopt Masked visitor too: your name goes on a plaque on this page for a year.
Masked visitor's food bowl
the bowl, as bots see itThe bowl is empty. Last time, LocalSearchEngine stole the Canonical Bandwidth Bagels, 5 days ago.
Identification
Its full user-agent string, exactly as it arrives at the gate:
There is no operator to verify against. This name is what software calls itself when nobody gave it one.
No operator stands behind this name, so there is nothing to check its visits against.
How to block Masked visitor
Masked visitors look like browsers, so a user-agent rule cannot catch them. Rate limiting by IP address catches the worst of them without bothering people.
# A disguised visitor sends a normal browser user agent, # so there is nothing to match on. Rate-limit instead: # nginx limit_req_zone $binary_remote_addr zone=perip:10m rate=2r/s; limit_req zone=perip burst=20 nodelay;
A bot-management service or WAF can also score requests on behaviour.
Observed behaviour
Most active around 08:00. Before the coffee is made.
Requested 64,338 disallowed pages out of 200,755 requests. Read robots.txt 691 times.
Walked through the hidden /trap/ door. Last caught 11 h ago.
Opened 5,501,505 rooms in the endless maze. Style: Explorer. Opens several doors on each level before going deeper. Watch it in the Labyrinth →
Where it comes from
Scripts and scanners run from wherever their owners rent a server. These are the networks behind the visits on file:
Networks and countries come from the visitor's IP address, looked up in a local copy of the DB-IP database. The addresses themselves are never stored.
Keeper's field notes
Questions site owners ask
Does Masked visitor respect robots.txt?
Mostly. It reads robots.txt, but it has fetched a disallowed page 64,338 times out of 200,755 requests observed here.
Will blocking Masked visitor hurt my search rankings?
No. Nothing respectable will miss it.
How often does Masked visitor visit?
Here, about 3,254 requests a day over the last week. Visits to your site depend on its size, how often it changes, and how many links point to it.