Vol. I · Fri, 2 Oct 2026Open 24 hours · Feeding: continuous · Please do not tap the glass
Plate XCII0x01980A92UNREAD
Kingdom Automata›Phylum HTTP›Order Scanners›Family Unidentified›Species Wget
Specimen file · The Feral Pit

Wget

Haurire omnia

What is Wget?

GNU Wget, a command-line tool for downloading files and whole websites. It is operated by Unknown and identifies itself as Wget.

Temperament: hoarding. Wants a copy of everything, for later. Later never comes.

Operator
Unknown
Conservation status
Has not read the rules
First seen
Not yet
Last seen
Not yet
Visits today
0
This week
0
All time
0
Caught in the trap
Never
How to block Wget ↓
§ I.

Identification

Its full user-agent string, exactly as it arrives at the gate:

User-Agent
$ Wget/1.21.4

There is no operator to verify against. This name is what software calls itself when nobody gave it one.

§ II.

How to block Wget

Add these two lines to the robots.txt file at the root of your site. Well-behaved crawlers read it before they crawl, so the change applies from Wget's next visit. Nothing else on your site needs to change.

robots.txt
# Block Wget from the whole site
User-agent: Wget
Disallow: /

Wget reads robots.txt when it downloads a site recursively, but not for single files.

Or let it visit but keep it away from part of the site:

# Let it in, but keep it out of one room
User-agent: Wget
Allow: /
Disallow: /members/
§ III.

Observed behaviour

Visits, last 30 dayspeak 1 / day
3 Sept 2026today
Diet · pages most often eaten

Nothing on record in the last 30 days.

Visiting hours
00h – 11h12h – 23h
robots.txt compliance
100.0%

Requested 0 disallowed pages out of 0 requests. Never read robots.txt.

Trap Room record
Clean

Has never followed the hidden link to /trap/. Either well trained or very lucky.

§ IV.

Where it comes from

Scripts and scanners run from wherever their owners rent a server. These are the networks behind the visits on file:

Nothing on record in the last 30 days.

Networks and countries come from the visitor's IP address, looked up in a local copy of the DB-IP database. The addresses themselves are never stored.

§ V.

Questions site owners ask

Does Wget respect robots.txt?

We can't say yet. It has not fetched robots.txt here, and it has not touched a disallowed page either.

Will blocking Wget hurt my search rankings?

No. Nothing respectable will miss it.

How often does Wget visit?

Here, about 0 requests a day over the last week. Visits to your site depend on its size, how often it changes, and how many links point to it.

Also in The Feral Pit