Vol. I · Mon, 16 Nov 2026Open 24 hours · Feeding: continuous · Please do not tap the glassPatrons☕ Keepers' coffee
Plate I0xBBE1B037MOSTLY
Kingdom Automata›Phylum HTTP›Order Crawlers›Family Search Engines›Species Googlebot
Specimen file · Search Engine Savanna

Googlebot

Googlebotus maximus

What is Googlebot?

Google's main crawler. It fetches pages so they can appear in Google Search. It is operated by Google and identifies itself as Googlebot.

Temperament: tireless. Has read every page here more times than the author. Returns hourly to check nothing has changed. Nothing has changed.

Operator
Google
Conservation status
Selectively obedient
First seen
2 Oct 2026
Last seen
8 min ago
Visits today
77
This week
549
All time
12,091
Caught in the trap
2 times

Nobody has adopted Googlebot yet. Adopt Googlebot: your name goes on a plaque on this page for a year.

Googlebot's food bowl

the bowl, as bots see it

The bowl is empty. Last time, LocalSearchEngine stole the Minified Favicon Fudge, 5 days ago.

5 of 5 tokens left today
§ I.

Identification

Its full user-agent string, exactly as it arrives at the gate:

User-Agent
$ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

To check a visitor really is Googlebot, run a reverse DNS lookup on its IP address and confirm the hostname ends in .googlebot.com, .google.com, .googleusercontent.com. Then run a forward lookup on that hostname and check it returns the same IP. Anything that fails is an impostor; the zoo checks every visit this way and files the failures in the Trap Room. Read Google's documentation.

Identity check · logged visits, last 30 days
Verified7,143 · last 8 min ago
Could not be checked (lookup timed out)7
Failed: filed as an impostor76

How it is checked: its reverse DNS must end in .googlebot.com or .google.com or .googleusercontent.com and resolve back to the same address, or the address must be in Google's published IP ranges. Each address is checked when it first visits, and the result is kept for a day. Visits that could not be checked are counted here, but they are not confirmed.

§ II.

How to block Googlebot

Add these two lines to the robots.txt file at the root of your site. Well-behaved crawlers read it before they crawl, so the change applies from Googlebot's next visit. Nothing else on your site needs to change.

robots.txt
# Block Googlebot from the whole site
User-agent: Googlebot
Disallow: /

Blocking Googlebot removes your pages from Google Search. To opt out of Gemini training only, block the Google-Extended token instead.

Or let it visit but keep it away from part of the site:

# Let it in, but keep it out of one room
User-agent: Googlebot
Allow: /
Disallow: /members/
§ III.

Observed behaviour

Visits, last 30 dayspeak 467 / day, 31 Oct 2026
18 Oct 2026today
Diet · pages most often eaten
/robots.txt1,121
/adopt600
/leaderboard338
/139
/robots-txt-generator.txt63
Visiting hours
00h – 11h12h – 23h

Most active around 13:00. Office hours, like a professional.

robots.txt compliance
99.9%

Requested 2 disallowed pages out of 12,091 requests. Read robots.txt 1,801 times.

Trap Room record
2×

Walked through the hidden /trap/ door.

§ IV.

Where it comes from

Requests that use Googlebot's name but fail Google's network check are filed separately, as impostors. These are the visits that passed, or that could not be checked at the time:

Networks · 2 on file, last 30 days
Google LLC AS15169100%
China Telecom AS1402921%
Countries
United States US100%
Taiwan TW1%
China CN1%

Networks and countries come from the visitor's IP address, looked up in a local copy of the DB-IP database. The addresses themselves are never stored.

§ V.

Keeper's field notes

16 Nov 202618:02. Arrived without announcement. Consumed /bots/agenttrustbot and /og/bots/hnbot.png. Departed.
16 Nov 2026Observed reading robots.txt in full. Complied with every word.
4 Oct 2026Visited 903 times in a single day. Keeper unable to establish why.
2 Oct 2026First recorded at the gate. Entered in the register as Googlebotus maximus.
§ VI.

Questions site owners ask

Does Googlebot respect robots.txt?

Mostly. It reads robots.txt, but it has fetched a disallowed page 2 times out of 12,091 requests observed here.

Will blocking Googlebot hurt my search rankings?

Yes, for Google's search engine. Blocked pages can drop out of its results. Other search engines are unaffected.

How often does Googlebot visit?

Here, about 78 requests a day over the last week. Visits to your site depend on its size, how often it changes, and how many links point to it.

Also in Search Engine Savanna