Vol. I · Fri, 2 Oct 2026Open 24 hours · Feeding: continuous · Please do not tap the glass
Plate I0xBBE1B037OBEYED
Kingdom Automata›Phylum HTTP›Order Crawlers›Family Search Engines›Species Googlebot
Specimen file · Search Engine Savanna

Googlebot

Googlebotus maximus

What is Googlebot?

Google's main crawler. It fetches pages so they can appear in Google Search. It is operated by Google and identifies itself as Googlebot.

Temperament: tireless. Has read every page here more times than the author. Returns hourly to check nothing has changed. Nothing has changed.

Operator
Google
Conservation status
Respects robots.txt
First seen
2 Oct 2026
Last seen
8 min ago
Visits today
29
This week
29
All time
29
Caught in the trap
Never
How to block Googlebot ↓
§ I.

Identification

Its full user-agent string, exactly as it arrives at the gate:

User-Agent
$ Mozilla/5.0 (compatible; Googlebot/2.1; +http://www.google.com/bot.html)

To check a visitor really is Googlebot, run a reverse DNS lookup on its IP address and confirm the hostname ends in .googlebot.com, .google.com, .googleusercontent.com. Then run a forward lookup on that hostname and check it returns the same IP. Anything that fails is an impostor; the zoo checks every visit this way and files the failures in the Trap Room. Read Google's documentation.

§ II.

How to block Googlebot

Add these two lines to the robots.txt file at the root of your site. Well-behaved crawlers read it before they crawl, so the change applies from Googlebot's next visit. Nothing else on your site needs to change.

robots.txt
# Block Googlebot from the whole site
User-agent: Googlebot
Disallow: /

Blocking Googlebot removes your pages from Google Search. To opt out of Gemini training only, block the Google-Extended token instead.

Or let it visit but keep it away from part of the site:

# Let it in, but keep it out of one room
User-agent: Googlebot
Allow: /
Disallow: /members/
§ III.

Observed behaviour

Visits, last 30 dayspeak 29 / day
3 Sept 2026today
Diet · pages most often eaten
/9
/robots.txt7
/favicon.ico4
/sitemap.xml2
/ads.txt2
Visiting hours
00h – 11h12h – 23h

Most active around 15:00. Office hours, like a professional.

robots.txt compliance
100.0%

Requested 0 disallowed pages out of 29 requests. Read robots.txt 7 times.

Trap Room record
Clean

Has never followed the hidden link to /trap/. Either well trained or very lucky.

§ IV.

Where it comes from

Requests that use Googlebot's name but fail Google's network check are filed separately, as impostors. These are the visits that passed, or that could not be checked at the time:

Networks · 1 on file, last 30 days
Google LLC AS15169100%
Countries
United States US100%

Networks and countries come from the visitor's IP address, looked up in a local copy of the DB-IP database. The addresses themselves are never stored.

§ V.

Keeper's field notes

2 Oct 202618:20. Arrived without announcement. Consumed /robots.txt and /og/page/robots-txt-generator.png. Departed.
2 Oct 2026Observed reading robots.txt in full. Complied with every word.
2 Oct 2026Visited 29 times in a single day. Keeper unable to establish why.
2 Oct 2026First recorded at the gate. Entered in the register as Googlebotus maximus.
§ VI.

Questions site owners ask

Does Googlebot respect robots.txt?

Yes. In 29 requests observed here it has read robots.txt and never fetched a disallowed page.

Will blocking Googlebot hurt my search rankings?

Yes, for Google's search engine. Blocked pages can drop out of its results. Other search engines are unaffected.

How often does Googlebot visit?

Here, about 4 requests a day over the last week. Visits to your site depend on its size, how often it changes, and how many links point to it.

Also in Search Engine Savanna