Is Googlebot Blocked by Cloudflare AI Bot Settings?

It depends on the current settings and rules for your Cloudflare zone. Cloudflare’s September 15, 2026 guidance says its Block and Block on pages with ads actions can apply to mixed-use crawlers, including Googlebot. If your goal is to discourage AI training while keeping qualifying search crawlers available, Cloudflare documents a different option: Disallow AI Training. Check each zone’s current controls and request evidence before drawing a conclusion about your site.

This audit is for site operators who manage Cloudflare and need to distinguish a crawler preference from an actual access decision. Cloudflare’s description of its controls does not establish what happened to a particular request, and access to a URL does not mean Google will index it.

Understand the three controls before changing one

Cloudflare groups AI crawler controls into Search, Training, and Agent categories. They address different crawler purposes, so review all three rather than relying on an old setting label. Cloudflare’s AI bot controls documentation describes the categories and actions; for the revised distinction between blocking and expressing a no-training preference, use its later guidance on mixed-use crawlers.

Cloudflare says Block and Block on pages with ads apply to mixed-use crawlers, including Googlebot. By contrast, Disallow AI Training publishes a preference in robots.txt and allows qualifying Accountable mixed-use crawlers to remain available for search. Cloudflare identifies Google among the operators it says meet its criteria. That is Cloudflare’s description of its policy and controls—not a promise that Google will crawl, index, or rank a URL.

Check how the current settings got there

Don’t assume every Cloudflare zone received the same migration or preset. In its September 15 guidance, Cloudflare describes different outcomes for existing zones depending on their prior configuration. Existing domains without granular controls were migrated according to the legacy Block AI Bots setting; Cloudflare says a legacy Block selection maps to Search: Allow, Training: Disallow AI Training, and Agent: Block on pages with ads. For existing domains with prior granular settings, Cloudflare says Training selections of Block or Block on pages with ads migrate to Disallow AI Training, while Search selections are preserved under the new controls.

New-domain presets are a separate case. Cloudflare says they depend on whether a site monetizes using ads. Its ad-supported preset allows Search, sets Training to Disallow AI Training, and blocks Agent on pages with ads. Settings can be changed during onboarding or later. Treat these as documented scenarios, not evidence of what any particular zone currently uses.

Audit your zones this week

Inventory the zones. List the domains you manage in Cloudflare, including separate production, staging, or business zones if they matter to your search presence.
Record the current controls. For each zone, note the action shown for Search, Training, and Agent, plus the date you checked. Record the displayed current action, not just a remembered legacy setting.
Compare Training with your policy. Decide whether your aim is to block mixed-use crawler access or to express a no-training preference while retaining qualifying search access. Compare that aim with the actual Training action shown.
Review relevant custom rules. Check applicable security rules and other controls that could affect the requests you care about. Record the rule and action where you can identify them; AI bot settings alone may not explain a request’s treatment.

A simple zone record can be a spreadsheet with columns for domain, check date, Search action, Training action, Agent action, relevant custom rules, and unresolved questions. This makes it easier to compare zones and revisit the decision after a configuration change.

Use the Training action to make the decision

If the policy is to block mixed-use crawler access, Cloudflare documents Block as the relevant action. If the policy is to express a no-training preference while preserving qualifying search access, investigate Disallow AI Training and confirm the effective setting and any applicable rules. These choices have different effects for mixed-use crawlers; don’t treat the two actions as interchangeable.

Hypothetical example: An existing ad-supported site used the legacy Block AI Bots selection. Before changing anything, its operator checks the migrated Training action in the zone. If the business wants to discourage training without blocking qualifying search crawlers, the operator compares the current value with Cloudflare’s documented Disallow AI Training option, checks Search and Agent as well, and reviews relevant custom rules. The legacy label alone is not enough to determine the current behavior.

Investigate specific important URLs

Choose a small set of URLs that matter to the business, such as key service pages or product pages. For a time when you suspect a crawl attempt occurred, inspect Cloudflare Security Events and any available request logs. Where the records provide it, look for the request path, timestamp, action, and matching rule. Cloudflare says Security Events cover requests actioned or flagged by security products, but events may be sampled and the dashboard is not a complete record of every request. A missing event cannot establish that Googlebot was allowed.

Compare what you can find with origin logs and Google Search Console’s Crawl Stats or URL Inspection. Google’s crawl-troubleshooting guidance recommends those Search Console tools and site logs when investigating crawling errors; a request reaching a site is not, by itself, proof that Google will index the page. A useful finding ties evidence to a particular URL and time—for example, a logged request and a recorded rule action—or clearly states which part could not be confirmed.

Keep the test identity in view. A successful browser or command-line fetch shows the response to that particular request, from that client and path; setting a Googlebot user-agent string does not by itself make the request verified Googlebot. Likewise, reviewing robots.txt shows a published preference, not necessarily the response a request receives at Cloudflare’s edge. Cloudflare describes robots.txt preferences and edge enforcement separately in its Bot Preference Sync announcement.

Verify the policy, then decide what to do next

Configuration: Do the current Search, Training, and Agent actions in each zone match the policy you intend?
Rules: Have you checked relevant custom rules and recorded any action that could affect the requests under review?
Request evidence: For selected URLs, do available Cloudflare or other request logs identify a path, time, action, and rule? If not, record the evidence gap rather than treating an absent event as an allow.
Google evidence: Have you compared the investigation with Search Console or site-log evidence, while keeping crawl access separate from indexation?

If the recorded controls match your intended policy, keep the zone inventory and check date so a later review can spot changes. If they do not match, decide whether to change the relevant action, then repeat the request-level checks for selected URLs. For help with that work, Splinternet Marketing’s SEO service can help audit the configuration, correlate available Cloudflare and Search Console evidence, and keep technical improvements under review. Brian Bateman’s managed service is $421 per month per business, month to month, and can be canceled at any time.

Sources

Editorial note: AI assists with research, drafting and automated checks. Sources are linked so you can verify the guidance. Platform requirements can change; confirm the details that apply to your setup.