AI search spreads across Maps, Gemini, and AI Mode. December 2025 shows Google’s ecosystem expanding and how brands must adapt to stay visible in 2026.
Cloudflare’s outage reminds you to check settings. Take 5 mins to unlock your content’s visibility in ChatGPT, Claude, and all AI answer engines.
If you’re reading this right now, there’s a good chance your site was unreachable earlier today. Cloudflare suffered a massive global outage that knocked out X, ChatGPT, Spotify, Shopify, and countless others for hours, triggering widespread 500 errors and dashboard failures.
The incident is resolved, but millions of site owners are logging into their Cloudflare dashboards right now to double-check settings and make sure everything’s stable.
While you’re already in your Cloudflare dashboard, take 5 extra minutes to flip a toggle and add one simple Web Application Firewall (WAF) rule — and make sure your content shows up in ChatGPT, Claude, Gemini, and every other AI–powered answer engine.
Back in July 2025, Cloudflare declared “Content Independence Day” and shifted to blocking AI crawlers aggressively by default (especially on new zones or when the global “Block AI Bots” toggle is on).
This was celebrated by publishers (The Atlantic, TIME, AP, etc.) but created a quiet visibility crisis. Thousands of sites accidentally vanished from AI search and RAG-powered (Retrieval Augmented Generation) answers because verified bots like GPTBot and ClaudeBot were getting blocked at the edge.
As Search Engine Land noted at the time, the default change caught many SEOs off-guard and sparked heated debate about whether blocking by default hurts discoverability in the AI era.
Barry Schwartz broke the story the same day:
“For Cloudflare to announce it will block AI bots and crawlers by default is a big deal. Any new site that signs up for Cloudflare will automatically, by default, set to block AI bots from accessing that content.”
— Barry Schwartz (@rustybrick), Search Engine Land, July 1, 2025
Duane Forrester (former Bing, former Yext, now Founder at UnboundAnswers.com) immediately warned:
“This won’t end well… businesses may struggle to appear in AI-powered answers without realizing crawler access is being blocked unless a fee is paid.”
— Duane Forrester, Search Engine Journal, July 2, 2025
Fast-forward to today: If you want your content cited in AI answers (and the referral traffic that can come with it), you need a “Verified Allow” posture — not blanket blocking.
1. Go to Security > Bots (or Security > Settings in some dashboards)
2. Find “Block AI Bots” / “AI Scrapers & Crawlers”
3. Toggle it OFF
If this is left ON, even cryptographically verified bots from OpenAI and Anthropic get blocked before any custom rules run.
This lets only real, trusted AI bots bypass JS challenges and Bot Fight Mode while spoofed traffic stays blocked.
1. Security > WAF > Custom Rules > Create Rule
2. Name: `Allow_Verified_LLM_Crawlers`
3. Expression:
“`
(cf.verified_bot_category in {“AI Crawler” “AI Assistant” “AI Search”} and cf.verified_bot)
“`
4. Action: Skip
5. Skip these components:
Pro tip: Deploy in Log mode for 24–48 hours first to confirm which legitimate bots are hitting your site.
Verified bots respect robots.txt. Explicitly allow the big ones (Empty Disallow = full allow):
“`
User-agent: GPTBot
Disallow:
User-agent: ClaudeBot
Disallow:
User-agent: CCBot
Disallow:
User-agent: PerplexityBot
Disallow:
“`
While robots.txt still does the heavy lifting today, consider adding a simple `/llms.txt` file with your best URLs. It’s not enforced yet by majors, but it’s gaining traction as a structured “here’s my premium content” menu for emerging AI agents.
Today’s outage was painful, but it’s also a free reminder that your Cloudflare config controls two futures: traditional search and the exploding AI-answer ecosystem.
Flip the toggle, drop in the rule, and you’re done.
Need the WAF rule exported as JSON or a second pair of eyes on your logs? Drop a reply or send us a message — we’re happy to help.
AI search spreads across Maps, Gemini, and AI Mode. December 2025 shows Google’s ecosystem expanding and how brands must adapt to stay visible in 2026.
Google reviews do more than influence customer choices. Learn how ratings, volume, recency, and responses improve local rankings and shape AI search answers.
Can’t find your website in a sea of search results? We’ll offer seven reasons why your content may not be ranking highly and explain how GPO can help.
