Content Caffeine #49: New crawl and click data surprises, paying publishers, and AI Mode experiments


Content Caffeine

For content-obsessed marketers and SEOs

Hi there,

You might like the spam update reaction below (both funny and true?).

Today's highlights include:

  • A new way to pay publishers
  • The crawl-to-click gap: What Cloudflare data shows
  • The latest AI Mode experiments
  • See the 'Inspiration' section for your new content strategy

As always, thanks for being here.

I'll be back on September 18.
Nicole

P.S. When you coin a new term, you can earn featured snippets. That's what happened with my "AI Quality Filter Theory."

I want to share your:

Big news, latest resource, or outstanding results.
I'd love to support you by featuring you here. It's FREE!

Smash the button
and show me what you've got.

Insights

A New Way To Pay Publishers

Perplexity just dropped $42.5 million on what's basically a "please don't sue us" fund for publishers.

The AI search company is offering to share 80% of revenue from their $5 monthly subscription service with news outlets whenever their content gets used in search results.

While other platforms have cut one-time licensing checks, Perplexity's deal might give publishers ongoing revenue.

Is this a good deal?

It sounds generous until you do the math. 80% of $5 is $4.

The newspapers I subscribe to usually bill around $25 per month to access all their articles, so I question how many publications would want to take less.

Lawsuits abound

Call me cynical, but I believe the program is an attempt to make nice before things get uglier in court.

Perplexity is currently getting hammered with lawsuits from major players like News Corp (owners of Dow Jones and the New York Post) for allegedly stealing content.

Cloudflare also called them out for sneaking past website protections to scrape articles.

Publishers are essentially being asked to gamble on a startup's growth promises while ignoring some pretty serious allegations about how the company operates.

It's like your neighbor offering to pay rent after you catch them using your swimming pool.

Nice gesture, but the trust is already broken.

Will 'non-news' sites ever get paid for the content these models train on?

Hit reply and tell me what you think of the deal.


Google News

1. Google avoids break up

District Judge Amit Mehta decided against ordering Google to sell its Chrome browser and Android products, but there will be no more exclusive deals.

So, while Google can no longer be the default search provider, they can still pay device makers (like Apple) to preload their products.

That must be seen as a win for Google—and both Google and Apple shares rose on the news.

Mehta also instructed Google to share some of its search data with third parties.

That data appears to include certain search index and user interaction data, though not advertising data.

After the Court's decision Google wrote in a blog post:

"Now the Court…will require us to share Search data with rivals. We have concerns about how these requirements will impact our users and their privacy, and we’re reviewing the decision closely."

Watch this space.


2. Spam update reaction

You've seen that Google rolled out its latest spam update on August 26, targeting sites that violate their search spam policies.

Shortly after the update announcement, Nate Hake took to X and accused Google of violating its own spam policies.

Naturally, the post attracted a huge number of views.

No comment from Google.


3. AI Mode experiments

Meanwhile, Google is testing changes to AI Mode that aim to encourage clicks to publishers.

Robby Stein (VP, Product) wrote on X:

"We’ve been experimenting with how and where to show links in ways that are most helpful to users and sites…"
And,
"We’ve found that people really prefer and are more likely to click links that are embedded within AI Mode responses…"

One minute, Liz Reid tells us everything is fine:

"With AI Overviews and more recently AI Mode, people are able to ask questions they could never ask before. And the response has been tremendous: Our data shows people are happier with the experience…"

The next minute, Google seems to have discovered that people like to scroll and click—like they always did!

Let's hope we see an increase in links to expert sources and sites that deserve to surface in the SERPs.


The crawl-to-click gap: AI bots, training and referrals

Cloudflare data shows crawling by search engines and AI services surged in the first half of 2025 — up 24% year-over-year in June — before slowing to just 4% year-over-year growth in July.

Anthropic remains the most crawl-heavy platform. Even after an 87% decline this year, it still crawled 38,000 pages for every referred page visit in July 2025.


Two highlights from the Cloudflare report:

  • Training crawling grows: Training now drives nearly 80% of AI bot activity, up from 72% a year ago.
  • Publisher referrals drop: Google referrals to news sites fell, with March 2025 down ~9% compared to January.

You'll get more data in this blog post.

Related: Glenn Gabe summarizes an article from the Financial Times suggesting that AI companies have sucked up all of the available free training data on the internet.


Do you follow me on LinkedIn? I share regular tips and stories I don't have room for here. Come and join me​.

This is not an ad...

Grow your brand mentions and visibility online.

Digital PR is fast becoming the new organic search.

If you need innovative ways to stand out from your competitors, we have ideas for you.

↩ Steer your brand North today.


Information

✶ OpenAI scraping mystery solved

OpenAI has been partially using Google search results scraped by a startup called SerpApi for ChatGPT responses on current events like news and sports.

Perplexity is also a customer.

h/t to Glenn Gabe.

Original article from The Information [paywalled].

✶ Important reminder

Anthropic will start training its AI models on your data, including new chat transcripts. If you use Claude, you can choose to opt out by September 28.

Go to Settings - Privacy - "Help improve Claude," and toggle to Off.

It's a major flip from their previous stance on user privacy, as Techcrunch points out.

✶ A clear choice for B2B marketing

In newsletter #46, I asked this question in a poll: What is the most effective channel for B2B marketing?

Your choices were SEO, social media or content marketing.

I didn't report on the results because only 2 people voted, so I went to LinkedIn and asked the question there.

Can you guess which option came out on top—before you check the results?


Inspiration

Where Is Your Moat?


Anyone can create content with AI tools.

And that's a problem.

You need to ask yourself "Where's my moat?"

Medieval castles had water-filled trenches (moats) around them to keep their enemies at bay.

In business, your moat makes it almost impossible for competitors to copy you, but 'typical' content strategies have the defensive power of a kiddie pool.

Building your moat

Your moat isn't built from what everyone else knows. It's unique to you and your brand.

Think about the brands that own their space. They're not publishing how-to guides, listicles, and industry overviews.

They're publishing original research, they're documenting the methods behind successful campaigns, they're telling stories about product launches that went sideways.

You might see patterns or have insights about your industry that others miss. Write about those.

The brands that truly stand out aren't the ones trying to out-AI the algorithms. They're doubling down on their irreplaceable human experiences.

Your failures, your breakthroughs, your hard-won insights, the conversations that shaped your thinking are things only you can teach.

What can only you teach your audience?

That's your content strategy.

That's your moat.


Thanks for being here!
I'll see you again on September 18.

In the meantime, feel free to ask me a question, send an interesting link, or tell me what's on your mind. I read all your emails!


Dates to watch

October Monthly Observances

  • Breast Cancer Awareness Month
  • Bully Prevention Month
  • Halloween Safety Month
  • Financial Planning Month
  • National Pizza Month

Weekly Observances

  • October 5-11: Fire Prevention Week
  • October 13-19: Earth Science Week
  • October 19-25: National Business Women’s Week

Days

  • October 1 – International Coffee Day
  • October 1 – World Vegetarian Day
  • October 1 – Yom Kippur
  • October 3 – National Techies Day
  • October 4 – National Golf Lovers' Day
  • October 5 – World Teachers' Day
  • October 5 – Oktoberfest ends
  • October 6 – Child Health Day
  • October 10 – World Mental Health Day
  • October 11 – National Coming Out Day
  • October 12 – National Farmers' Day
  • October 13 – Indigenous Peoples’ Day
  • October 13 – Columbus Day
  • October 13 – Thanksgiving Day (Canada)
  • October 16 – World Food Day
  • October 19 – LGBTQ Center Awareness Day
  • October 24 – United Nations Day
  • October 25 – Make a Difference Day
  • October 28 – National First Responders' Day
  • October 30 – Mischief Night
  • October 31 – Halloween

Keep in touch

12424 Black Hills Drive, Austin, TX 78748
Unsubscribe · Preferences

Content Caffeine

My team and I have been helping brands reach their SEO traffic and conversion goals through content and links for over 10 years. Recognized by industry leaders and household brands as an authority in both organic content and digital PR.

Read more from Content Caffeine
a chart showing a green line going up, a pile of coins and a Google logo

Content Caffeine For content-obsessed marketers and SEOs Hi there, Today, you'll see a couple of thought-provoking perspectives on the state of content and how SEOs can thrive going forward. Let's dive in.Highlights include: Gary Illyes Reveals His Stance on AI Content What Percentage of Web Pages Are Created With AI? (and what this means) How To Become a Commercially Minded SEO Tools Are Underrated As always, thanks for being here. I'll be back on September 4.Nicole P.S. Well, this post on...

A screenshot from Google Search showing the appearance of more embedded links in AI Mode answers.

Content Caffeine For content-obsessed marketers and SEOs Hi there, ICYMI: The Google Search Liaison role is defunct. Danny Sullivan stepped away from the public-facing role he's held for nearly 8 years to "work on various projects within the Google Search team." Is this the end of any transparent communication with the search community?Today's highlights include: Just do normal SEO Rank tracking doesn't exist Brand mentions by LLMs: Mentions vs citations As always, thanks for being here. I'll...

pie chart showing how AIO citations rank in search. the 3 pie slices are colored blue (76.10%), Orange (14.4%) and yellow (9.5%)

Content Caffeine For content-obsessed marketers and SEOs Hi there, Google added an AI Mode button on the right side of the search box. Have you seen it yet? We may never be able to accurately track traffic from AIOs and AI Mode, but getting your brand (or your clients' brands) mentioned in AI-enabled search is a huge credibility lever. I discuss the best way to do that in the 'Inspiration' section below.Today's highlights include: SEO ranks #1 in B2B June Core Update: Winners and Losers...