GEO

Preregistered experiment with 1,100 users: forcing Google searches into AI Mode cut publisher click-through by 18.8 points

Developing

Not peer reviewed; an early 2026 AI Mode version with no ads, short forced exposure, and Google changed its markup mid-experiment.

A person typing on a laptop in a bright kitchen
Photo: Woman working on laptop while sitting on kitchen counter by Shixart1985, CC BY 2.0, via Wikimedia Commons

In 60 seconds

  • Researchers Stephanie T. Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson and Danae Metaxa ran a preregistered field experiment with 1,100 US Chrome users.
  • After three baseline days, users were randomised for seven days to Current Search, No-AI or AI Mode (redirected to AI Mode).
  • AI Mode reduced publisher click-through by 18.8 percentage points (95% CI 15.3 to 22.2); users spent longer per session and completed fewer sessions per day.
  • Trust in Google's information, usefulness and satisfaction declined.

Impact Layer

What changed
There is now randomised evidence on what forced AI Mode does to publisher clicks.
Why it matters
Most traffic forecasts are based on observational data; this is a controlled estimate, within limits.
Who is affected
Publishers, SEO leads and anyone forecasting organic traffic.
Martech stack area
SEO & Search, Content
Stack impact
Use it as a bound for planning, not a forecast: the exposure was forced and short.
Business impact
Moderate A preprint summarised by Let's Data Science. The authors caution the 18.8-point figure is not a forecast for all Google users or any site, and Google changed AI Overview markup during the experiment.
Related vendors
What to watch
Peer review and replication with newer AI Mode versions and ads.

A preregistered field experiment, “AI in Search Reduces Publisher Referrals Without Improving User Experience: Experimental Evidence” (arXiv 2608.18352, preprint dated August 18, 2026), was covered by Let’s Data Science on September 21. Its authors are Stephanie T. Wang, Jeffrey Gleason, Yakov Bart, Christo Wilson and Danae Metaxa.

Design

1,100 US-based Chrome users who mainly used Google Search had three baseline days, then seven intervention days in one of three browser-extension conditions: Current Search (no change), No-AI (intended to hide AI Overviews and redirect AI Mode searches) and AI Mode (searches redirected to AI Mode).

Results, AI Mode versus current Search

  • Publisher click-through fell by 18.8 percentage points (95% CI 15.3 to 22.2, p < 0.001).
  • Users spent longer per search session and completed fewer sessions a day.
  • Trust in Google’s information, perceived usefulness, satisfaction, agency and relevance declined.

Caveats

The preprint is not peer reviewed. Google changed AI Overview markup during the experiment, reducing the extension’s success at hiding Overviews in the No-AI condition. The tested AI Mode was an early 2026 version with no ads, and exposure was forced and short. The 18.8-point figure is not a forecast for all Google users or any specific site.

Our read

This is a controlled estimate, which is rare. Use it as a plausible upper bound for traffic loss when AI answers replace the list, and keep measuring your own referral data.

Sources

  1. Let's Data Science: Study finds Google AI search reduces referrals (opens in a new tab) News report ·

    Source of the design, results and caveats; the arXiv preprint itself was not read directly.

Get MarTech Roundup in your inbox

What changed in marketing technology, why it matters, and what to do next.

We will email you to confirm. You can unsubscribe at any time.

Related stories