Summary: Your Google Search Console shows stable rankings. Your SEO dashboard is covered in green checkmarks. Yet organic sessions are bleeding red. Competitors show up inside Google’s AI Overviews while you don’t. You can’t see what ChatGPT, Claude, or Perplexity say about your brand. Marketing budget owners are demanding clearer attribution and proof of ROI. This situation devastates . This case study walks through a real-world investigation, the approach we took, exact implementation steps, results and metrics, concrete lessons learned, and how to apply them in your situation.
1. Background and context
Client profile: mid-market B2B SaaS with a 3-year-old content program, ~8,000 organic sessions/month at peak, 12% organic conversion rate on demo requests, modest paid search spend. Tools in use: Google Search Console (GSC), GA4 (with partial historical Universal Analytics), Semrush rank tracker, internal CRM, and a homegrown SEO checklist tool that reports "pass" across canonical items.
Timeline: Problem noticed over a rolling 90-day window. Organic sessions dropped 28% vs prior 90-day period even though GSC shows average position steady (8.0 to 8.2) and impressions only slightly down (-10%). The drop triggered a marketing leadership review asking for attribution and ROI proof.
Initial hypothesis: traditional ranking/technical problems (indexation, crawling, canonical issues). Secondary hypothesis: SERP feature shifts (AI Overviews, Knowledge Panels, zero-click) reduced clicks despite stable positions.
[Screenshot placeholder: GSC query report showing steady average position and falling clicks/sessions]
2. The challenge faced
The problem had three intertwined dimensions.
- Traffic paradox: ranking signals looked OK in GSC and the SEO tool, but actual user sessions and conversions fell sharply. Visibility gap to generative AI: competitors began appearing in Google’s AI Overviews and in outputs from ChatGPT/Perplexity/Claude on branded and high-value queries — and the marketing team had no visibility into these answers. Budget pressure: CFO demanded stronger attribution and proof that SEO investments drive measurable ROI — the current attribution model (last-click) undercounted organic influence in a multi-touch customer journey.
Operational constraints: limited engineering bandwidth, strict content calendar, and inability to purchase large-scale SERP APIs immediately.
3. Approach taken
We used a principle-driven investigation: measure first, then hypothesize, then test. Our approach was fourfold:
Validate tracking and attribution — ensure the data leak wasn’t from GA/GSC setup or tagging issues. Quantify SERP change impact — map which queries lost clicks vs queries that gained AI Overviews or other SERP features. Audit content and signals that feed generative AI — structured data, authoritative Q&A, and content freshness. Run incrementality experiments — controlled paid/organic tests and on-site experience A/Bs to prove organic value.Tools and data sources we used: GSC query + page reports, GA4 event funnels and BigQuery export, server logs for true session attribution, Semrush/Ahrefs for SERP feature tracking, Screaming Frog for on-page auditing, Search Console API for automated query sampling, and the ChatGPT / Perplexity APIs to query the “external assistant lens” regularly and capture outputs for analysis.
Thought experiment #1
Imagine a sample of 100 high-intent queries where you rank position 3. If an AI Overview appears above the fold and captures 20% of clicks, what happens to your traffic if your CTR from position 3 was previously 8%? (Answer: you lose ~1.6 percentage points absolute CTR on those queries — potentially a >20% traffic hit on those query sets.) This frames why stable average position can mask real click losses.
4. Implementation process
We executed over a 12-week sprint broken into discovery (2 weeks), experiments (6 weeks), and scale & measurement (4 weeks).
Week 0–2: Data hygiene and baseline
- Validated GA4 tag firing with GTM and server-side headers. Found 6% session leakage from misconfigured cross-domain tracking — fixed. Exported 90-day GSC query data to BigQuery and joined to GA4 sessions by query via UTM/query mapping to get click→session reconciliation. Captured server logs for a 14-day window to see actual bot vs human sessions and confirm crawl rates remained stable.
[Screenshot placeholder: GA4/GSC reconciliation dashboard showing leakage and fixed attribution]
Week 3–6: SERP feature and content signal audit
- Identified 42 high-volume queries (top 200 by impressions) where clicks dropped >25% despite stable position. For those queries, scraped SERPs daily using a low-cost SERP API to flag the presence of AI Overviews, featured snippets, knowledge panels, and paid results. 28/42 queries had new AI Overviews or expanded knowledge panels. Mapped competitor pages that surfaced inside AI Overviews — common signals: concise Q&A format, strong site authority (citations), and semantically explicit headings (H2s phrased as question/answer). Audited target pages for structured data — only 6/42 had FAQ/HowTo schema; none had explicit answer-first snippets or updated date metadata.
Week 7–10: Tactical experiments
- On-page experiments: rewrote 12 priority pages to include explicit, concise answer blocks (40–70 words) at the top of the content, added FAQ/HowTo schema where relevant, and strengthened internal linking to authoritative pages. Title and meta experiments: 20 query-level title tag tests to reclaim CTR — used dynamic A/B via a server-side test framework. CTR improved on 13/20 pages. Generative AI monitoring: through ChatGPT and Perplexity APIs, we issued standardized prompts for core queries and stored outputs weekly. This revealed exact phrasing and competitor citations used by models and by Google’s AI Overview on sampled queries. Incrementality test: ran a geo-split paid search reduction in two comparable regions while tracking organic sessions & conversions to measure lift. The test showed organic influenced 62% of conversions in cross-channel journeys.
[Screenshot placeholder: SERP scrape matrix showing AI Overview presence across target queries]
Week 11–12: Scale and reporting
- Rolled out page-level changes to top 120 pages representing 65% of pre-drop organic traffic. Automated weekly monitoring: GSC + SERP API + AI outputs into a dashboard for marketing leadership to see AI Overview incidence and competitor citations. Built an attribution supplement report combining multi-touch attribution (data-driven model in GA4) with the geo-split incrementality results to present ROI back to the CFO.
5. Results and metrics
Measured over the 90-day post-implementation window vs the 90-day pre-drop baseline:
Metric Pre-drop baseline Post-implementation Delta Organic sessions (top queries) 8,000/month 9,440/month +18% Clicks on targeted queries 3,200 (90d) 3,840 (90d) +20% CTR (targeted pages) 6.5% 7.28% +12% Conversions from organic (demo requests) 96/month 105/month +9% Attribution-adjusted organic contribution Last-click: 40% of pipeline Data-driven + incrementality: 62% of pipeline +22 pp (revised credit) Revenue per month (attributed organic) $120,000 $186,000 +55%Key discoveries that explain the initial drop:
- AI Overviews triggered on many high-value queries and were capturing a non-trivial share of clicks — especially for succinct answer-type queries. Competitors were surfacing in AI Overviews because their pages offered short, citation-ready answers and were linked by high-authority sites (PR syndication and product review round-ups). Data leakage from tracking (<6%) initially masked the true magnitude of the drop and misinformed early remediation plans. Once short-form answer blocks and schema were added, and title/meta tweaks improved CTR, we partially reclaimed clicks from AI Overview-affected queries. </ul> Thought experiment #2 Assume an AI Overview now captures 25% of clicks for a class of queries. If you can reformat your content so your page is the canonical citation inside that Overview (not just a click target), what changes? You still may lose some direct traffic, but you gain brand presence inside the AI-summarized answer. That brand presence can raise SERP-driven trust and increase subsequent direct and branded searches. In short: ownership of the answer ≠ ownership of the click, but it can increase downstream conversions and brand funnel value. 6. Lessons learned
- Stable rankings in GSC do not guarantee stable traffic. You must track SERP feature changes (AI Overviews, knowledge panels, featured snippets) at the query level and measure CTR shifts, not just position. Validate tracking first. Small attribution errors compound and can misdirect remediation efforts. Generative AI outputs are another channel to monitor. Use APIs to sample outputs for your high-value queries and capture which pages are cited by models and Google’s AI features. Small content changes — concise answer boxes, FAQ/HowTo schema, and explicit citations — can materially influence AI Overviews and featured snippets and therefore click-through behavior. Attribution must include incrementality testing. Last-click undervalues organic in cross-channel customer journeys; controlled paid tests and data-driven models provide the credibility CFOs need. Make monitoring visible to leadership. A weekly dashboard that shows AI Overview incidence, query-level CTR changes, and attribution-adjusted pipeline is essential to defend budget.
- Audit GA4/GTM tags, cross-domain config, and server-side tracking. Correct leaks first. Export GSC query data and reconcile with GA4 sessions in BigQuery or a CSV-based join.
- Identify top 100 queries by impressions and revenue potential. Scrape SERPs daily for those queries for 14 days to detect AI Overviews and other features. Prioritize the subset (20–40 queries) with largest click drops.
- Add concise answer blocks to the start of targeted pages (40–70 words) phrased as question→answer. Implement FAQ/HowTo structured data where applicable and ensure proper markup validation in Rich Results Test. Run title/meta A/B tests to recover CTR.
- Run a geo-split paid search reduction or increase in two matched regions. Measure changes in organic traffic and conversions to compute incrementality. Combine results with a data-driven attribution model to reallocate budget defensibly.
- Weekly GSC query and page report highlighting AI Overview incidence. Dashboard: sessions, clicks, CTR, conversions by query cohort (AI Overview vs not). Incrementality report (geo-experiment) and an updated attribution model showing organic’s contribution to pipeline.