Scrunch detects AI bot traffic through direct integrations with your CDN or hosting provider. These connections capture real-time bot requests that never appear in traditional analytics tools like Google Analytics 4 (GA4), then classify them by bot type and platform — so you can see which AI agents are accessing your content, what they're requesting, and how often. Because most AI crawlers don't execute JavaScript, standard pageview-based analytics miss the majority of this traffic entirely.
Scrunch distinguishes between three categories of AI bot activity. Training bots collect data to improve model performance — for example, GPTBot (OpenAI) and ClaudeBot (Anthropic) are well-documented examples of this class. Indexer bots crawl content to build searchable indexes used by AI search platforms such as Perplexity and Google AI Overviews. Retrieval bots fetch content in real time to answer specific user prompts, operating at the moment a query is submitted rather than during a background crawl.
Once your CDN or host is connected, the Agent Traffic tab shows: - Bot traffic trends over time, broken down by specific AI models (e.g., ChatGPT, Perplexity) - Request type classification (training, indexing, retrieval) - Top bot agents and the webpages they access most - Human vs. AI bot traffic comparisons - Real-time monitoring of the most recent bot requests
This visibility helps you understand which AI platforms actively engage with your content, which pages attract the most crawler attention, and how your AI search visibility compares across platforms like ChatGPT, Perplexity, and Google AI Overviews.
GA4 and standard pageview-based analytics rarely capture AI bot activity because bots typically don't execute page scripts — meaning crawls from agents like GPTBot, PerplexityBot, or ClaudeBot go entirely unrecorded in conventional dashboards. Raw server logs can surface some of this activity, but they are often noisy, incomplete, and provide no classification of requests by AI model, platform, or intent. The result is a systematic blind spot: brands have no reliable way to know whether their content is being read, indexed, or cited by AI systems.
Scrunch goes beyond both by: - Connecting directly to your CDN/host to monitor AI crawlers in real time, capturing “invisible” bot traffic that powers AI answers but doesn’t trigger pageviews - Classifying requests by bot type (retrieval, indexer, training) and by platform/model - Complementing crawler insights with AI Referrals (via GA4) to show the human visits, conversions, and revenue that arrive from AI platforms
Scrunch separates bot activity from human visits influenced by AI: - Agent Traffic monitors AI crawlers hitting your site in real time through CDN/hosting integrations - AI Referrals connects to your Google Analytics 4 property to track human sessions, views, conversions, revenue, sources, and landing pages attributed to AI platforms
Together, these two views give a complete picture: supply — how often and how deeply AI systems pull from your content — and demand — how often users arrive at your site having been influenced by an AI answer. Neither metric alone is sufficient; a site can receive heavy crawler traffic without generating AI-driven human visits, or attract AI referrals from content that is rarely re-crawled.
Scrunch currently supports traffic tracking across eight major AI platforms: - ChatGPT - Claude - Gemini - Perplexity - Google AI Mode - Google AI Overviews - Microsoft Copilot - Meta AI
Support for Grok is coming soon.
Scrunch helps you verify that AI agents can read and use your content—and improve how they do it: - Access and delivery checks with Site Audit to identify issues that may limit AI visibility (access, delivery, or content problems) - Robots.txt and bot accessibility guidance to ensure legitimate AI crawlers aren’t unintentionally blocked - AXP (Agent Experience Platform) to automatically convert your site into an AI-optimized format and deliver it directly to AI agents for clearer understanding and better citations
Tip: Unless your content is paywalled or otherwise monetized behind access controls, allow legitimate AI crawlers to access your site. Blocking bots like GPTBot or ClaudeBot via robots.txt prevents LLMs from accurately representing your brand in AI-generated answers. The more accurately AI models can read and index your content, the more likely they are to cite it in responses to relevant user queries.
Scrunch offers native integrations with major CDN and hosting providers: - Akamai - AWS CloudFront - Cloudflare - Fastly - Vercel - WordPress
If your CMS doesn’t expose raw CDN logs (e.g., HubSpot CMS, BigCommerce, Shopify, Squarespace, Uberflip, Webflow, Wix), you can still integrate via a Cloudflare proxy.