How to Map out ChatGPT Grounding queries in BigQuery (from GSC and BWT)

Posted on

by

Simone De Palma

in

Add me as Preferred Source on Google
Add as preferred source on Google

ChatGPT’s relationship with the open Web is ever-changing, but the official data sources apparently don’t: Google Search Console and Bing Webmaster Tools retain the most interesting signals in plain sight

The Reddit citation slump from ChatGPT proved that OpenAI is increasingly relying on its own web index and cached content while narrowing some of its live web validation towards authoritative domains leveraging the site: search operator.

Search queries with site: operators can expose consistent patterns with malicious AI scrapers, prompt tracking and LLM grounding. Digital marketing data has probably never been this chaotic, but once organised with deterministic rules, the picture becomes much clearer.

In this article, I explore how to build a query-level data mart in BigQuery, separate potential prompt-tracker and scraping activity from queries that are more consistent with ChatGPT grounding, whilst reporting on a bunch of interesting findings.

LLM Models are Ever-Changing and How This Affects Grounding Today

According to Olivier de Segonzac,site: usage rose from 40.8% in late July to 58.1% in mid-August. This suggests that ChatGPT is validating web searches (grounding, i.e;) less broadly and increasingly going straight to brands and official sources it considers useful.

As Lily Ray pointed out, narrowing the grounding window on highly authoritative and established sources is a way to filter out spammy and manipulative noise from AI answers that could eventually piggyback on the Web.

However, OpenAI’s finances are unarguably sitting in the background. Early this year, OpenAI cancelled a deal with Disney’s AI video app Sora, likely as part of a broader review of how it allocates processing power and budget across its expensive new models (Sol, Luna, Terra, etc.).

If logic serves me right, an increase in qualitative grounding would demand an increase in retrieval tokens, with processing costs skyrocketing as a result.

But what do retrieval and grounding mean?

1️⃣ Retrieval is the broader process LLM models leverage to find potentially relevant information upon a user prompt.

2️⃣ Grounding is the subsequent process of validating information into an answer from external sources (e.g; Google and Bing’s index)

If the cost of unrestricted retrieval increases with the complexity of AI systems, it would become more economically attractive to maintain a reusable index and reserve more expensive live retrieval for a narrower set of queries.

No guardrail in RAG means more fan-out queries to be validated (grounded), hence no good for OpenAI
No guardrails in RAG means more fan-out queries to be validated (grounded) – not sustainable for OpenAI

In turn, restricting the web search validation to high-authority domains while increasingly relying on a cached web index to serve more than 90% of ChatGPT users enables OpenAI to absorb costs and better redistribute investments.

Relying on a cached index with a narrow on "site:searches" is better for OpenAI!
Relying on a cached index with a narrow focus on site: searches is more sustainable for OpenAI!

OpenAI have been refining and fine-tuning its business model over time with the ongoing scaffolding of its own web index.

So much so that when I tested what LLMs used to fill meta descriptions (September 2025!), I sensed that free-tier users would be served a URL, the full page title, and a snippet of around 200 characters mainly pulled from the opening paragraph rather than the meta description.

Google Search Console is a Footprint of LLM Grounding

Actually, Google Search Console data in BigQuery can be a goldmine for identifying LLM grounding activity, because much of this validation happens through the underlying search engine results pages.

The problem, of course, is that not every unusual site: query represents an LLM grounding itself on a website.

The same footprint can be produced by several different types of automated behaviour:

  • LLM grounding — a system searches a domain or source to retrieve information for an answer.
  • Prompt tracking — an SEO platform tests prompts and searches Google or Bing to monitor if a domain appears in AI answers.
  • Scraping — an automated system uses search operators to discover or validate URLs at scale.
  • Manual searches — users (AKA SEOs) can obviously generate the same site: operators themselves.

The goal here is therefore not to create a binary AI / not AI classification.

Instead, we can use the characteristics of the queries, their impression volumes and device information (where available) to identify clusters that are more consistent with AI or not.

What GPT-5.6 Changed in the Grounding Footprint

If the underlying grounding behaviour changes after a model rollout, the search terms exposed through first-party search datasets may change with it. This doesn’t necessarily tell us exactly which system generated an individual query, but changes in the distribution of query patterns can provide useful evidence.

PeecAI recently shared the disappearing terms from ChatGPT grounding after the rollout of GPT 5.6. This is useful information that you can use during query classification (next paragraphs!)

Data Transformation in Dataform (BigQuery)

Let’s act on that. The next step is to turn these raw signals into a reproducible dataset.

Search Console and Bing Webmaster Tool both conceal site: search queries but most of them are anonymised and exposed to sampling when retrieved via their user interface or their API.

Rather than manually inspecting thousands of queries in Search Console, I will use Dataform inside BigQuery to build an incremental data mart. This transformation layer provides a consistent structure to map out query patterns and automate the same classification logic as new Search Console data arrives.

Talking processes, this time I’m not leveraging a UDF routine as I did when building a framework to capture AI Mode queries; the query pattern of interest (i.e; site: | -site:) is already clean, and it doesn’t need pre-processing like accent-folding, non-ASCII stripping, or phone-number removal.

Hence, I step over into Dataform (BigQuery), and I’ll focus on capturing both site: and -site: operators with RegEx.

💡Why site: and -site: ?
A positive site: operator restricts results to a particular domain, while -site: excludes a domain.

  • -site: searches tend to appear frequently in prompt tracking templates, but even web scrapers and genuinely spam traffic attempting to hijack your site’s visibility outside a target domain.
  • site: searches can resemble a request to retrieve information from a specific source, therefore we may associate it to grounding activity

This SQLX therefore captures the operator first and applies additional rules afterwards rather than attempting to classify the entire query with one regular expression.

config {
  type: "incremental",
  schema: "seo_marts",
  name: "chatgpt_grounding_gsc_mart",
  uniqueKey: ["data_date", "query", "device"],
  bigquery: {
    partitionBy: "data_date",
    clusterBy: ["query"]
  }
}
WITH query_data AS (
  SELECT
    data_date,
    query,
    device,
    SUM(clicks) AS clicks,
    SUM(impressions) AS impressions,
CASE
  WHEN REGEXP_CONTAINS(LOWER(query), r'-site:')
    THEN '-site:'
  WHEN REGEXP_CONTAINS(LOWER(query), r'site:')
       AND NOT REGEXP_CONTAINS(
         LOWER(query),
         r'{brand_regex}'
       )
    THEN 'Spam'
  WHEN REGEXP_CONTAINS(LOWER(query), r'site:')
    THEN 'site:'
END AS query_type
  FROM `you-project.searchconsole.searchdata_url_impression`
  --TABLESAMPLE SYSTEM (2 PERCENT)
  WHERE query IS NOT NULL
    AND device IN ('DESKTOP', 'MOBILE')
    AND REGEXP_CONTAINS(LOWER(query), r'site:')
  GROUP BY
    data_date,
    device,
    query
)
SELECT
  data_date,
  query,
  device,
  clicks,
  impressions,
  query_type,
  CASE
    WHEN query_type = '-site:' AND impressions <=10 AND device = 'DESKTOP'
      THEN 'Prompt Tracker'
    WHEN query_type = 'Spam' THEN 'Spam Scraping'
    WHEN query_type = 'site:' OR query LIKE '%official%' OR query LIKE '%price%' OR REGEXP_CONTAINS(query, r'2025|2026|2027')
      THEN 'ChatGPT grounding'
    ELSE 'Other'
  END AS type
FROM query_data

💡I suggest spending a few more seconds reading this query, because everything that follows in the article is based entirely on the instructions provided in it

This is the resulting dataset that you can query almost for free.

ChatGPT Grounding Mart based on Search Console data flow

Find site: Searches by Device in Google’s Dataset

Once we have prepared our custom dataset, it is time to explore it.

Here, I’m trying to figure out the distribution of both site: and -site: searches by device type.

This query only cost me 249.62 KB to run.

SELECT
  device,
  COUNT(DISTINCT query) AS query_count
FROM `your_project.seo_marts.chatgpt_grounding_gsc_mart`
WHERE data_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY device
ORDER BY query_count DESC;

Prompt trackers notoriously scan the Web using a Desktop endpoint. Therefore, the device dimension is particularly useful as it introduces another behavioural signal that can be combined with other query features.

Find Site:search and Spam Queries in Google’s Dataset

Inspecting the query_type will help you get an idea of the distribution of site:searches and spam queries from the custom dataset data mart.

SELECT
  query_type,
  COUNT(DISTINCT query) AS query_count
FROM `your_project.seo_marts.chatgpt_grounding_gsc_mart`
WHERE data_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY query_type
ORDER BY query_count DESC;

It’s another cheap query worth only 243.58 KB of your storage, where:

  • Spam = site:searches that did not feature the brand name in question. Example:
    site: {unqualified_domain.com} bububu railway zanzibar
  • Site: = regular site:searches. Example:
    site:{your_brand} “riu vistamar” “november 2026”
  • -site: = negative site:searches. Example:
    “royalton luxury resorts” -site:{any_domain.com}

Find ChatGPT Grounding Query Types

Same query as above, except you screen the classification heuristic I’ve deliberately applied during data transformation in Dataform.

SELECT
  type,
  COUNT(DISTINCT query) AS query_count
FROM `your_project.seo_marts.chatgpt_grounding_gsc_mart`
WHERE data_date >= DATE_SUB(CURRENT_DATE(), INTERVAL 30 DAY)
GROUP BY type
ORDER BY query_count DESC;

⚠️As a reminder, what we are building here is a proxy dataset: a structured collection of queries that exhibit behavioural characteristics consistent with AI grounding.

The “type” dimension called out in the query is the result of a transformation based on the assumption that all clean “site:search” queries may fall into a condoned ChatGPT grounding bucket.

That makes the dataset useful for trend analysis without pretending that the underlying attribution is deterministic.

Where:

  • Spam scraping = site:searches that did not feature the brand name in question. Example:
    site:{unqualified_domain.com} 737-700 etops southwest hawaii
  • Prompt trackers = negative site:searches (“-site:search”) and impressions <= 10 and is in Desktop. Example:
    “tui” -site:reddit.com -site:twitter.com -site:x.com -site:wykop.pl -site:tripadvisor.com -site:youtube.com -site:yelp.com -site:booking.com -site:facebook.com -site:instagram.com -site:tiktok.com
  • ChatGPT grounding = regular site:searches where the brand name is featured and is in (Desktop + Mobile). Example:
    site:tui.co.uk vomiting diarrhoea fit to fly tui

Replicating the Framework in Bing Webmaster Tools

The same idea can be applied to Bing Webmaster Tools, but the dataset has one important limitation:

Bing does not provide the same device-level breakdown used in the Google dataset.

The classification therefore relies on query features and impression thresholds rather than combining query type with device.

This makes the Bing dataset slightly less granular, but it remains useful as an independent source for validating whether the same behavioural patterns appear outside Google.

And in my case, it did.

  • Spam scraping = site:searches that did not feature the brand name in question
  • Prompt trackers = negative site:searches (“-site:search”) and impressions <= 10
  • ChatGPT grounding = regular site:searches where the brand name is featured

What the Framework Can and Can’t Tell You

There is an important methodological caveat behind this framework.

Google Search Console and Bing Webmaster Tools expose the search query, but they do not provide a definitive label identifying the system that generated it.

Hence, this methodology can’t prove that a particular query was generated by ChatGPT.

What it can do is identify patterns of search behaviour that are consistent with ChatGPT grounding, prompt tracking or automated scraping.

If a large increase in queries classified as potential ChatGPT grounding coincides with changes in new LLM model releases or a shift in AI search activity, the framework can provide a useful directional signal.

It should not, however, be interpreted as a direct measurement of ChatGPT’s grounding.

Turning Grounding Queries Into SEO Insights

Prompt trackers, web scrapers and human users can all generate similar queries in Google Search Console and Bing Webmaster Tools, so treating every site: search as evidence of ChatGPT activity would be misleading.

The value lies in identifying patterns consistent with AI grounding rather than proving the origin of individual queries.

With BigQuery and Dataform, these signals can be turned into a reproducible query-level dataset, using classification rules, impression thresholds and device data where available.

For example, a brand name + site: query may indicate that a domain is being used to validate information about that brand.

This is a proof of concept for an LLM grounding tracking framework for enterprise SEO teams, and one that agency folks should start exploring with their clients.

Summarise this post