Advanced AI Search Tracking: Mapping out ChatGPT Answers with Server Logs

Add me as Preferred Source on Google

Relying solely on search queries grounded in Search Console and Bing can highlight intent, but it may not fully capture broader patterns of engagement. To understand how AI systems access and display content, we need to look beyond queries and move closer to the infrastructure itself.

Traditionally, log files have been difficult for SEO teams to work with – often locked behind technical barriers and owned by infrastructure or security teams. It’s probably quicker to find a needle in a haystack than to earn access to server logs in an SEO role.

Fortunately, the rapid growth of AI platforms has changed this dynamic. As AI crawlers and agents increasingly interact with websites, their impact on infrastructure and cybersecurity has pushed log providers to make this data far more accessible and usable.

As ChatGPT dominate AI referrals, collecting proprietary server logs tied to the ChatGPT-User agent, with valid status codes, provides a strong starting point for shifting behavioural analyses from what users might ask to how AI systems actually access your site.

In this article, I’ll show how server logs can reveal AI access patterns, measure zero-click interactions, and provide actionable insights for improving visibility across both search engines and AI-driven platforms.

⚠️ Disclaimer

This is part 2 of a series about improving AI search tracking.
Catch up with the previous article on how your Google Search Console and Bing data can track AI search behavioural trails.

This is not a prompt tracking framework

The only thing you should measure is how often your brand appears in LLM responses for your most representative topics

Does the LLM model correctly associate your brand with its marketed categories?

Granted that tracking tools provide inconsistent output, does the model consistently place your brand in the right ‘bucket’?

Technical Requirements

The main requirements to replicate this framework are:

1. Access to server logs – via web server, CDN (e.g; Akamai) or a third-party tool (i.e; Botify log analyzer) – I strongly suggest you get access to your web server or CDN as they allow for fewer margin of errors in the way the server signals are parsed and delivered into your dashboard.
2. Data & Scripting. Basic Python & SQL capabilities that you can source easily from LLMs
3. Verification. Use DNS checker to run IP lookups that will identify potential bot impersonators you’ll want to remove from the extraction

💡Tip

If fetching the last 14 or 30 days of server hits overloads your CDN capacity, remember that eventually all you need is not accuracy but a good sample

What is ChatGPT-User

This user agent is only used when ChatGPT answers a user’s question and decides to fetch your page to support its response.

So every hit from ChatGPT-User is a direct signal of user intent — it reflects real prompts happening inside ChatGPT.

It tends to retrieve only the raw HTML and skip JavaScript altogether. So if key content appears only after JS execution or if metadata is injected dynamically, ChatGPT-User is unlikely to see it or use it in its output. 

Allowing or disallowing ChatGPT-User from your website is not as straightforward. Recent updates in the OpenAI documentation confirmed that ChatGPT-User may no longer follow robots.txt rules as it might tandem with GPTBot or even be triggered by custom GPTS.

If you really want to block it from accessing your website to avoid using your content to furnish answers in chats, make sure you restrict the IP with either your hosting provider or at the CDN level with aggressive WAF.

Query your Log Files

Compose a SQL query retrieving your server logs for:

• Filtering server traffic (7-day sample) – a more retrospective outlook may exhaust bandwidth.
• Filtering by HTTP 200/304 status codes only
• Filtering by text/html content type
• Filtering by ChatGPT-User user agent
• Filtering by verified cliIP ranges only
• Detecting repeated bot hits to the same URL from the same IP
• Clustering rapid requests within a 5-second window into a single retrieval session
• Deduplicating noisy parallel crawler activity to avoid inflated counts
• Aggregating requests into cleaned retrieval events per URL

These requirements are encapsulated in the following query:

SELECT
    reqPath,
    COUNT(*) AS cleaned_bot_visits
FROM (
    WITH base AS (
        SELECT
            reqPath,
            accLang,
            cliIP,
            reqTimeSec,
            transferTimeMSec,
            downloadTime,
            toUnixTimestamp64Milli(reqTimeSec) 
              - toUnixTimestamp64Milli(lagInFrame(reqTimeSec, 1)
                OVER (PARTITION BY cliIP, reqPath ORDER BY reqTimeSec ASC)) AS time_diff
        FROM akamai.logs
        WHERE $__timeFilter(reqTimeSec)
          AND statusCode IN ('200', '304')
          AND reqHost = 'www.yoursite.com'
          AND rspContentType = 'text/html'
          AND UA LIKE '%ChatGPT-User%'
    ),
    sessions AS (
        SELECT
            *,
            sum(
                CASE
                    WHEN time_diff IS NULL THEN 1
                    WHEN time_diff > 5000 THEN 1
                    ELSE 0
                END
            ) OVER (
                PARTITION BY cliIP, reqPath
                ORDER BY reqTimeSec ASC
                ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
            ) AS session_id
        FROM base
    )
    SELECT
        cliIP,
        session_id,
        reqPath,
        accLang,
        min(reqTimeSec) AS session_start_time,
        toUnixTimestamp64Milli(max(reqTimeSec)) - toUnixTimestamp64Milli(min(reqTimeSec)) AS duration_ms,
        COUNT(*) AS count_requests,
        groupArray(reqPath) AS reqpath
    FROM sessions
    GROUP BY
        cliIP,
        reqPath,
        accLang,
        session_id
)
GROUP BY reqPath
ORDER BY cleaned_bot_visits DESC

Reconnect the URL path to the domain

Export the logs into a CSV file and concatenate the reqPath against the domain.

Run GSC API crawl to Get Clicks and Impressions

Use either Data Studio or search analytics for sheets to fetch impressions and clicks for ChatGPT-user-requested URL paths.

URL Paths Pre-Processing & Clustering

The next step is pre-processing URL paths so we can convert them into traditional search queries and group them into clean clusters.

You can crack on with a Google Colab, as it covers this section.

 FIRST MAKE A COPY OF THIS COLAB

The script will:

• Normalise URL paths similar to search queries.
• Perform HDBSCAN clustering and assign a score to each similar URL.

The choice of the number of clusters is arbitrary, but dependent on how many rows you managed to export from the above query.

If you end up with around 1000 rows, aim for 15-30 clusters – adjust MIN_CLUSTER_SIZE (20 is a good start)

Once fine-tuned to your decisions, run the script. A similar output will be returned.

The output file will be saved in a CSV file.

Make sure to open it and remove all rows where cluster_probability < 1 – this might contain outliers from irrelevant cluster assignments.

One-Hot Encoding to Rename Clusters 

Upload the document to Gemini, Claude or ChatGPT and ask to perform a simple NLP task, such as translating your HDBSCAN cluster scores into human-readable topic labels.

I’m not going to recommend one LLM or another – Gemini 3 is now the default model for AI Overviews and ChatGPT may handle content better than Claude.

However, turning numbers into words is a basic NLP operation that shouldn’t make a significant difference between models.

Prompt example:

Give me a non-generic content topic that represents all pages in each cluster. Populate the cluster_label column using the table copied from my clipboard.

As an example, I used Gemini

And copied and pasted the suggestions in the CSV file to ultimately operate a VLOOKUP.

That’s how I mapped out cluster numbers to their corresponding topic labels.

Now, you have two options based on the strategic driver:

Clearly, the second option is usually the most requested in SEO services.

Below is an example of how that tactic will render in practice based on the process detailed in this article.

At this point, you might consider submitting these prompts to tracking platforms such as Peec AI – again, this is not a sponsored link!

Server Logs are not without Limitations

In the previous article, we noted that proprietary data may not reflect the full picture, as the multiple preprocessing steps involved can introduce confirmation bias.

Similarly, this approach also comes with some caveats.

1. The link between the generated response and the referenced source is not always factual; in some cases, the citation is produced by template logic rather than true source attribution.
2. LLMs tend to provide sources or citations, even when they aren’t fully accurate. Tracking citations shows how confident the model is, not necessarily what’s correct.
3. This method doesn’t scale well unless you run it on your own server.
   Pulling more than 30 days of logs can use a lot of resources and may trigger limits from your CDN—this happened to me when I overqueried.

AI Search Tracking moves from Attribution to Understanding

Tracking AI visibility is not about chasing perfect attribution. It is about building directional clarity in an ecosystem where traditional analytics no longer apply. 

By combining first-party search data with server-side signals, you can move from speculation to structured observation — identifying patterns that explain how AI systems surface your brand.

This framework will not tell you exactly which prompt, topic, query or keyword triggered an answer, but it will help you understand why your content is being selected.

Simone De Palma

Simone De Palma is an SEO Manager at TUI and the founder of SEO Depths.

He graduated in Marketing and Management from Università IULM before completing a degree in Digital Marketing and Data Science at Leeds Beckett University.
Simone has worked as an SEO Specialist in digital agencies in Italy and the United Kingdom and he’s a contributor for the Search Engine Land and Majestic SEO podcast

When he’s away from his double screens, he enjoys cooling down with a refreshing swim at the pool. You could find him exploring art museums or enjoying the company of a classic romance

Add your content…

• Linkedin