Itโs pretty common for eCommerce site owners to fret over missed revenue potential when key pages arenโt getting indexed by search engines.
A big pain point is when Google discovers product pages but doesnโt index them. This can be especially frustrating, as Google has tightened its indexation rules in recent years to crack down on duplicate content (which makes up nearly 60% of the web) and low-quality, AI-generated content.
For eCommerce sites with thousands of pages, getting more pages indexed can be a game-changer for boosting conversions and revenue. But that pesky โDiscovered โ currently not indexedโ status in Google Search Console is a tricky one, especially for sites with big catalogs, loads of product variants, or dynamic content.
In this post, Iโll break down what โDiscovered โ currently not indexedโ really means, why it happens, and how you can tackle it to keep it under control.
What Does Discovered โ Currently Not Indexed Mean?
When Googlebot crawls your site, it finds URLs through sources like XML sitemaps, internal links, and the siteโs HTML. But just because a URL is discovered doesnโt mean itโll be crawled or indexed.
The โDiscovered โ currently not indexedโ status means Google knows the URL exists, but it hasnโt yet decided to crawl and index that page.
Why Pages Remain Discovered and Not Crawled
There are a couple of main reasons for pages to remain stuck in this status: server performance and the quality of the websiteโs content.
๐กBONUS H/T
Googleโs Martin Splitt has recently nailed down this topic in a short YouTube video. Find it here for a more in-depth take
Server Performance Issues
If your server struggles to handle crawling requests for a large number of URLs, it can significantly delay Googlebotโs ability to process all pages. This issue is fairly common for large eCommerce sites with thousands of product variants.
When Googlebot tries to crawl multiple URLs at once but hits a slow or overloaded server, some pages might get stuck as โdiscoveredโ without being fully crawled.
This lag leaves many pages in despair, especially if your server struggles to handle the crawling load.
๐กBONUS READ
Wouldnโt you say images can be among the most common eCommerce issues? Hereโs the top 3 issues with images on luxury online shops
Website Quality and Content Issues
Googlebot prioritizes pages based on their perceived quality. If your website has thin content, duplicate pages, or low-quality content (e.g., spammy or unhelpful pages), Google may decide to skip crawling those pages altogether.
This is particularly relevant for product pages with minimal content or duplicate product variants. For example, if Googlebot detects that multiple pages on your site provide nearly identical content, it may choose not to index them, leaving them in the โdiscoveredโ state.
Other Not-So-Common Reasons
Beyond server and content issues, several other factors may contribute to leaving pages in the Discovery limbo:
- XML Sitemap Errors
If your sitemap throws an HTTP 3xx or HTTP 4xx status code, Googlebot may be unable to crawl pages that are primarily discovered through the sitemap. Even if the URLs are discovered, they may not be crawled or indexed. - Canonical Tag Errors
Issues like canonical loops, chains, or incorrect tags (e.g., extra characters) can confuse Googlebot, causing it to skip crawling certain pages. - JavaScript Rendering Issues
If your website relies heavily on JavaScript, the initial HTML response may show minimal content or identical code across multiple pages. This can result in Google treating pages as duplicates, delaying the rendering and indexing process. - Duplicate Content
Google tends to store one main version of a page while flagging others as duplicates. This can occur when multiple URLs lead to the same content, often due to issues like trailing slashes, capitalization differences, or URL parameters (e.g., UTM tags). - Misleading Internationalisation Signals
If your eCommerce translations are inconsistent across different country versions, search engines might get mixed signals. This can lead to pages lingering in the โDiscoveredโ phase without ever being crawled or indexed.
Debugging Discovered โ Currently Not Indexed
Addressing this issue requires a strategic approach to identify the root cause.
Here are some steps you can take:
- Rule Out Rendering Issues
Since your time is valuable, start by inspecting a sample page with the URL Inspection tool in Google Search Console. Check if Googlebot can fully render the page. Look in the โOther Errorsโ tab to see if any critical resources fail to load properly, and if they do, loop in your developers. - Check Crawl Stats
In Google Search Console, review crawl stats to see if your server is responding slowly or returning HTTP 5xx errors. A high error rate could point to server problems that need fixing. - Test Dynamic Elements
Use the URL Inspection tool to manually check pages that rely on โLoad Moreโ buttons or other dynamic elements to ensure theyโre loading correctly. - Investigate Content Quality Issues
Review the on-page content to ensure itโs descriptive, unique, and matches the search intent for each affected page template. Quality content plays a big role in whether Google decides to index a page.
How to Fix Discovered โ Currently Not Indexed
As mentioned earlier, Google is becoming more selective about which new content it accepts into the SERP. Once you identify the root causes, here are some targeted actions you can take:
- Fix Rendering Issues
If JavaScript is causing rendering delays, consider solutions like pre-rendering or server-side rendering (SSR) to make sure content is accessible before bots try to index it. - Enhance Server Performance
If your server is slow, work with your hosting provider to explore ways to boost performance. This could involve upgrading server resources, optimizing your setup, or adding caching strategies. - Address Thin or Low-Quality Content
Merge thin content with related pages to create more valuable resources, or remove low-quality pages entirely. For pages that donโt target organic search, use the โnoindexโ meta tag to prevent them from cluttering the index. Optimize Sitemap and Canonical Strategies
While it may not always be necessary, itโs important to double-check that your XML sitemap is valid and returns a 2xx status code. Also, verify that canonical tags are correctly set up. If you want a specific version indexed, use a self-referencing canonical tag and divert non-canonical URLs (e.g., URLs with trailing slashes or UTM parameters) to the main version.- Ensure Consistent Translations Across Meta Tags and Content
This is key for eCommerce targeting multiple countries. Itโs not just about getting the hreflang tags rightโsearch engines look at everything, from URLs and meta tags to HTML annotations and on-page content, to determine the correct language for your site. Make sure translations are consistent across the board and avoid confusing search engines even before to get the crawling started.
Conclusion
The โDiscovered โ currently not indexedโ issue in Google Search Console is a common source of concern for large eCommerce sites, particularly those with large product catalogs and dynamic content.
By identifying the root causes and addressing server performance, content quality, and technical issues, you can improve your siteโs chances of getting fully crawled and indexed by Google.
Implementing these best practices should help move many of those discovered pages into the crawling queue, leading to better indexing, more traffic, and increased visibility for your website.
FAQ
What does โDiscovered โ Currently Not Indexedโ mean?
The โDiscovered โ Currently Not Indexedโ status in Google Search Console indicates that Google is aware of a URL but hasnโt yet decided to crawl and index the page. This can happen even though the URL has been found via sources like XML sitemaps or internal links.
Why do pages remain in the โDiscovered โ Currently Not Indexedโ state?
Pages often stay in this state due to server performance issues or low-quality content. A slow server may struggle with handling multiple crawl requests, while pages with thin or duplicate content might be deprioritized by Google for crawling and indexing.
How can server performance affect indexing?
If your server is slow or overloaded, Googlebot might not be able to fully crawl all discovered URLs. This can leave many pages stuck in the โdiscoveredโ state without being crawled, which is particularly common for large eCommerce sites with many product variants.
What are some other reasons pages might not be indexed?
Other reasons include XML sitemap errors, incorrect canonical tags, JavaScript rendering issues, crawl budget limitations, and duplicate content. These factors can prevent Google from properly crawling or prioritizing certain pages.
How can you fix โDiscovered โ Currently Not Indexedโ issues?
Solutions include fixing rendering issues with pre-rendering or server-side rendering (SSR), enhancing server performance, improving content quality by merging thin content or removing low-quality pages, and ensuring your XML sitemap and canonical tags are correctly set up.