Crawled – Currently Not Indexed

Google’s own description is short. In the Page indexing report, the status means the page was crawled by Google but not indexed, and it may or may not be indexed in the future; there is no need to resubmit the URL for crawling^[1]. It sits among the reasons a page is not indexed, which the report introduces as pages that have not been indexed, “but not necessarily because of an error”^[1].

It marks a gap between two stages. Google Search works in three stages, crawling, indexing and serving search results, and not all pages make it through each one^[2]. Google says that indexing is not guaranteed and that not every page it processes will be indexed, and it names common causes: low quality of the content, robots meta rules that disallow indexing and a site design that makes indexing difficult^[2]. The status itself does not say which applies.

Many URLs in the status are there for good reason. Google tells site owners not to expect every URL to be indexed, because some are duplicates or hold nothing meaningful, and says that non-indexed URLs can be fine when the reason is right^[1]. What matters is whether the pages in the status are the ones the site wants.

The status is also hard to test. Google lists it among the indexing issues that the live test of its URL Inspection tool cannot test^[3]. A change is judged by the report weeks later, not by testing the page the same day.

Any count of it needs a base. The same number of URLs is a different figure when divided by the URLs Google knows, by the URLs in the sitemap or by the URLs the site wants. In QueryRev’s study, the 6,294 sites whose robots.txt could be read are 63.1% of the 9,973 domains of the panel, and the study names the base beside every share^[4].

In the Page indexing report, open the row for the status in the table of reasons pages are not indexed. It shows the affected URLs and the history of the count. The list of example URLs is limited to 1,000 and is not guaranteed to show every URL in a status^[1], so large sites sample it, or filter the report by sitemap to read one section at a time^[5].

Sort the URLs into two groups: those the site wants in search, and those it does not, such as filter and sort combinations and internal search pages. The first group is the finding. Its measure is the share of wanted URLs that sit in the status, which is the wanted URLs in the status divided by all the wanted URLs of the same section.

For single URLs, use the URL Inspection tool, which shows the indexed state of a page, how Google found it, its last crawl and the canonical Google chose^[3]. The report itself cannot be searched or filtered by URL^[1]. Track the count and the share by month, and note what changed on the pages before it moved.

Sample data: an invented site, to show the method.

A sample store opens the status and finds 1,200 URLs. A sample of the list sorts them: 640 are filter and sort combinations and 360 are internal search pages, none of which the store wants indexed. The other 200 are product pages it wants in search.

The 1,000 need nothing. For the 200 the store reads the pages one by one: many share a description with their sibling colour variants, and about half are reachable only through the sitemap. It writes its own description for each product, links the pages from their categories and consolidates the colour variants under one canonical. It then watches the 200 for several weeks, since a live test cannot confirm the change, and expects some to stay in the status.