Again in Could Google’s Gary Illyes sat for an interview on the SERP Conf 2024 convention in Bulgaria and answered a query in regards to the causes of crawled however not listed, providing a number of causes which might be useful for debugging and fixing this error.
Though the interview occurred in Could, the video of the interview went underreported and never many individuals have really watched it. I solely heard of it as a result of the all the time superior Olesia Korobka (@Giridja) just lately drew consideration to the interview in a Fb publish.
So despite the fact that the interview occurred in Could, the data continues to be well timed and helpful.
Cause For Crawled – Presently Not Listed
Crawled Presently Not Listed is a reference to an error report within the Google Search Console Web page Indexing report which alerts {that a} web page was crawled by Google however was not listed.
Throughout a reside interview somebody submitted a query, asking:
“Can crawled however not listed be a results of a web page being too just like different stuff already listed?
So is Google suggesting there’s sufficient different stuff already and your stuff shouldn’t be distinctive sufficient?”
Google’s search console documentation doesn’t present a solution as to why Google might crawl a web page and never index it, so it’s a official query.
Gary Illyes answered that sure, one of many causes may very well be that there’s already different content material that’s comparable. However he additionally goes on to say that there are different causes, too.
He answered:
“Yeah, that that may very well be one factor that it could possibly imply. Crawled however not listed is, ideally we might break up that class into extra granular chunks, but it surely’s tremendous laborious due to how the info internally exists.
It may be a bunch of issues, dupe elimination is a kind of issues, the place we crawl the web page after which we resolve to not index it as a result of there’s already a model of that or an especially comparable model of that content material out there in our index and it has higher indicators.
However yeah, but it surely it may be a number of issues.”
Basic High quality Of Website Can Influence Indexing
Gary then referred to as consideration to a different motive why Google would possibly crawl however select to not index a website, saying that it may very well be a website high quality subject.
Illyes then continued his reply:
“And the final high quality of the of the location, that may matter plenty of what number of of those crawled however not listed you see in search console. If the variety of these URLs could be very excessive that would trace at basic high quality points.
And I’ve seen that so much since February, the place out of the blue we simply determined that we’re indexing an unlimited quantity of URLs on a website simply because …our notion of the location has modified.”
Different Causes For Crawled Not Listed
Gary subsequent supplied different causes for why URLs is perhaps crawled however not listed, saying that it may very well be that Google’s notion of the location might have modified however that it may very well be a technical subject.
Gary defined:
“…And one risk is that once you see that quantity rising, that the notion of… Google’s notion of the location has modified, that may very well be one factor.
However then there is also that there was an error, for instance on the location after which it served the identical actual web page to each single URL on the location. That is also one of many causes that you just see that quantity climbing.
So yeah, there may very well be many issues.”
Takeaways
Gary supplied solutions that ought to assist debug why an online web page is perhaps crawled however not listed by Google.
- Content material is just like content material already ranked within the search engine outcomes pages (SERPs)
- Very same content material exists on one other website that has higher indicators
- Basic website high quality points
- Technical points
Though Illyes didn’t elaborate on what he meant about one other website with higher indicators, I’m pretty sure that he’s describing the state of affairs when a website syndicates its content material to a different website and Google chooses to rank the opposite website for the content material and never the unique writer.
Watch Gary reply this query on the 9 minute mark of the recorded interview:
Featured Picture by Shutterstock/Roman Samborskyi
