Google posted a brand new “Search Off The File” podcast yesterday on the subject of crawling the place John Mueller, Lizzi Sassman, and Gary Illyes spoke about how Google crawls, some concepts on the way to make crawling extra environment friendly and a few misconceptions round crawl frequency and web site high quality.
I do not suppose something on this podcast is especially new, exterior of Gary saying Google is trying right into a “new sort of chunked switch” proposed by somebody on the IETF Monitor, Web Engineering Activity Drive. IETF is a requirements physique which can also be the place Robots Exclusion Protocol is a part of.
So Gary stated, “I am watching that carefully to see the place it is going.” If the “new sort of chunked switch” works, that may make crawling a bit extra environment friendly.
Gary did say considered one of his initiatives in 2024 was to look to make crawling more efficient and that he wants to crawl less.
I hoped they’d point out IndexNow from Bing however they didn’t.
It’s a good pay attention, hit play and pay attention within the background:
Discussion board dialogue at X.
