In this episode of SOTR, John Mueller, Lizzi Sassman, and Gary Illyes talk about misconceptions around crawl frequency and site quality, what's challenging about crawling the web nowadays, and how search engines could crawl more efficiently.
Resources:
Episode transcript → [ Ссылка ]
Gary's post on LinkedIn → [ Ссылка ]
Crawling episode with Dave Smart → [ Ссылка ]
If-Modified-Since → [ Ссылка ]
About the IETF → [ Ссылка ]
Robots Exclusion Protocol → [ Ссылка ]
Proposal for new kind of chunked transfer → [ Ссылка ]
Watch more Search Off the Record → [ Ссылка ]
Subscribe to Google Search Channel → [ Ссылка ]
Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
#SOTRpodcast
Speaker: John Mueller, Lizzi Sassman, Gary Illyes
Products Mentioned: Search Console - General
