Again in mid-June, I seen that Google was not exhibiting a lot of my photographs in Google Search and Uncover and likewise some readers had been pointing it out to me. So I used the useful Google Search Console URL Inspection device to search out out these S3 URLs I used to be utilizing to host my photographs had been blocking Googlebot from crawling. Here’s a little bit of a case research from yours actually of an indexing/crawling concern I had for my picture URLs.
This AWS bug led to an 83% drop within the impressions my photographs had been getting from Google Search and Google Pictures. It led to a 76% drop in picture search associated clicks to this website. I’m nonetheless down a number of weeks later by about 16% in impressions and 26% in clicks from picture search however it’s a big enchancment.
Right here is the Google Search Console Search Efficiency report exhibiting the impressions and clicks chart over time. You will note the drop round June fifteenth, then it begin to choose again up round July eighth. Additionally, you will see that my picture visitors has nonetheless not totally returned to its regular numbers pre-AWS bug, even after two months:
When Googlebot was attempting to entry my picture URLs on S3, Google was getting a 404 not discovered error. However after I visited the URLs with my pc, they loaded simply fantastic. These are the identical picture URLs I’ve been utilizing on this website for properly over a decade and poof, at some point, AWS determined to dam Googlebot. I reached out to each Google and AWS in regards to the concern and I believe it was a fairly large concern. Tons of websites use S3 for picture and file storage, so Googlebot was possible getting tons of 404 errors. The bizarre half is that I noticed zero public complaints in regards to the concern.
In any occasion, that is what Googlebot noticed once they tried to crawl these URLs:
AWS fastened it after a number of days:
That is what my photographs seemed like within the URL Inspection device in Google Search Console:
It ought to look one thing like this:
Since then, I made a decision to maneuver my photographs to AWS’s CloudFront – a service that was not obtainable after I first made this website – which is why I used S3 again then for photographs. The S3 concern with Googlebot continues to be fastened and dealing fantastic. However I’m not going again to S3 for photographs.
I ought to thank Glenn Gabe for additionally noticing the photographs going away early on in Google Uncover. Glenn additionally wrote up this image migration article which I reviewed earlier than making the swap from AWS S3 to AWS CloudFront. I didn’t migrate my outdated photographs, I left them, as a result of AWS fastened the difficulty. However since late June, all my new photographs are utilizing CloudFront.
To be clear, this was not a Google bug, however an AWS change that led to AWS S3 blocking Googlebot. It’s now resolved but it surely looks like the injury has been achieved… If the graphs change extra, I’ll replace this story under to doc the adjustments. However to this point, it has been flat for the previous 5 weeks or so, so I’m not anticipating massive adjustments sooner or later.
Discussion board dialogue at X.
