Crawl or No Crawl Mar 3 2022 - Day 16
Welcome to Crawl or No Crawl
Wed Mar 3rd 2022
Semrush Sensor is reading 5.1 -
So Let’s play Crawl or No Crawl. [screen share]
Today officially isn’t a test day - but yesterday there was a little snafu in the auto publish feature of yesterday’s test page - so I pushed it into google indexing api - and today got a discovery full render crawl to the page
And index now got a bing discovery full render crawl
If you want to have access to a discussion on indexing and specifically how NOT to ask Google to index your new content - tune into Seo fight club next Tuesday at 11 AM CST - I’ll be sharing that discovery on the show and going deep into new content indexation techniques
Link to SEO Fight Club
/ @seofightclub
Link to Google Index Detector
https://www.AmericanWayMedia.com/Inde...
Now to the server logs -
Google -
51 bot server records -
Things google picked up -
ads.txt
robots.txt
Today however - no sitemaps which is not typical
Of the 51 server logs for google bot
39 were mobile
Desktop 12
They both seemed to crawl the same types of files - didn’t see a pattern to only using one for a certain file type
Now to Bing -
25 bingbot records
And Again. - NO mobile bots today - only desktop
Bingpreview came - pulled some page resource files -
I did see a validator crawl -
Most of the time that type of crawl is where it’s www or no www -
This time it was the trailing slash vs no slash at the end of the URL
Those things matter - but it’s the consistency that is what matters - wordpress is already out of the box adding the trailing slash - which is why if you put a URL in search console without one - it says that that URL is NOT in google - know before you go I say
Thanks for playing Crawl or No Crawl
Subscribe so you’re the first to know what is happening.
Confessions of an SEO new episode this Friday - wherever you get your podcasts
https://www.podchaser.com/podcasts/co...