Google Indexing API Not Sending Rendering Bots - Crawl or No Crawl
Welcome to Crawl or No Crawl - this is Carolyn Holzman and today is Nov 10 Friday
Quick housekeeping notes and then I’ll share the latest results
I’m in the process of moving this research over to SUbstack - where it will become a subscription resource so to keep receiving all the daily information and access to the research you’ll need to become a paid subscriber or as I see it a sponsor of the work and as a result only you and those like you will have access to this timely information.
There will be a free option which will be infrequent updates - and probably not during the crazy days like they are right now - and you can always jump in later I just can’t promise it will be at the same price.
Here’s the link to where you can at least sign up on the early bird list to get notified when you can subscribe - the emails will come from substack -
https://crawlornocrawl.substack.com/p...
It will start out at $18 a month - hosted on Substack and you’ll receive the daily emails and the posts will have the videos along with the written data and other resources - so it would be somewhere between .75 - .85 cents per report but just know you’re seeing information that the whole world is not.
This work is taking up a lot of my day and bandwidth and that is the only reason why I’m not publishing videos daily - \\ I’m cutting back on some other activity so that I can be able to do go back to a more frequent schedule very quickly and definitely in this interesting Q4 timeframe.
Some of you have been asking about The indexer testing is coming along but as you will see - all indexing and serving is running at a subpar level at this time and I’m choosing not to launch this when google’s whole indexation system is at their own beta stage - see right there, this research is giving value so you know why your new content and revised content might be having challenges. Esp if you’re relying on the Google Indexing API - I do plan to provide indexation tool subscribers with similar access to the research within the system. But this subscription will have more details and background while the indexer will provide general information on the indexation environment- expected number of days etc on an ongoing basis.
I can not stress enough that the Indexing API is NOT what it was this time last year - from data in the server logs it does NOT send the rendering bots - you get one html crawl and that’s it - if ANY of your optimizations. -either because you meant to put them in there or your wordpress theme puts them in there - google is NOT seeing that if you’re relying on the API to index.
Some recent developments from last week - and I’m still digging into the data - which requires sifting through the server logs
We had new chrome bots, inspection tool bots update - another core update launch. Which you can see the recrawing in the logs - and as of yesterday a reviews update - all that will impact the ranking of your content.
PLUS confirmation that Google says - we’re all at mobile indexing first - so what I think you may see more of in search console as for why your content isn’t indexed therefore not able to be served - is going to be mobile related - In a post on substack you’ll see the details on a page that was published on Oct 30th - and that is having issues around mobility -
I did have a video and some images that extended beyond the viewport - but even after correcting for that - google is still reporting an issue - it maybe that there is a bug or a delay in clearing that protocol buffer - we’ll see
Search console continues to provide faster indexation and javascript rendering crawls -
The Indexing API continues to be slow - I’ll also post about the test pages where they were published on the 5th and the 7th this week - and in the post I’ll share what I think contributed to them being found yesterday -
Thanks for coming by - please make sure to at least get on the notification list on substack - I’m be publishing here for a little while longer as we make the shift - and this channel will continue.
Get on the list!
https://crawlornocrawl.substack.com/p...