Can Google Crawl Textual Content Within JavaScript

JavaScript JS Can Google Crawl Textual Content Within JavaScript

Google has increasingly become pragmatic when it comes to crawling textual content hidden within JavaScript. A perfect example of where Google has confirmed this would by using Google’s head of spam (or at least using a video that explains it all). Matt Cutts has openly stated in a Google Webmaster Help video that you should not block Googlebot from crawling JavaScript or CSS; this was 2 years ago. Since then Google has advanced the way Googlebot detects legitimate content on a page that might have ‘hidden’ content within JavaScript for the purpose of UX.

New …or Not So New Webmaster Help Video

Just 4 days ago, Matt Cutts, released another video on this subject where he talked about content being hidden essentially within JavaScript/AJAX on the basis of UX. In this video he stated that Google “has become pretty good at indexing JavaScript and being able to render this in to our search results.” See the video below:

I do have to add, before I continue, that Matt Cutts posted this video 4 days ago, but states in the video that it was “recorded on May 8th, 2013”.

Misconception of JavaScript in SEO

There’s a common misconception that Google cannot render anything in JavaScript or that it’s not best practice to have content hidden within JavaScript. But then you’re then thinking too much about optimising for search engines, rather than people. People don’t want to read a full page of text (unless you’re on Wikipedia via desktop) and would rather have content segmented with perhaps the help of AJAX. A good example of this is when it comes to mobile, is to have content segmented, so that the person on mobile has the ability to easily navigate to part of the content they wish to see. If you’ve ever browsed Wikipedia via mobile you’ll see that it uses AJAX to hide content, so that you don’t have to navigate through a massive blob of text to get to the part you want to. This is all down to user design, and Google has obviously identified this as a common theme on many websites.

Using iframes to pull content externally into Lightboxes

What’s interesting, for me at least, is the fact that in that same video I mentioned above, Matt Cutts explained how Google is working on pulling content via iframes and states they are just a “couple months’ away” from achieving this. Remember: This was on the 8th of May, 2013 and that 2 months has long gone.

So I checked to see if this was the case, as 4 months ago, a client for the agency I work at used an iframe to pull external content into a lightbox on a different page. What happened was that the external source was indexed into Google and didn’t really add any content (in the search engines eyes) to the actual product page. I somehow figured this out, and stopped them from continuing this practice and they’ve since used more SEO-friendly lightboxes that use content hidden on the actual page with JavaScript. When you type an exact match sentence you’ll see the text appears in the meta-description for that webpage, which obviously means Google is able to crawl this sort of content. You can also tell that Googlebot can crawl text within JavaScript if you actually disable JavaScript in you browser and can see the text that’s hidden within the JavaScript on the webpage.

Whilst it’s recommended that you should not use JavaScript excessively, it still has a vital role to play and can still be crawled/indexed by Google. For other search engines, I’m not sure at what level they are able to crawl/index JavaScript – which is why it’s best to keep JS/AJAX usage to a minimum, if possible.

Author: Jonathan Jones

I first first started creating websites back in 2005. This led me to creating a free web hosting business in 2007, which still exists today. I ventured into creating types of websites such as blogs and forums using Wordpress, vBulletin, Invision Power Board, Drupal and Joomla. I've since worked on some of the leading brands in the UK finance sector, in the Search Engine Optimisation (SEO) field, and now work for MoneySuperMarket, the #1 price comparison website in the UK. Social: Google+ and Twitter.

Share This Post On