How to Quickly Check the Backlink Profile of Over 5,000 URLs
Jan25

How to Quickly Check the Backlink Profile of Over 5,000 URLs

18SHARESFacebookTwitter Migrating to new Content Management Systems (CMS’s) can be difficult at times, especially if the site that is being migrated has thousands of duplicated URLs that have high quality backlinks associated with them. If it’s your job then you better come up with a solution and hopefully this article will provide a few. I was recently tasked to analyse or come up with a way to segment these 5,000 URLs to make it more manageable. I’ll admit I don’t have much experience when it comes to migrating very large websites, so it took me a couple hours to come up with an idea as to how we could go forward. Rather than sit like a lemon all day long and ignore all of these soft 404s that were popping up left, right and centre in Google Webmaster Tools I came up with a few ideas and utilised a few industry tools to help. The first idea that I came up with was using Regular Expressions in Excel to categorise each URL based on the keywords that are in the actual URL. I then realised that I was doing this the wrong way round. The idea behind this was to categorise these URLs and to write a rule to say that if a URL contains a certain keyword then it should redirect to the most appropriate page that likely contains the same keyword. The problem with this is that it isn’t accurate and you could possibly be redirecting URLs to a page that isn’t actually that relevant. So what should I do instead? Seeing as this client is quite large, it’s worth going through to see what the backlink profile is for each individual URL that we grabbed from Webmaster Tools (over 5,000). And that’s easier said than done if you don’t have the right tools. Luckily for me, the agency I work at has all the tools that you’d ever want for this to work. ScrapeBox or MajesticSEO At first I tried using ScrapeBox on a VPS that we have to scour the backlink profile for all of those 5,000 URLs using the Mozscape API. I left this running over night, but when I got home I looked for other solutions as I knew that Moz’s Index isn’t that large and I might not get all the backlink data as a result. I found out that MajesticSEO has a tool called the “Bulk Backlink Checker“. At this point I celebrated as I’ve found that MajesticSEO is arguably one of the best backlink checking tools in the SEO industry. I simply pasted all 5,000 URLs into a .txt file and...

Read More