Robotic Process Automation: Google Search Engine Result Page (SERP) Analysis for SEO

“The best place to hide a dead body is on the second page of Google search results”. This internet meme indicates the importance of being on the first page of the search engine be it Google, Bing or Yahoo. Research shows that more than 50% of all search engine users click on a result on the first page. Hence keyword ranking has a major impact on web traffic, lead generation and conversions.

To track keyword ranking of websites, the crucial aspect is to focus on the relevant keyword. In most cases, people rely heavily on paid third party keywords research analysis tools.

But the issue with these tools i.e they are not personalized. The keywords they return might not necessarily provide the right keywords that are business relevant.

The daily task of our SEO company was to manually find the Search Engine Ranking Position (SERP) for the keyword data set relevant to our various business regions. Beaking down, her task involved manually searching for each keyword based on the relevant region (For example UAE) for the company website ( and creating an excel sheet with each keyword and the corresponding rank position and page on the search result.

This is a highly tedious task which would eat up more than an hour. Our team at the Verbat Innovation lab aimed to automate the keyword Search Engine Rank Page Analysis (SERP) using the RPA platform.

What is Robotic Process Automation (RPA)?

Robotic Process Automation (RPA) is a relatively new concept and hasn’t enjoyed the same fame as Artificial Intelligence and Machine Learning. Giving it a simple definition, RPA can be defined as a software tool that partially or completely automates manual and repetitive tasks performed by a human. These tools imitate human behaviour on software applications and help organisations automate basic daily tasks.

Small and medium-sized organizations are set to benefit from this technology as these employ a handful of people to handle a myriad of tasks, including lowering operational costs, maximizing productivity and improving the quality of products and services. The major technology providers offering RPA tools are UiPath and Blue Prism.

We have used the UiPath as our RPA platform. 

System Work Flow

The system workflow is designed by utilizing the web scraping feature of UiPath. 

Web scraping technique involves automatic logging into a website, data extraction, and transforming into a required format of another application. Our automation program used the excel application scope to extract data from the excel sheet containing the keyword data set and also to transfer the extracted data to an excel sheet.  

The steps involved include

Step 1: Browser is set in incognito.
Step 2: The region relevant to the search (For example UAE) is set.
Step 3: The 1st keyword is selected and pasted on the web browser from the excel sheet containing the keyword data set using excel application scope.
Step 4: Position and page position of for the keyword is extracted from the Google search result.
Step 4: Another excel table is created and the searched keyword and the corresponding search result position is pasted on to the excel sheet.
Step 5: The process is repeated for all the remaining keywords on the keyword data set excel sheet.

Manual SERP analysis for would normally take around 2 minutes for each keyword not to mention the effort involved. On the other hand, the Automation process using RPA took around 7 seconds for each keyword. The error rate was reduced to 0%, manual effort to 80% and processing time reduced to 50%. 

it companies in dubai mobile app development companies uae Web Hosting UAE |