Detecting Headerless Bot Traffic in Google Analytics

While the Google Analytics Bot Filtering option does a good job, it doesn’t always remove all bots.

During our 100+ Google Analytics audits, we discovered sites where well-meaning IT departments and developers deployed bots and forgot to tell the analytics teams.

In one case, the bot accounted for 35% of all traffic on a site that had 325,000 visits a month.

In our experience, bots replicate desktop users, and we’ve detailed a simple way to detect them below

For our audits we use the Analytics Edge Excel plugin to download the same information from Google Analytics as it makes the whole process a lot quicker.

The following report details the same methodology that we use

1) Create a custom report that gets the following information.

Screen Resolution is the size reported for the whole of the users’ screen.

Browser Size is the size of the users’ viewport and is the space available actually to show the website they are looking at

Install the custom report from the Google Analytics Gallery

2) Export the information as an Excel file. If you have a lot of rows, you may need to make multiple exports and then combine (this is why Analytics Edge is so useful as it grabs all the data into one spreadsheet)

3) Load the data into Excel and create two new columns called Screen Height and Screen Resolution

3) Use the Excel formula =RIGHT(C2,(LEN(C2)-FIND(“x”,C2))) to split out the height from the width for the screen resolution

4) Use the Use the Excel formula =RIGHT(E2,(LEN(E2)-FIND(“x”,E2))) to find the browser height

5) Create a new column and subtract the Browser Size from the Browser Resolution

6) Most browsers will have a difference of around 80 to 150 pixels depending on how many toolbars they have have

7) Where there is no difference, you’ve got a headerless browser running.

In the screenshot below you can see it’s coming from an older version of Chrome which gives us further indication of bot activity

8) The next thing to then look is the bounce rate to see this far higher than than average. In this case, a bounce of rate of 99.7% strong suggested bot activity and with all of it coming from one network, an advanced filter was setup to exclude it

We’ve audited over 100 Google Analytics accounts and in 95% of them we’ve found at least one serious issue. If you’re interested in finding out how we can ensure you have clean and useful Google Analytics data, please contact us