Best Way to Block HTTP Referrer Spam with Nginx

Referrer Spam Domains
Referrer Spam Domains

If you’ve watched your website analytics for any length of time, you’ve probably seen referrer spam.  Your web traffic appears to be growing, and not only that, you’re getting referral traffic from other sites!  But wait just a second, is your traffic coming from domains like the ones in this list. If so, don’t get excited, it’s not real traffic…those sites aren’t actually backlinking to your website! This is known as referrer spam (also known as referral spam, log spam or referrer bombing).

What is Referrer Spam?

Like email spam, the goal of referrer spam is to build web traffic to the spammer’s site or domain.  In the case of bogus referral traffic, the goal is to flood webserver logs with entries that are intended to become backlinks to the spammer’s domain.  Of course, for these log entries to become valuable backlinks, the logs must be public on the Internet and get indexed by search engines…who puts their logs on the internet?!  …it must be effective though, because the spam bots just keep on spamming!

Do Bogus Referrers Hurt My Website?

No.  To your website, it’s just traffic.  The problem this fake traffic causes, is in your analytics…especially if your web traffic is low.  It will look like you have more traffic than you really do, it will drive up your bounce rate, and it will generally make your traffic analytics harder to interpret.

How to Block Referrer Spam with Nginx

The good news is that blocking referrer spam isn’t hard.  In Nginx, all you have to do is create a separate config file with all the domains you wish to block, and then include that file in your main Nginx config.  So your nginx.conf file might look like this.

http {
...
include   referrers.bad;
...
}

I like to include this referrers.bad file in my main Nginx config so that it is applied to visitors to all websites hosted on this server.  You could include it elsewhere if that was a better fit for your needs.

Next create the referrers.bad file.  This file will use the map directive that contains a list of all the domains you want to block.  Here is an example.

map $http_referer $bad_referer {
default 0;

"~*blockme\.com" 1;
"~*blockmetoo\.com" 1;
}

This block of configuration maps the value of the $http_referer server variable to a value that is stored in a variable called $bad_referer.  You can use this $bad_referer variable elsewhere in your server config.  In a server block, for example, you can return a 404 response to the visitor if the $bad_referer variable is set to 1.

if ($bad_referer) {
return 404;
}

Keeping Up with the Spammers

The above method is an effective way of blocking traffic from a list of domains.  The problem becomes keeping that list up-to-date can be challenging.  Fortunately, there is a Github project dedicated to building and maintaining this list.

https://github.com/Stevie-Ray/referrer-spam-blocker

Here you will find config files in multiple formats that are pre-populated with common spamming domains.  If you just need the Nginx file, it can be found here.

https://github.com/Stevie-Ray/referrer-spam-blocker/blob/master/referral-spam.conf

Happy spam blocking!

Leave a Reply

Your email address will not be published. Required fields are marked *