How We Test Ad Blockers?

Wondering if your ad blocker is really doing its job? With most other apps you may get an instant result to see if they’re working properly, but ad blockers are a bit of a mystery. You could browse for days without a single ad in sight and then – bam! You’re hit with the most annoying looking ad ever, blocking most of your screen. And that’s the part you can see. There is a lot of behind-the-scenes work an ad blocker is supposed to handle to keep you safe from invisible trackers. 

It is really hard to tell if an ad blocker is actually working its magic without using it extensively in a controlled digital environment, simulating different use cases. And you could do it yourself just by following the same process we do. But let’s be honest here, do you really wanna be spending days testing different ad blockers?

That’s where we come in. Here at About Cookies, our team of experts have developed a testing method that puts ad blockers through their paces so you don’t have to. And we’re here to explain our process to you. We follow a five-stage process that helps us assign an objective score based on our results.

Stage 1: Compatibility and Installation

We first check the ad blocker’s compatibility. We figure out how many devices it supports and to what extent. Does it only offer browser extensions, or does it provide system-wide protection? Does it have Android and iOS apps? Is it compatible with Chrome’s Manifest V3 platform? Since most ad blockers have a Chrome extension, that’s where we usually install it first. But if it lacks support on Chrome, we move on to Firefox or see if it offers system-wide ad blocking. The ad blocker gets scored based on how widely compatible it is and how easy and quick the installation process is.

How We Score This Section:

This section carries 10% of the weight of our overall score. 5% for Compatibility and Another 5% for Installation.

Compatibility (out of 10)

  • Chromium Extension: 2 points
  • MV3 Compliance (Chrome): 1 point
  • Firefox Add-On: 2 points
  • Safari Extension: 1 point
  • Windows App (System-wide): 1 point
  • macOS App (System-wide): 1 point
  • Android App: 1 point
  • iOS App: 1 point

Installation (out of 10)

  • Quick Installation: 4 points
  • Beginner-Friendly: 3 points
  • Clear Instructions: 3 points

Stage 2: Ease of Use and Customizability

Once we’ve got it installed, we explore the user interface to see how easy it is to use. Ideally, an ad blocker should work with just one click. Then we dive into the settings to see how much control the user has. Can they change custom filters, allow lists, custom rules, or manually block elements? We understand that ease of use and customizability can sometimes be at odds. A highly customizable ad blocker might be overwhelming for some users, while a simpler one might lack advanced features. We evaluate both aspects to give you a balanced view.

How We Score This Section:

We’ve assigned 10% weight for ease of use and 5% for customizability.

Ease of Use (out of 10)

  • One-Click Use: 4 points
  • User-Friendly Settings: 3 points
  • Automatic Updates: 3 points

Customizability (out of 10)

  • Accessible Filter Lists: 4 points
  • Custom Filters: 4 points
  • Manual Element Blocking: 2 points

Stage 3: Pricing and Customer Support

Next, we evaluate the pricing. Is it completely free, or does it offer a paid tier? What are the available plans? Is there a free-to-use version or just a free trial? Does the free trial require a credit card? How usable is the free version? Does it lack essential features that may render it unusable or give poor performance? After assigning a score for pricing, we move on to customer support. We usually look for a live chat option as that usually leads to fast response. But most ad blockers offer email or phone support, while some have none, just a knowledge base. The response time and quality of the support also affect the score here.

How We Score This Section:

This part of the evaluation weighs 20% of the final score. 15% for pricing, 5% for support.

Pricing

  • Value for Money: 5 points
  • Free Version Usability: 5 points

Customer Support

  • Availability of Support: 4 points
    • Live Chat: 2 points
    • Email: 1 point
    • Phone: 1 point
  • Response Time: 3 points
    • Immediate Response: 3 points
    • Within 24 Hours: 2 points
    • Within 7 Days: 1 point
  • Support Quality: 3 points

Stage 4: Real-World Testing on Websites

Then, we test the ad blocker on specific websites. Starting with YouTube, then moving on to news sites, social media, and e-commerce platforms. This helps us assess how effective the ad blocker is at blocking ads and annoyances. It also lets us evaluate its features and see if it lives up to its claims. Our testing grounds include:

  • YouTube: We check if it blocks video ads, banner ads, and ads on Shorts.
  • News Sites: Platforms like Forbes and The New York Times help us evaluate how well it handles sponsored content and overlay ads.
  • Social Media: We test on sites like Reddit and X (formerly Twitter) to see if it blocks promoted posts and social media widgets.
  • E-commerce Platforms: Sites like Crocs help us check for retargeting ads and pop-ups.

How We Score This Section:

The score for YouTube ad blocking carried 10% of the final score as it is one of the key reasons people opt for an ad blocker. The performance on the other websites carry another 10%.

YouTube 

  • Blocks Video Ads: 4 points
  • Blocks Banner Ads: 4 points
  • Blocks Shorts Ads: 2 points

Other Websites

Here each of the websites are tested differently as they have different types of ads.  Each of the sites get 1 point for blocking all ads, half a point for partial blocking, and 0 for not blocking anything. Then we total the points and scale them to a score out of 10.

Stage 5: Independent Benchmark Tests

To add an extra layer of credibility, we use third-party tests that serve as industry benchmarks. These tests help us measure how well the ad blocker performs against a standard set of criteria. This section carries 15% of the weight.

Our Go-To Benchmark Tests:

  • AdBlock Tester: This site checks how many ads and scripts each ad blocker can block. The score out of 100 is scaled to 10.
  • D3ward Ad Block Test: It tests ads from different providers to see if the ad blocker can block them all. The score here is also scaled to 10.
  • Can You Block It: This website runs various ads and annoyances to see if the ad blocker can handle them.
    • Simple Test: 3 points
    • eXtreme Test: 3 points
    • Advanced Test: 4 points
  • Cover Your Tracks: This test checks if the ad blocker protects you from trackers and fingerprinting.
    • Blocks Tracking Ads: 4 points
    • Blocks Invisible Trackers: 4 points
    • Randomizes Fingerprints: 2 points

All the points from the benchmarks are summed up and calculated to an average score out of 10.

Putting It All Together

After completing all five stages, we compile the scores from each category, each weighted based on its importance. Here’s how the overall scoring breaks down:

CategoryScoreWeightWeighted Avg
Features9.50.21.9
YouTube100.11
Websites7.90.10.79
Independent Tests7.20.151.08
Pricing100.151.5
Compatibility60.050.3
Installation100.050.5
Ease of Use100.11
Customizability100.050.5
Support100.050.5
Bonus0.5N/A0
Final Score:9.6

The numbers on the Score column are added as examples to demonstrate the process. We also give an explanation for the bonus point. An ad blocker can get up to 0.5 in bonus points that gets added to the final score.

Wrapping Up

We take the guesswork out of our testing process and try to keep it as objective as possible. By meticulously testing and scoring each one, we aim to provide you with clear, actionable insights. Whatever features you prioritise in an ad blocker, our comprehensive evaluation should have that covered.