How We Test VPNs

Too many security products trade on fear, uncertainty, and doubt among customers and the media. At the same time, giving a positive review to a flawed product risks putting people’s privacy and even their safety in danger. This is especially true for virtual private networks, or VPNs. When we test VPNs, we consider their performance and available features with the goal of writing reviews that are factual and useful to our readers.

This is harder than it sounds. If we relied entirely upon objective measurements, it would be trivial for a vendor to game the system by inflating particular stats like server count or number of simultaneous connections. If we relied only on subjective observations, we’d miss the features that make it unique. Combining the two—objective measurement and subjective observations—is messier, but leads to better and more comprehensive analysis.

Our readers may not always agree with our conclusions, but we strive to include enough information in our reviews so that readers can form their own opinions, too. In fact, we encourage them to do so.

VPNs Are For Privacy, Everything Else Is Gravy

To evaluate a product, you must first understand what it’s for. For example: A MacBook Pro is an excellent laptop, but probably a terrible waffle iron. When it comes to VPNs, we consider them a privacy tool first, and evaluate them primarily on those grounds.

Privacy tools are distinct from security tools. While the two concepts overlap, a privacy tool shields your devices from efforts to track and identity them, or proactively removes identifiable information. A security tool identifies, removes, and (ideally) prevents the use of malicious software or hardware that would harm you, your machines, or your files.

It’s important to make this distinction because of misleading advertising from VPN companies. A VPN cannot protect you against every threat. While we consider VPNs useful tools, that utility is limited and to say otherwise puts people at risk. We strongly encourage readers to use standalone antivirus, enable multi-factor authentication wherever available, and use a password manager to create unique and complex passwords for each login they have.

Our Testing Criteria

When we evaluate the privacy-protecting abilities of a VPN, we look at:

• The technology it uses,
• The servers it makes available,
• The presence of privacy-enhancing tools (primarily multi-hop connections and VPN access to Tor), and
• The measures a company takes to ensure that the VPN itself does not become a threat to user privacy.

This last point can include everything from third-party audits to using an anonymizing login system like the ones employed by Editors’ Choice winners IVPN and Mullvad VPN.