How we test

At EmailDeliverabilityReport, we test the deliverability of emails from email marketing providers, amongst other things, and we use coordinators and testers, as well as automated systems and tools that facilitate testing and ensure high data quality.

Our test system

There are 4 important components in our test system:

Coordinators – coordinators regulate which tests need to be done and assign tasks to corresponding testers. The coordinators are also responsible for ensuring that the test results find their way into the central database after validation.
Testers – amongst other things, the testers have numerous email accounts with the major email service providers in order to be able to receive and subsequently evaluate test emails that they receive from the email marketing providers from the coordinators during the course of a test campaign.
ETI – the „Email Testing Interface“ is a key element in the overall test system. Coordinators and testers use the system to communicate with each other and exchange data. After automatic and manual validation of new data, the ETI writes it directly to the central database.
Central database – after the testers deliver the results to the coordinators via the ETI, the data is validated several times and then stored in the central database so that it can be used, amongst other things, in the reports on the website.

Here's how our test system looks simplified:

With our current capacities and resources, more than 100.000 emails can be analyzed in detail every month. In particular, email deliverability is particularly examined, but other tests also take place, such as checking email delivery servers.

By the 20th of each month, all tests must be completed and the test results must be available in the ETI to the coordinators so that new data can be stored in the central database after validation and can be used for statistical purposes at the beginning of the next month.

If serious technical flaws or security vulnerabilities are identified during testing, we will contact the relevant provider directly via email or social media and notify them about the issues.

How we rate email marketing providers

Once an email marketing provider goes through all the monthly test campaigns and the results are in, the provider is given an overall score that ranges from 0 – 100, with 100 being the best score. We refer to this value as the „Provider Score“.

We categorize all providers into 4 score ranges:

90+ Such providers can be absolutely convincing technically, are best suited for mass sending of marketing emails, and often have a good reputation among email service providers.
70 – 89 Such providers perform well in tests, but have minimal issues with either the technology or the reputation of their IP network, and have mild to moderate deliverability issues.
50 – 69 Such providers perform just well enough to send bulk emails, but have significant deficiencies in the readings, often have major issues with email deliverability, IP network reputation, or technology.
0 – 49 Such providers are not suitable for professional email marketing. Comparatively many sent emails end up in the junk folder or do not arrive at their destination. You lose money with every email campaign sent.

Most email marketing providers are quite quick to correct clear deficiencies in particular, so it is very rare for a provider to fall below the 70-point mark - those who are above it are clearly on the right track.

You like what we do?

Anyone who wants to support our work can find all the information they need on the sponsors page.