A Plan for Email over IPv6, part 1 – Introduction, and How Filters Work in IPv4
Last week, myself and a colleague from work did a presentation at the Virus Bulletin conference entitled “A Plan for Email Over IPv6.” I have written about this previously on this blog, but this paper contains updates to my previous plan as well as goes into further detail beyond what is in my IETF draft.
Our presentation went well, much better than it did at the IETF this past summer in Vancouver, and the feedback I got about the idea is much better than I ever received on discussion lists. So without further ado, here it is.
A Plan For Email Over IPv6
“ There’s a storm coming, Mr. Wayne.”
- Anne Hathaway as Catwoman in The Dark Knight Rises
Introduction
As the number of available IP address space in IPv4 is depleting, and the amount of Internet-connected devices continues to increase, the world is moving to IPv6. Slowly but surely, it is coming.
But not for email.
Amongst email receivers, there is no agreement on how to perform IPv6 over email in the short term, although there is agreement that eventually it will have to be figured out.
The reason for the lack of consensus of transmitting email over IPv6 is spam filtering:
Some email experts believe that spam filtering will be done the same way in IPv6 as we do it in IPv4: with the use of IP address [1] blocklists[2].
Others believe that IP blocklists will be ineffective in IPv6.
There are some that believe email will never move to IPv6
Still others that think that all email will eventually be sent over IPv6 but not anytime soon. In the meantime, just use IPv4 to send mail.
With so many opinions on the topic, why is email over IPv6 such a problem, and why are email receivers so reluctant to do it?
Background
Before we get started, let’s define what we mean by email over IPv6. It does not mean transmitting email to the user’s mail server over IPv4, and the user then accesses their mail server over IPv6. No, what we mean is that email travels over the public Internet over IPv6. How the user connects to their mail server is irrelevant to the discussion.
Not this:
But this:
The biggest reason why no email providers are eager to transmit email over IPv6 is because there is currently no way to deal with the problem of abuse. Today, spammers make extensive use of botnets. Each day, they compromise new machines and start using them to spew out spam. Each of these bots use different IP addresses, and their IP addresses change all of the time. If you had 10,000 IP addresses today that were sending out spam, then tomorrow there would be 10,000 again but at least 9700 of them would be different IP addresses than were used today [3].
The reason that there is so much rotation in IP addresses is because modern spam filters make use of IP blocklists. When a blocklist service detects that an IP is sending spam, it adds it to the blocklist and rejects all mail from it. There are exceptions to this listing process such as a legitimate IP that sends a majority of good mail (such as a Hotmail or Gmail IP address), but in general, mail servers reject all mail from blocklisted IPs. The reason they do this is the following:
Resource Optimization - 70% of all email flowing across the Internet (not including internal mail within an organization) is spam. If a sending IP is on a blocklist, a mail server can reject it in the SMTP transaction and save on all of the processing costs associated with accepting the message and filtering it in the content filter.
Most mail servers today would topple over and crash because they could not keep up with the load if they had to handle all of the mail coming from blocklisted IPs since it would increase the number of total spam messages by a factor of 10.
Storage - Spam filters that mark messages as spam in the content filter must store it for the end user. This mail is either stored in a user’s junk mail folder or in a cloud-based spam quarantine. By rejecting email up front, mail servers can reduce the amount of hardware required to store email. This reduces the costs associated with filtering mail.
Spam Effectiveness - Spam filters achieve better antispam metrics, and better user experiences, by using IP blocklists. Modern content filters are good, but rejecting 100% of mail from a spamming IP address means that there is no possibility of a false negative from that IP address.
By contrast, if a spam filter does not use an IP blocklist, the content filter has to learn to recognize the spam coming from that IP address, update the filter and then replicate out the changes. This method is slower than pulling down a blocklist and then using it as the first line of defense. Without an IP blocklist, a spam filter will catch between 80% and 99% of the mail coming from a blocklisted IP. While many spam filters get close to that 99% range, it’s still not 100%.
Reduced Risk - Fewer spams arriving in users’ inboxes reduces risks for organizations and for individual users. If spam and other malicious messages, such as messages with links to malware, are rejected up front, a user cannot later go digging through their spam quarantine or junk mail folders, retrieve the message and click on the malicious link. Rejecting this mail precludes this possibility from ever occurring since the user will never see it.
All of these reasons make the use of IP blocklists indispensable.
Posts in this series:
- A Plan for Email over IPv6, part 1 – Introduction, and How Filters Work in IPv6
- A Plan for Email over IPv6, part 2 - Why we use IP blocklists in IPv4 and why we can't in IPv6
- A Plan for Email over IPv6, part 3 - A solution
- A Plan for Email over IPv6, part 4 - Population of the whitelists
- A Plan for Email over IPv6, part 5 – Removals, key differences and standards
[1] In this paper, whenever I use the term “IP,” I mean “IP address.”
[2] The terms “blocklist” and “blacklist” are synonymous.
[3] I have confirmed this by digging through our own IP statistics and checking the uniqueness of abusive IP addresses.