Antispam counter measures explained Part 1: How the Sender Policy Framework really works
Antispam counter measures explained Part 2: Advanced SPF records, Best practice and biggest mistakes
Antispam counter measures explained Part 3: How DKIM really works and best practice

 

In this first part of “Antispam counter measures explained ” I will start with a detailed explanation of how the Sender Policy Framework actually works, show you best practices and common mistakes. During my employment as an Exchange consultant I have seen many poorly configured implementations of the Sender Policy Framework. Even if you search the Internet there are a lot of sites that provide misleading explanations. Either if you are an inexperienced administrator or a SPF veteran, this blogpost might surprise you on how SPF actually works. If you think you know SPF let’s get some statements and see if I can surprise you and make things interesting:

• Do you know what the difference is between “-all” , “?all” and “~all”  in an included statement? ………. There is no difference
• SPF2.0 is not Sender Policy Framework
• Do you know that you can use “+all” to authorize your own mail servers ?
• Do you know you can utilize a Real-time Block List within SPF policy ?
• Why should you always prefix a include with a “?” ?

If this got you interested, take a seat and relax as were going to take you on this journey to become a skilled SPF ninja. If you are familiar with the basics continue reading here.

First things first. SPF framework has been around since the draft in 2003 and reached it current RFC ( RFC 4408 ) in 2006. This means that the SPF standard has not changed since 2006. SPF should not be confused with “spf2.0” which is a Microsoft Protocol for Sender ID filtering. The Sender ID filtering never had a major support base and even Microsoft has moved away from it and adopted SPF. The main purpose of SPF is to protect your email domain from sender address forgery aka spoofing.

At the moment we can derive the following statistics from http://spf-all.com.

In the left Pie chart you can see that there are only 27.3% domains protected with SPF, that leaves 72.7 % ( 64 million ) potential domains for spoofing. If we take a look at the right pie chart statistics we can see that there are more than a 1/4 million domains configured with malformed SPF records producing Permanent errors. SO this leave me to start the actual How does it work and how do I protect my domain correctly from being spoofed with SPF.

What solution offers the Sender Policy Framework

The Sender Policy Framework is an open standard specifying a technical method to prevent sender address forgery. The current version of SPF protects the envelope sender address (RFC 5321.MailFrom), which is used for the delivery of messages ( Not to confuse with the header sender address or Display sender address RFC 5322.from ). SPF allows the owner of a domain to specify their mail sending policy, e.g. which mail servers they use to send mail from their domain. The technology requires two sides to play together: (1) the domain owner publishes this information in an SPF record in the domain’s DNS zone, and when someone else’s mail server receives a message claiming to come from that domain, then (2) the receiving server can check whether the message complies with the domain’s stated policy. If, e.g., the message comes from an unknown server, it can be considered a fake.

DNS sample record:

“v=spf1 mx a:tech-savvy.nl include:spf.protection.outlook.com -all”

I want to implement Sender Policy Framework but where should I start

If you want to start implementing SPF start by inventorying the following list:

1. Create a list of your domains
2. Create a list of your mail servers
3. Create a list of registered A records for those domains
4. Inventory the 3rd party`s that send mail on your behalf

Next you need to make a decision of what would be the best way to group your outbound mail gateways and or 3th party`s. Most of the time it is the best way to take their public IP addresses, MX registrations or A record registrations. You can now create a SPF record using any record creation wizards found on the internet. ( example http://wizard.easyspf.com/ has a nice one ). Next you should take a test drive before you publish your record ( example http://www.kitterman.com/spf/validate.html has a nice preproduction test tool).

If all of your servers validate and some random chosen internet servers fail it`s time to publish your SPF record into you DNS zone. When it is deployed you can test it with a test tool like http://tools.bevhost.com/spf/.

How does SPF Work

Let`s dive a level deeper in to the matter and see what`s really going on under the hood.

The SPF flow logic:

For each piece of mail received by a MTA with support for SPF validation turned on the following flow logic will be followed:

1. Look at the envelop From field that is offered by the sending MTA (RFC 5321.MailFrom) and extract the domain local part.
2. Query the domain in the domain local part for any type of SPF record ( TXT only ,  SPF (type 99) records are deprecated sinds 2014 by RFC 7208 and should not longer be used)
3. Evaluate the policy set by the sender:
   • Evaluation the policy in sequence.
   • Stop processing at the first mechanism that produces a match.
   • Invoke actions set by any modifier .
   • Call the qualifier of the match and use that as the verdict.
4. Process the email based on the qualifier outcome of the policy.

The domain owner is advising you to follow the qualifier outcome as he published the SPF policy , but as a receiving party you are free to configure what you will do with the email verdict. You can choose to honor it,  or to deviate from the policy.

SPF Mechanism:

A SPF Mechanism is a designation where the Sender`s details are being compared to.  There are two types of mechanisms. Basic and Designated. Designated mechanisms support  the “:” and “/” characters after the name to further specify a subset of data. For example a subnet or a domain name. ( Example V=spf1 A:tech-savvy.nl/24 -all )

Basic mechanisms contribute to the language framework. They do not specify a particular type of authorization scheme. The basic mechanisms are:

• all
• include

Designated sender mechanisms are used to designate a set of <ip> addresses or text strings as being permitted or not permitted to use the domain for sending mail. The Designated mechanisms are:

• a
• mx
• ptr
• ip4
• ip6
• exists

The following conventions apply to all mechanisms that perform a comparison between  sender IP and an IP address at any point:

• If no CIDR-length is given in the directive, then the IP addresses are compared for equality. (Here, CIDR is Classless Inter-Domain Routing.)
• If a CIDR-length is specified, the IP address is compared against the CIDR subnet.
• When a mechanism fetches host addresses to compare with the sender IP, A or AAAA records are fetched based on IPv4 or IPV6. Even if the SMTP connection is via IPv6, an IPv4-mapped IPv6 IP address (see RFC 3513, Section 2.5.5) MUST still be considered an IPv4 address.

Several mechanisms rely on information fetched from DNS. For these DNS queries if the DNS server returns an error (Other than “domain does not exist”) or the query times out, the mechanism throws the exception “TempError”.

SPF Qualifiers:

A SPF qualifier is the verdict of a mechanism that is being invoked If the mechanism matches. (Note: The behavior of qualifiers is different in a include statement. This will be explained in PART 2 )

The possible qualifiers, and the results they return are displayed in the table below. A mechanism can have only 1 qualifier.

“+” Pass                (The sender is authorized by the domain owner)
“-”  Fail                 (The sender is NOT authorized by the domain owner)
“~” SoftFail          (The domain owner believes the host is NOT authorized but is not willing to make that strong of a statement)
“?”  Neutral          (The domain owner has explicitly stated that he can NOT or does NOT want to assert whether or not the IP address is authorized)

SPF mechanism Evaluation:

Each mechanism is considered in turn from left to right. When a mechanism is evaluated, one of three things can happen: it can match, not match, or throw an exception. If none of the mechanisms match and there is no “redirect” modifier, then the evaluation returns a result of “Neutral”, just as if “?all” were specified as the last directive. If there is a “redirect” modifier, evaluation will proceed and the result of this new evaluation is then considered the result of the current evaluation. The only exception is that if no SPF record is found, or if the target-name is malformed, the result is a “PermError” rather than “None”.

Note that records SHOULD always use either a “redirect” modifier or an “all” mechanism to explicitly terminate processing !!!.

When a mechanism is evaluated as a match, processing ends and the qualifier value is returned as the result of that record. There is no further processing of the SPF record.

The following example shows the Evaluation flow:If a mechanism has no qualifier the default qualifier “+” will be used. So a mechanism without a qualifier should be read as “+mechanism”. When a mechanism matches and the qualifier is “-“, then a “Fail” result is returned and if a explanation modifier is present it will be read, evaluated and send with the NDR message.

This is the end of Part 1 which covers the basics of SPF. In part 2 we will take a look at advanced  SPF records and multiple mechanisms working together to create a hardened more secure policy. We will provide best practice and common mistakes.

Stay tuned,

Martijn van Geffen