Sender Policy Framework
555928
224921876
2008-07-11T00:40:12Z
82.71.32.108
/* Deployment */
<table align="right" border="0" cellspacing="5"><tr><td>__TOC__</td></tr></table><!-- float: right working for legacy browsers -->
In [[computing]], '''Sender Policy Framework''' ('''SPF''') allows the owner of a domain name to publish information about its use in an email return address during an [[SMTP|Simple Mail Transfer Protocol (SMTP)]] session. SPF allows software to identify and reject messages that are not authorized to have the domain named used in the SMTP MAIL FROM ([[Return-Path]]) command. This is intended as a means of detecting some forms of [[e-mail spam]] or other forgeries. SPF is defined in RFC 4408.
== Principles of operation ==
Normal SMTP allows any computer to send an e-mail claiming to be from anyone. Thus, it's easy for
[[e-mail spam|spammers]] to send [[electronic mail|e-mail]] from forged [[e-mail address|addresses]]. This makes it difficult to trace back to where the spam truly comes from, and easy for spammers to hide their true identity in order to avoid responsibility. Many believe that the ability for anyone to forge ''sender addresses'' (also known as Return-Paths) is a security flaw in modern [[Simple Mail Transfer Protocol|SMTP]], caused by an undesirable side-effect of the deprecation of ''source routes''.
:''Further information: [[Sender Rewriting Scheme]] (SRS)''
SPF allows the owner of an Internet domain to use special format of [[Domain Name System|DNS]] [[Domain Name System#Types of DNS records|TXT records]] to specify which machines are authorized to transmit e-mail for that domain. For example, the owner of the example.org domain can designate which machines are authorized to send e-mail whose sender e-mail address ends with "@example.org".
Receivers checking SPF can reject messages from unauthorized machines ''before receiving the body of the message''. Thus, principles of operations are quite similar to those of [[DNSBL]], except that SPF leverages the authority delegation scheme of the real [[Domain Name System]].
The sender address is transmitted at the beginning of the [[SMTP]] dialog. If the [[server (computing)|server]] rejects the sender, the unauthorized [[Client (computing)|client]] should send a [[Bounce message]] to that address. If the server accepts the sender, and subsequently also accepts the recipient(s) and the body of the message, it should insert a Return-Path header in the message's body in order to save the sender address. While the address in the Return-Path often matches other originator addresses in the mail header like "From:" or "Sender:" this is not necessarily the case, and SPF does not prevent forgeries of these other addresses.
Spammers can send e-mail with an SPF PASS result if they have an account in a domain with a sender policy, or abuse a compromised system in this domain. However, doing so makes the spammer easier to trace and prosecute.
The main benefit of SPF is to people whose e-mail addresses are forged in the Return-Paths. They receive a large mass of unsolicited error messages and other auto-replies, making it difficult to use e-mail normally. If such people use SPF to specify their legitimate sending IPs with a FAIL result for all other IPs, then receivers checking SPF can reject forgeries, reducing the amount of [[Backscatter (e-mail)|back-scatter]].
SPF has potential advantages beyond helping identify unwanted e-mail. In particular, if a sender provides SPF information, then receivers can use SPF PASS results in combination with a white list to identify known reliable senders. Scenarios like compromised systems and shared sending mailers limit this use.
=== FAIL and forwarding ===
SPF does not allow plain message forwarding. When a domain publishes an SPF FAIL policy, then legitimate mails sent to receivers forwarding their mail to third parties can be rejected and bounced if
# the forwarder doesn't rewrite the Return-Path, unlike mailing lists,
# the next hop doesn't white list the forwarder, and
# this hop checks SPF.
This is a necessary and obvious feature of SPF – checks '''behind''' the "border" [[Mail transfer agent|MTA]] ([[MX record|MX]]) of the receiver can't work directly.
Publishers of SPF FAIL policies must accept this potential problem. They should test, e.g. with a SOFTFAIL policy, until they are satisfied with the results. See below for a list of alternatives to plain message forwarding.
=== HELO tests ===
For an empty Return-Path as used in [[Bounce message|error messages]] and other auto-replies, an SPF check of the HELO-identity is mandatory. Actually it checks an artificial <tt> postmaster@mail.example.org</tt> address for a HELO or EHLO <tt>mail.example.org</tt>.
With a bogus HELO identity the result NONE would not help, but for valid host names SPF also protects the HELO identity. This SPF feature was always supported as an option for receivers, and later SPF drafts including the final specification recommend to check the HELO always.
This allows to white list sending mailers based on a HELO PASS, or to reject all mails after a HELO FAIL. It can also be used in [[reputation system]]s (any white or black list is a simple case of a reputation system).
== Implementation ==
Compliance with SPF consists of three loosely related tasks:
; Publish a policy : Domains identify the machines authorized to send e-mail on their behalf. Domains do this by adding additional records to their existing DNS information: every [[Domain name]] that has an [[Domain name system#Types of DNS records|A]] or [[MX record]] deserves a TXT record specifying the policy if it is used either in an email address or as HELO/EHLO argument.
; Check and use SPF information : Receivers use ordinary DNS queries, which are typically cached to enhance performance. Receivers then interpret the SPF information as specified and act upon the result.
; Revise mail forwarding
: Plain mail forwarding is not allowed by SPF. The alternatives are
:* '''remailing''', i.e. replacing the original sender with one belonging to the local domain,
:* '''refusing''', i.e. answering <code>551 User not local; please try <user@example.com></code>,
:* '''[[whitelisting]]''' on the target server, so that it won't refuse a forwarded message, and
:* '''[[Sender Rewriting Scheme]]''', a more complicated mechanism that handles forwarding [[Backscatter (e-mail)|backscatter]] back.
Thus, the key issue in SPF is the specification
for the new DNS information that domains set and
receivers use. The records are laid out like this
(in typical DNS-syntax):
example.org. IN TXT "v=spf1 a mx -all"
"v=" defines the version of SPF used. The
following words provide ''mechanisms'' to use to
determine if a domain is eligible to send mail.
The "a" and "mx" specify the systems permitted
to send messages for the given domain. The "-all"
at the end specifies that, if the previous
''mechanisms'' did not match, the message should
be rejected.
=== Mechanisms ===
Eight ''mechanisms'' are defined:
{|
| valign="top" | ALL || Matches always, used for a default result like <tt>-all</tt> for all IPs not matched by prior mechanisms.
|-
| valign="top" | A || If the domain name has an A (or AAAA for [[IPv6]] ) record corresponding to the sender's address, it will match. (That is, the mail comes directly from the domain name.)
|-
| valign="top" | IP4 || If the sender is in a given [[IPv4]] range, match.
|-
| valign="top" | IP6 || If the sender is in a given [[IPv6]] range, match.
|-
| valign="top" | MX || If the domain name has an [[MX record]] resolving to the sender's address, it will match. (That is, the mail comes from one of the domain's mail servers.)
|-
| valign="top" | PTR || If the [[Forward Confirmed reverse DNS]] domain of the sending IP ending in the domain name.
|-
| valign="top" | EXISTS || If the given domain resolves, match (no matter the address it resolves to). This is rarely used. Along with the SPF macro language it offers more complex matches like [[DNSBL]]-queries.
|-
| valign="top" | INCLUDE || If the ''included'' (a misnomer) policy passes the test this mechanism matches. This is typically used to ''include'' policies of more than one [[Internet service provider|ISP]].
|}
=== Qualifiers ===
Each ''mechanism'' can be combined with one of four ''qualifiers'':
* '''<tt>+</tt>''' for a PASS result, this can be omitted, <tt>+mx</tt> is the same as <tt>mx</tt>.
* '''<tt>?</tt>''' for a NEUTRAL result interpreted like NONE (no policy).
* '''<tt>~</tt>''' for SOFTFAIL, a debugging aid between NEUTRAL and FAIL.
* '''<tt>-</tt>''' for FAIL, the mail should be rejected (see below).
=== Modifiers ===
The ''modifiers'' allow for future extensions to
the framework. To date only the two ''modifiers''
defined in the RFC 4408 have been widely deployed:
<tt>exp=some.example.com</tt> gives the name of a
domain with a [[Domain Name System|DNS]] TXT record
(interpreted using SPF's macro language) to
get an explanation for FAIL results – typically a
[[Uniform Resource Locator|URL]] which is added to the SMTP
error code. This baroque feature is rarely used.
<tt>redirect=some.example.com</tt> can be used
instead of the ALL-''mechanism'' to link to the
policy record of another domain. This ''modifier''
is easier to understand than the somewhat similar
INCLUDE-''mechanism''.
=== Error handling ===
As soon as SPF implementations detect syntax
errors in a sender policy they '''must''' abort
the evaluation with result PERMERROR. Skipping
erroneous ''mechanisms'' cannot work as expected,
therefore <tt>include:bad.example</tt> and
<tt>redirect=bad.example</tt> also cause a
PERMERROR.
Another safety guard is the maximum of '''ten'''
''mechanisms'' querying DNS, i.e. any ''mechanism''
except from IP4, IP6, and ALL. Implementations can
abort the evaluation with result SOFTERROR when it
takes too long or a DNS query times out, but they
'''must''' return PERMERROR if the policy directly
or indirectly needs more than '''ten''' queries
for ''mechanisms'', any <tt>redirect=</tt> also
counts towards this ''processing limit''.
A typical SPF HELO policy <tt>v=spf1 a -all</tt>
needs three DNS queries: (1) TXT, (2) SPF, and (3)
A or AAAA. This last query counts as the first
''mechanism'' towards the limit (10), in this
example it's also the last, because ALL needs no
DNS lookup.
== Caveats ==
SPF avoids abuses of domain names; it does not validate that a given e-mail actually comes from
the claimed user. Users within a given domain can forge each others' address, if permitted by the signature process. In addition, domains that share a mail sender (e.g. with virtual hosting) can forge each others' domain even if dire policies are applied.
=== SPF FAIL rejection ===
SPF FAIL policies can be an effective but dangerous tool.
Some publishers of SPF policies try to avoid the dangers
by using SOFTFAIL (designed for limited testing periods)
instead of FAIL.
But SOFTFAIL can be even more dangerous than FAIL with
receivers rejecting FAIL and accepting SOFTFAIL tagged
as ''potential junk''.
A reject in a forwarding scenario is a clean decision. The forwarder will send an error message to the address
in the Return-Path. Typically the
[[Bounce message|error message]] (bounce)
contains an explanation of the SMTP error and the failing
(forwarded to) address. The original sender can then
send his mail again directly to this address bypassing
the forwarder, a crude emulation of the normal SMTP
error code <tt>551 user not local</tt>.
However an accepted SOFTFAIL tagged as ''potential junk''
could be deleted by the final recipient. This is a user
who has arranged his forwarding in a way that cannot
work with SPF; this user could also be careless with
checking his ''potential junk'' and simply delete it.
The same line of argument also suggests that receivers
should take the SPF recommendation to reject SPF FAIL
seriously where possible. Accepting SPF FAIL results
as ''potential junk'' can be more dangerous than simply
rejecting FAILed mails.
While senders with an SPF FAIL can be expected to know
what it means, the same is obviously not the case for a
receiver arranging his forwarding in a way that cannot
work with SPF.
=== Checkpoints ===
Checking SPF behind the "border" [[Mail transfer agent|MTA]] ([[MX record|MX]]) is not impossible; the relevant info is usually noted in a <tt>Received</tt> timestamp line added by one of the MXs of the receiver. But at this time it is too late to reject SPF FAIL; the checking entity can essentially only delete FAILing mail.
Experts should be able to get this right, but it is no ''plug and play'' situation, therefore the SPF specification recommends to check SPF only at the "border" (MX) in the SMTP session, not later. Later SPF checks can also have unexpected results if the publishers of sender policies do not plan modifications of their policy carefully with regard to [[Domain Name System|DNS]] cache expiration.
===DoS attack===
An Internet draft<ref>[http://tools.ietf.org/html/draft-otis-spf-dos-exploit SPF DoS Exploitation] (expired Internet draft)</ref> discussed concerns related to the scale of an SPF answer leading to network exploits as a means to corrupt the DNS. This issue is also covered in the security considerations of the SPF RFC. The SPF project did a detailed analysis <ref>[http://www.openspf.org/draft-otis-spf-dos-exploit_Analysis OpenSPF analysis of the DoS attack draft]</ref> of this draft and claimed that SPF does not pose any unique threat of DNS DoS.
== History ==
The SPF concept was presented at [[YAPC]] and
[[O'Reilly Open Source Convention|OSCON]] (O'Reilly Open Source Convention) in
2003, in a short paper titled
"Repudiating Mail-From" written by [[Paul Vixie]]
in [[2002]]. Other predecessors were
"Reverse [[MX record|MX]]" by Hadmut Danisch,
and "Designated Mailer Protocol" by Gordon Fecyk.
In June 2003, [[Meng Weng Wong]] merged the RMX
and DMP specifications<ref>The [[IETF]] proposal for RMX is [http://www.tools.ietf.org/html/draft-danisch-dns-rr-smtp-04 draft-danisch-dns-rr-smtp-04]. For a comparison among RMX, DMP and SPF, see [http://old.openspf.org/dmpvsrmx.html RMX and DMP compared] on the historical openspf site.</ref> and solicited suggestions
from other programmers. Over the next six months,
a large number of changes were made and a large
community had started working on SPF.
Originally SPF stood for '''Sender Permitted From'''
and was sometimes also called SMTP+SPF, but it was
changed to Sender Policy Framework in February 2004.
In early 2004, the [[IETF]] created the [[MARID]]
working group and tried to use SPF and Microsoft's
CallerID proposal as the basis for what is now
known as [[Sender ID]].
After the collapse of [[MARID]] the SPF community
returned to the original "classic" version of SPF.
In July 2005 this version of the specification was
approved by the
[[Internet Engineering Steering Group|IESG]] as an
IETF ''experiment'', inviting the community to observe SPF during the two years following publication.
On [[April 28]], [[2006]], the SPF [[Request for Comment|RFC]] was published as
experimental RFC 4408.
=== Controversy ===
In 2004, [[Steven M. Bellovin]] wrote a long e-mail that discusses his concerns with SPF.<ref>[http://www.interesting-people.org/archives/interesting-people/200401/msg00037.html Steve Bellovin expresses doubts] (Jan 2004)</ref> Some of these include:
* SPF uses TXT records in DNS, which are supposed to be free-form text with no semantics attached. SPF proponents readily acknowledge that it would be better to have records specifically designated for SPF, but this choice was made to enable rapid implementation of SPF. In July 2005, [[Internet Assigned Numbers Authority|IANA]] assigned the Resource Record type 99 to SPF. SPF publishers may publish both record types and SPF checkers may check for either types. It will likely take many years before all DNS software fully supports this new record.
* As of the time he wrote his message, there was no consensus that this is the right way to go. Some major e-mail service providers have not bought into this scheme. Unless and until they do, it doesn't help much, either for their customers (who make up a substantial proportion of the user population) or for everyone else (since their addresses could be forged). It's worth noting that since this concern was raised, Google's GMail and AOL, among others, have embraced SPF.<ref>{{cite web |url=http://postmaster.aol.com/spf/ |title=SPF Information |publisher=AOL Postmaster |accessdate=2007-10-04}}</ref>
* Bellovin's strongest concerns involve the underlying assumptions of SPF (SPF's "semantic model"). When using SPF, the SPF DNS records determine how a sender is allowed to send. That means that the owner of the domain will control how senders are allowed to send. People who use "portable" e-mail addresses (such as e-mail addresses created by professional organizations) will be required to use the domain owner's SMTP sender, which may not currently even exist. Organizations providing these "portable" addresses could, however, create their own '''[[Mail submission agent|Mail Submission Agents]] (MSAs)''' (RFC 4409) or offer [[Virtual Private Network|VPNs]]. Besides SPF only ties the [[Simple Mail Transfer Protocol|SMTP]] Return-Path to permitted MSAs; users are still free to use their RFC 2822 addresses elsewhere.
=== Deployment ===
{{weasel}}
Despite its limitations, many people have decided that the pros of SPF outweigh its cons and have begun implementing SPF. Anti-spam software such as [[SpamAssassin]] version 3.0.0 and [[Anti-Spam SMTP Proxy|ASSP]] implement SPF. Many [[mail transfer agent]]s (MTAs) support SPF directly such as [[Courier Mail Server|Courier]], CommuniGate Pro, [[Wildcat! BBS|Wildcat]], and [[Microsoft Exchange]], or have patches/plug-ins available that support SPF, including [[Postfix (software)|Postfix]], [[Sendmail]], [[Exim]], and [[Qmail]]. Many prominent domains decided to post SPF data for their domains as of mid-2004, including [[Amazon.com|Amazon]], [[AOL]], [[EBay]], [[Google]], [[w:de:United Internet|GMX]], [[Hotmail]], [[Microsoft]], and [[W3C]].
In a survey published [[as of 2007|2007]], 5% of the <tt>.com</tt> and <tt>.net</tt> domains had some kind of SPF policy – this might include overall useless policies like <tt>v=spf1 ?all</tt>.<ref>{{cite web |url=http://www.onlamp.com/pub/a/onlamp/2007/01/11/dns-extensions.html |title=Handicapping New DNS Extensions and Applications |first=Cricket |last=Liu |publisher=ONLamp |year=2007 |month=January |accessdate=2007-10-04}}</ref> In April 2007, BITS, a division of the Financial Services Roundtable, published e-mail security recommendations for its members including SPF deployment.<ref>{{cite web |url=http://bitsinfo.org/downloads/Publications%20Page/BITSSecureEmailFINALAPRIL1507.pdf |title=BITS Email Security Toolkit |publisher=BITS | year=2007 |month=April |accessdate=2008-06-13 |format=PDF }}</ref>
In 2008 the [[MAAWG]] (Messaging Anti-Abuse Working Group) published a paper about email-authentication covering SPF, [[Sender ID]], and [[DomainKeys Identified Mail|DKIM]].<ref>{{cite web |url=http://www.maawg.org/about/publishedDocuments/MAAWG_Email_Authentication_Paper.pdf |title=Trust in Email Begins with Authentication |accessdate=2008-06-13 |publisher=[[MAAWG]] |first=Dave |last=Crocker |year=2008 |month=March |format=PDF }}</ref>. In their ''"Sender Best Communication Practices"'' the MAAWG stated: ''"At the very least, senders should incorporate SPF records for their mailing domains"''.<ref>{{cite web |url=http://www.maawg.org/about/MAAWG_Sender_BCP/MAAWG_Senders_BCP_Ver2.pdf |title=MAAWG Sender Best Communications Practices - Version 2.0|accessdate=2008-06-13 |publisher=[[MAAWG]] |date=2008-06-02 |format=PDF }}</ref>
== See also ==
* [[:Category:E-mail authentication|E-mail authentication]]
* [[Sender Rewriting Scheme]]
* [[Author Domain Signing Practises]]
==References==
{{Reflist}}
==External links==
* [http://www.openspf.org SPF homepage] and [http://dir.gmane.org/index.php?prefix=gmane.mail.spam.spf Mailing list archives]
* [http://www.emailbattles.com/archive/battles/spam_aabeeighag_ef/ ''emailbattles'': SPF Claws Sender-ID] (Aug 2005)
* [http://e-com.ic.gc.ca/epic/internet/inecic-ceac.nsf/en/gv00329e.html Canadian recommendation for ISPs] (May 2005)
* [http://www.cbronline.com/article_news.asp?guid=44D2955C-3C04-4BA1-AC45-AF8277B8B455 Interview with co-author W. Schlitt] (Mar 2005)
* [http://david.woodhou.se/why-not-spf.html David Woodhouse discusses flaws in SPF] (Jan 2005)
* [http://www.spamcop.net/fom-serve/cache/329.html SpamCop FAQ entry about bogus bounces] discusses also SPF (2005)
* [http://www.openspf.org/whitepaper.pdf M. Wong's Deployment White Paper] (PDF, Nov 2004)
* [http://www.theregister.co.uk/2004/09/03/email_authentication_spam/ The Register: Spammers embrace email authentication] (September 2004)
* [http://www.circleid.com/posts/an_interview_with_the_lead_developer_of_spf_part_i/ CircleID interview with co-author M. Wong] (Jun 2004)
* [http://bradknowles.typepad.com./considered_harmful/2004/05/spf.html Brad Knowles considers SPF as harmful] (May 2004)
* {{cite web|url=http://homepages.tesco.net./~J.deBoynePollard/FGA/smtp-spf-is-harmful.html|title=SPF is harmful. Adopt it.||author=Jonathan de Boyne Pollard|}} (2004)
* [http://www.openspf.org/mailflows.pdf Diagram of e-mail flow and SPF's impact] (PDF, [http://www.openspf.org/mailflows-l.png PNG] )
* [http://www.openspf.org/for-mit-spam-conference.html MIT Spam Conference Handout on SPF] (Jan 2004)
[[Category:E-mail authentication]]
[[Category:Internet protocols]]
[[Category:Spam filtering]]
[[de:Sender Policy Framework]]
[[es:Sender Policy Framework]]
[[fr:Sender Policy Framework]]
[[it:Sender Policy Framework]]
[[nl:Sender Policy Framework]]
[[ja:Sender Policy Framework]]
[[pl:Sender Policy Framework]]
[[pt:Sender Policy Framework]]
[[ru:Sender Policy Framework]]
[[sv:Sender Policy Framework]]