As it looks like strict DMARC policies are here to stay, OnlineGroups.net has improved its handling of posts from a domains with a strict DMARC policy. This means that as other providers switch to strict policies, we will automatically be able to take responsibility for delivering emails. Here is an explanation of the change we have made, a summary of recent DMARC-related trends and some thoughts about where we are heading with all this.
OnlineGroups.net’s improved handling of DMARC
When Yahoo first enabled a strict DMARC policy, we responded by setting the From header in all email from yahoo.com to a new address from our domain. This ensured that those emails were delivered to group members using email providers that implement DMARC.
It was a deliberately minimal solution as we wanted to see how other email providers and mailing list manager developers responded. Before long, AOL switched to a strict DMARC policy. We quickly added support for emails with aol.com in the From address and then built a more flexible solution which we deployed yesterday.
OnlineGroups.net now checks the DMARC policy of the domain in the From address of emails to online groups, before deciding whether to re-write the From address with an address at our own domain. This is a much more discerning approach that will respond if more providers adopt strict policies and will more accurately respond when they don’t (so far yahoo.co.nz still has a DMARC policy of p=none).
When we do re-write the From address, we use the prefix ‘user-‘, then the user ID of the group member, and then ‘@onlinegroups.net’. Thus if your profile is http://onlinegroups.net/p/1VJQc2umoIC, your email address would become user-1VJQc2umoIC@onlinegroups.net.
The new code is available for GroupServer, will be in the next release of GroupServer, and has been released as a stand-alone library on Pypi. This is a small step forward. We have more improvements in the pipeline.
An update on DMARC trends
Less than two weeks after Yahoo responded to a phishing attack by setting a strict DMARC policy, AOL also suffered a malicious email attack. The attackers succeeded in accessing about 20% of AOL users’ email accounts and obtaining details of their contacts. They proceeded to send phishing emails to those contacts purportedly from the AOL users’ addresses.
Following Yahoo’s lead, AOL switched their DMARC policy to p=reject. This meant that email providers who had implemented DMARC would reject any email with an AOL From address that could not be verified to have actually come from AOL. This will have mitigated the impact of the security breach, and causing new adverse side-effects for some email providers, sites that provide “email this” links and mailing list managers.
So far no other large email providers have taken the opportunity to switch to a strict policy before they have a security breach. In fact one provider, Comcast has announced that they are not planning to adopt a strict DMARC policy for their users’ email, coolly remarking that “it does not yet appear to be typical DMARC usage”.
Meanwhile the Mailing List Manager community appears to be entering the Acceptance stage of the grieving process already.
Threadable were very quick to begin re-writing the From address of emails from providers with a strict DMARC policy.
GNU Mailman has released a new version Mailman 2.1.18 with features that can set the From address to the list address, and change the moderation settings for posts with From addresses at domains with strict DMARC policies.
Google Groups is rewriting the From header (almost certainly where it finds a p=reject policy), as in this example:
First Last <email@example.com>
is rewritten to
'First Last <firstname.lastname@example.org>' via List Title <email@example.com>
Yahoo! Groups itself this far does not seem to be rewriting email addresses where the From is an AOL address.
Taking responsibility for sending the email
One of DMARC’s suggested ways for a mailing list to interoperate with DMARC is to take ownership of the email message. This has sparked a lot of conversation about taking ownership of the email. I do not think ownership is a helpful term in this context. If anyone does have ownership of an email message, it is the author. I prefer to think about taking responsibility for the email message.
By accepting an email as a post to an online group (aka mailing list), the provider agrees to take some responsibility for the email, in particular for getting it to its intended recipients. This is a core function of mailing list managers. The Mailing List Manager holds the list of recipients. That is what the list in their name refers to. This means that participants in a conversation do not each have to maintain a list of recipients as they do when using reply-all.
Like other mailing list managers, OnlineGroups.net and GroupServer have always taken responsibility for delivering a message to its recipients, allowing the recipient to take responsibility for how they receive it. A recipient can select their preference from the following options.
- By email, usually one per post. OnlineGroups.net and GroupServer offer particularly granular choice to the recipient in that they can associate multiple email addresses with their profile, and use a different address to receive posts in each group.
- As a digest, summarising multiple posts. OnlineGroups.net and GroupServer provide a particular kind of digest that lists and links to the latest Topics to receive new posts.
- In web feeds so you can follow a discussion using a news-reader.
- On the web itself, where it is easy to follow a discussion when there are several active topics at one time.
RFC 5322, the specification for email, refers to the responsibility for writing an email.
The “From:” field specifies the author(s) of the message, that is, the mailbox(es) of the person(s) or system(s) responsible for the writing of the message.
OnlineGroups.net and GroupServer respect that responsibility by allowing a group member to use either email or the web to author their email (in the latter case choosing the address to appear in the From header).
Like other mailing list managers we have always changed the message in certain ways. We add a subject line prefix. We add a footer to the message body. We may change the To, Reply-to and other headers. We may detach and upload files to the web interface. Now we are making one new change (arguably in breach of the RFC) in changing the From address to one at our own domain. Perhaps we will make more.
The future options for mailing list managers
There are two reasons to consider taking even more responsibility the sending of group emails. The first is to ensure the delivery messages to recipients. The same changes will offer more privacy to message authors. The impact may be a reduction of the choice that users have in the matter.
If the current trend of requiring greater authentication of email continues, we will inevitably end up implementing DMARC ourselves. To do that we will have to be sure that all email sent from our servers has a From address at our domain. What we hoped would be a privacy feature, where we would optionally offer email obfuscation may become a necessity for getting emails delivered at all.
There has been much discussion about the Reply-to header in mailing list emails. In the reply-to-sender case, it is moot as the DMARC debacle does not affect the Reply-to header. In the reply-to-list case, obfuscating the From address interferes with the ability to directly email the sender. Some mailing list manager providers have got around that by writing a munged version of the original email address to the name portion of the From header. Threadable have resolved the problem by cc:ing the post to the original sender, where Reply-to is the list (I think I would be annoyed at getting two copies of my own posts).
We have chosen a different approach, currently providing no support for identifying the author’s original email address, beyond a new X-gs-formerly-from header. In time we will provide further options. One may be to forward emails to the rewritten address to the original address of the user in question. As the user ID is in the new address, we will be able to support that even if the user changes the addresses on their profile. Another may be to invite the replier to use the Request contact feature on the original author’s profile.
Before long we may have to expand our handling of DMARC to encompass subdomains of organisations with a strict DMARC policy.
For services like ours that offer custom domains to their customers, there will be further complications. We may have to re-write From addresses to the domain of every online groups site with a custom domain. We may have to work with our clients to implement DKIM and SPF on those domains. As Laura of Word to the Wise points out, DMARC is not really about domains, it is about organisations and their identity. And that is the raison d’être of OnlineGroups.net: to provide online groups for organisations. It is why we provide a site for your online groups, rather than putting them on our site. It is why we invite you to use your domain and visual style on your site. It is why we offer you the choice of downloading, installing and even modifying the open source system GroupServer that powers OnlineGroups.net.
I am more certain than ever that the recent DMARC developments are going to help us, and other mailing list managers, to get better.