Quantcast
Channel: Terry Zink: Security Talk
Viewing all 243 articles
Browse latest View live

A Powershell script to help you validate your DKIM config in Office 365

$
0
0

One of our support engineers (not me, so let’s give credit where credit is due) wrote a script to help you, as a customer of Office 365, validate for DKIM configuration once you have enabled it in the Admin Portal. We’ve added a few more checks to make it more clear, but you can also use this.

To verify your DKIM configuration:

1. Copy/paste the below script into a file, Validate-DkimConfig.ps1

2. Connect to Exchange Online using Powershell, making sure that the directory you are in is the same as where you saved the script above.

3. Type the following command in Powershell:

. .\Validate-DkimConfig.ps1.

4. To check the config for a single domain, run the following command:

Validate-DkimConfig <domain>

To show the full signing config, use the –showAll switch:

Validate-DkimConfig <domain> –showAll

To validate all domains for your organization, run the following command:

Validate-DkimConfig

You will be able to see if anything is wrong because the output is color coded.


function Validate-DkimConfig
{
    [cmdletbinding()]
    Param(
        [parameter(Mandatory=$false)]
        [string]$domain,
        [parameter(Mandatory=$false)]
        [switch]$showAll
    )

    if ($domain) {
        $config = Get-DkimSigningConfig -Identity $domain
        Validate-DkimConfigDomain $config -showAll:$showAll
    }
    else {
        $configs = Get-DkimSigningConfig
        foreach ($config in $configs) { Validate-DkimConfigDomain $config -showAll:$showAll}
    }

}

function Validate-DkimConfigDomain
{
    [cmdletbinding()]
    Param(
        [parameter(Mandatory=$true)]
        $config,
        [parameter(Mandatory=$false)]
        [switch]$showAll
    )

    # Display the configuration
    $domain = $config.Domain;
    $onmicrosoft = if ($domain.EndsWith("onmicrosoft.com")) { $true } else { $false }
    $actions = @()

    Write-Host "Config for $domain Found..." -ForegroundColor Yellow
    if ($showAll) {
        $config | fl
    }
    else {
        $config | Select Identity, Enabled, Status, Selector1CNAME, Selector2CNAME, KeyCreationTime, LastChecked, RotateOnDate, SelectorBeforeRotateonDate, SelectorAfterRotateonDate | fl
    }

    if (!$config.Enabled) {
        Write-Host "Config $($config.Name) Not Enabled" -ForegroundColor Yellow
        Write-Host
        $actions += "Config $($config.Name) needs to be Enabled"
    }

    # Get the DNS ENtries
    Write-Host "Locating DNS Entries..." -ForegroundColor Yellow
    $cname1 = "selector1._domainkey.$($domain)"
    $cname2 = "selector2._domainkey.$($domain)"
    $txt1 = $config.Selector1CNAME;
    $txt2 = $config.Selector2CNAME;

    $cname1Dns = Resolve-DnsName -Name $cname1 -Type CNAME -ErrorAction SilentlyContinue
    $cname2Dns = Resolve-DnsName -Name $cname2 -Type CNAME -ErrorAction SilentlyContinue
    $txt1Dns = Resolve-DnsName -Name $txt1 -Type TXT -ErrorAction SilentlyContinue
    $txt2Dns = Resolve-DnsName -Name $txt2 -Type TXT -ErrorAction SilentlyContinue

    # Validate Entries
    Write-Host "Validating DNS Entries..." -ForegroundColor Yellow   

    Write-Host   
    Write-Host "Config CNAME1 : $($config.Selector1CNAME)"
    if (!$onmicrosoft) {
        if ($cname1Dns) {
            Write-Host "DNS    CNAME1 : $($cname1Dns.NameHost)"
            Write-Host "TXT Hostname  : $($cname1)" 
            $match = if ($config.Selector1CNAME.Trim() -eq $cname1Dns.NameHost.Trim()) { $true } else { $false }
            if ($match) {
                write-host "Matched       : $($match)" -ForegroundColor Green
            } else {
                write-host "Matched       : $($match)" -ForegroundColor Red
                $actions += "Publish CNAME TXT Entry $($cname1) with value $($txt1)"
            }
        }
        else {
            write-host "DNS NotFound  : $($cname1)" -ForegroundColor Red
            $actions += "Publish DNS CNAME Entry $($cname1) with value $($txt1)"
        }             
    }

    Write-Host
    Write-Host "Config CNAME2 : $($config.Selector2CNAME)"
    if (!$onmicrosoft) {
        if ($cname2Dns) {
            Write-Host "DNS    CNAME2 : $($cname2Dns.NameHost)"
            Write-Host "TXT Hostname  : $($cname2)"
            $match = if ($config.Selector2CNAME.Trim() -eq $cname2Dns.NameHost.Trim()) { $true } else { $false }
            if ($match) {
                write-host "Matched       : $($match)" -ForegroundColor Green
            } else {
                write-host "Matched       : $($match)" -ForegroundColor Red
                $actions += "Publish DNS CNAME Entry $($cname2) with value $($txt2)"
            }
        }
        else {
            write-host "DNS NotFound  : $($cname2)" -ForegroundColor Red
            $actions += "Publish DNS CNAME Entry $($cname2) with value $($txt2)"
        }       
    }

    Write-Host
    Write-Host "Config   TXT1 : $($config.Selector1PublicKey)"
    if ($txt1Dns) {
        $key = $txt1Dns.Strings[0].Trim()
        Write-Host "DNS      TXT1 : $($key)"
        $match = if (Compare-PublicAndConfigKeys $key $config.Selector1PublicKey) { $true } else { $false }
        if ($match) {
            write-host "Key Match     : $($match)" -ForegroundColor Green
        } else {
            write-host "Key Match     : $($match)" -ForegroundColor Red
            $actions += "Public Key in TXT Entry $($txt1) needs to be republished..."
        }
    }
    else {
        write-host "DNS NotFound  : $($txt1)" -ForegroundColor Red
        $actions += "Microsoft TXT Entry $($txt1) not found so Signing Config needs to be recreated..."
    }

    Write-Host
    Write-Host "Config   TXT2 : $($config.Selector2PublicKey)"
    if ($txt2Dns) {
        $key = $txt2Dns.Strings[0].Trim()
        Write-Host "DNS      TXT2 : $($key)"
        $match = if (Compare-PublicAndConfigKeys $key $config.Selector2PublicKey) { $true } else { $false }
        if ($match) {
            write-host "Key Match     : $($match)" -ForegroundColor Green
        } else {
            write-host "Key Match     : $($match)" -ForegroundColor Red
            $actions += "Public Key in TXT Entry $($txt2) needs to be republished..."
        }       
    }
    else {
        write-host "DNS NotFound  : $($txt2)" -ForegroundColor Red
        $actions += "Microsoft TXT Entry $($txt2) not found so Signing Config needs to be recreated..."
    }

    # Write out neccessary Actions
    Write-Host
    if ($actions.Count -gt 0) {
        Write-Host "Required Actions..." -ForegroundColor Yellow
        foreach ($action in $actions) { write-host $action}
    }
}

function Compare-PublicAndConfigKeys([string] $publicKey, [string] $configKey)
{
    $match = $false;

    if (![string]::IsNullOrWhiteSpace($publicKey) -and ![string]::IsNullOrWhiteSpace($configKey))
    {    
        $regex = "p=(.*?);"
        $foundPublic = $publicKey -match $regex
        $publicValue = if ($foundPublic) { $matches[1] } else { $null }
        $foundConfig = $configKey -match $regex
        $configValue = if ($foundConfig) { $matches[1] } else { $null }

        if ($foundPublic -and $foundConfig)
        {
            if ($publicValue.Trim() -eq $configValue.Trim())
            {
                $match = $true;
            }
        }
    }

    $match;
}


I hope you find this useful.


How I personally use Outlook with Office 365

$
0
0

Sometimes people ask me how they should configure Outlook and Office 365 (Exchange Online Protection, or EOP) so they work together in the best way. This is tough for me to recommend because it depends on the local set up.

However, I can talk about how I personally use it. I am both a normal end user through my account at work, @microsoft.com. And, I am end user and administrator through my personal domain, @terryzink.com. I’m the only person on my personal domain although I do have many email accounts on it.

So, here is how I set up my own accounts so they work best for me from an antispam point of view. One is for me as an administrator of my personal domain with a very simple configuration (one user), and one as an end user in a large organization. This guide does not tell an administrator of a large organization how to set things up.

One more thing – your mileage may vary.

1. In your DNS provider (or following your organization’s DNS change-management process), set up the required SPF, DKIM, and DMARC records for your domain

For my personal domain and for Microsoft

2. In the Exchange Admin Center, I  turn off all of the Advanced Spam Filtering (ASF) options (your mileage may vary)

For my personal domain

I don’t have any of the Advanced Spam Filtering rules enabled for my personal domain – not SPF Hard Fail, not “Image Links to Remote Sites”, not “Backscatter NDR”, nor anything else. If you are an on-prem customer who uses Office 365 to relay outbound email, I recommend turning on the “Backscatter NDR” setting. I talk more about this here:

http://blogs.msdn.com/b/tzink/archive/2015/04/18/office-365-will-slightly-modify-its-treatment-of-anonymous-inbound-email-over-ipv6.aspx

The other ASF settings are too aggressive for me, if you’re reading this blog post you may want to enable more.

For Bulk Mail, I use the default level of protection which sends anything with a Bulk Confidence Level (BCL) of 7 or higher to junk.

image

I set both spam and high confidence spam to be marked with an x-header.

For Microsoft

I can’t divulge Microsoft’s configuration for ASF rules, but we send all spam and high confidence spam to the junk folder.

Some customers enable more ASF rules, but the majority don’t enable any.

3. Turn off Outlook’s Junk Email Filtering

For my personal domain

I don’t use Outlook for personal email because I like pressing Ctrl+U to see the raw source of a message. Instead of Outlook, I use a combination of the Thunderbird email client and pull my messages over POP3 (not IMAP, long story) and sort spam based upon an x-header using inbox rules, and I also check mail on my phone.

The problem with checking mail on my phone and using POP3 is that I download all of my email, spam and non-spam alike. I turned off moving junk mail to my Junk folder in Outlook Web Access, I am debating whether or not to turn it back on or to change the spam action to Modify Subject Line so that when my phone downloads something, I know that it’s spam.

For Microsoft

For my @microsoft.com account, I don’t have much in the way of configurability since I am not the administrator of the domain. However, for the most part it isn’t necessary.

I find Outlook’s junk mail double filtering too aggressive and sends too much good email to Junk, so I turned it off (right-click on a message > Junk > Junk Email settings > first tab, Options).

image

Under the Safe Senders tab, I uncheck “Also trust e-mail from my Contacts” and “Automatically add people I e-mail to the Safe Senders List.” If I want to add to my Safe Senders List, I right-click on the message > Junk > Never Block Sender.

image

Outlook still sends my messages to the spam folder. It does this because in EOP, when a message is marked as spam EOP sets the Spam Confidence Level (SCL) MAPI property in Exchange and sets the X-MS-Exchange-Organization-SCL header with the SCL 5, 6, or 9. Outlook knows that if the MAPI SCL property is 5-9, move it to Junk.

However, if my mailbox was hosted on-prem, I’d have to either create a local ETR to interpret the headers of the message that EOP stamps and set the SCL locally, as describe here:

https://technet.microsoft.com/en-us/library/jj837173(v=exchg.150).aspx

Or, you could have the X-MS-Exchange-Organization headers preserved when sending from EOP to your on-premise Exchange server and send the message over TLS.


4. Set up Advanced Threat Protection (ATP)

ATP is available from EOP either as a paid add-on, or you can upgrade to our E5 SKU from Microsoft which bundles a bunch of other services in addition to ATP.

For my personal domain

I have all the default options enabled with Do-not-track-clicks off, Do-not-allow-click-throughs off, and the only “Do not rewrite” URL is Netflix.com for some reason (I was probably testing it). I don’t have anyone enabled for Safe Attachments but I’m the only one in my domain. I don’t notice any degradation of service for my personal email. Almost every email I get is in HTML format so the rewritten links don’t look any different. When I click on any rewritten link, the lookup time is negligible and I never notice any lag.

For Microsoft

Much of Microsoft is on both Safe Links and Safe Attachments. Here, because we get so much plain text email, I do notice that sometimes it is inconvenient for messages to have URLs in plain text wrapped. We are looking into giving administrators the option to not rewrite plain text URLs. I might do that if I were an admin, but the tradeoff is that spammers can just send their messages in plain-text to some end users.

Other than that, I don’t notice anything special about my mailbox, the feature is seamless.


That’s it. That’s basically all I do to get basic mail flow working and configured with the optimal antispam settings (I do have some ETRs that are unique to my domain’s configuration).

Administrators will need to set up connectors, may want to set up ETRs for Criteria Based Routing, IP Allow rules, ETR Allow rules, etc. Microsoft as a company has all that stuff set up, but that’s out of scope of this blog post.

I hope this information helps.

DMARC one year later, and what have we learned?

$
0
0

It has been one year since I posted that Office 365 now supports inbound DMARC verification. What do we see in terms of how much mail it blocks in production?

Well, we’ve learned a lot of things; some of it good, and some of it bad. I took a look at our network-wide statistics yesterday and a total of 350 million messages arrived at our system with a DMARC record in the 5322.From address (I don’t have numbers on total email). Of that, about 3/4 of the messages passed DMARC and 1/4 of it failed:

image

That’s a lot of messages failing DMARC, but how much of that is marked as spam?

In Office 365, we don’t reject messages that fail DMARC with p=reject because (a) a lot of our customers have complex routing where they have other services in front of ours that break SPF, DKIM, and DMARC on legitimate email, and (b) a lot of senders don’t have all of their ducks in a row when it comes to SPF, DKIM, and DMARC; and (c) there is still some forwarded email (e.g., mailing lists) that fail DMARC that our customers could not get if we respected DMARC rejects.

So, given that, we override DMARC p=reject and stamp action=oreject for a DMARC fail/p=reject policy, or opctreject if it failed DMARC but the percent=xx randomly overrode a DMARC enforcement (the same for quarantine). Given that, how does the distribution of DMARC fail look like?

image

You can see that the majority of messages have a “don’t do anything special if this message fails DMARC” policy. Roughly one out of five messages that fail DMARC could get junked.

What do we do with messages that fail DMARC and subsequently go through the filter? Do they all get delivered to the inbox?

No. As it turns out, about 9 in 10 get delivered eventually. Other parts of our filter may still mark them as spam, so even if they fail DMARC with p=none, there is still a chance the message will be junked.

image

Of that 11%, what sends it to the spam folder? If not the DMARC policy, then what?

And of the non-spam that fails, does our filter also agree that the message is not-spam? Or our customers saying “We don’t care if it fails, it’s a good message so don’t bother even trying to filter it” ?

Here it is below:

image
Of messages that fail DMARC with p=none, our content filter still marks a message as spam for the majority of those cases. However, sometimes an end user’s blocked sender list (6%) flags it as spam, sometimes a customer’s Exchange Transport Rule (ETR) marks it as spam (4%, no doubt due to this blog post I wrote), and other times we think it’s malware (1%).

image

The picture is more complex for messages that are marked as non-spam that fail DMARC with p=none. That happens to 89% of DMARC fail/p=none messages, but you can see that much of the time end users are overriding with a safe sender (5%), or the message is an internal message that routes out and back in again within their organization (20%) or they have an IP allow or ETR allow rule (20%). Our filter only gets a chance to make a decision in just over half the cases.

So, clearly, there is a large justification for people to override a DMARC fail action if it happens so often. You may say “Oh, it’s not required because the filter won’t mark it as spam anyway” but in fact these are all pre-emptive overrides just in case because the message is failing sender authentication.

And to prove it, let’s go back to the messages that fail DMARC with p=reject. These should be marked as spam 100% of the time.

But they are not, here’s what that looks like:

image

In fully one quarter (!) of the cases, a DMARC fail/p=reject is not being marked as spam.

How can this be? Why aren’t 100% of messages that fail DMARC being marked as spam? Don’t we know that all sorts of bad things can happen?

Here’s why these messages are “rescued” from filtering:

image

From the above:

  • 6% of the messages are internal, meaning that someone has a hybrid mail configuration set up in Exchange Online Protection with the proper connectors between the cloud and their on-prem environment, but these messages are failing DMARC. However, since they treated as internal messages, filtering is not enforced. This is a good thing.

  • 27% (which sounds like a lot) is due to an IP or ETR allow. People know that sometimes a message will fail DMARC but yet they still need to override the verdict because it is a known breakage (e.g., mailing lists, or external senders to themselves that aren’t in their SPF or DKIM records). That part is a good thing.

    However, customers also do ETR allow rules just on a domain match. This is not a good thing because it’s easy to spoof a domain. I don’t have the breakdown on that.

  • 6% of the messages are rescued because of end user safe senders. This is a mixed bag. End users want to receive email from legitimate senders who get breakage due to forwarding or routing. But sometimes end user safe sender spammers, too. This is a mixed good/bad thing.

  • Finally, 61% of messages in this category are not marked as spam because our filters said not to mark it as spam. This is because we know that a lot of our customers have complex routing where SPF, DKIM, and DMARC are broken before we see the email message. If we enforced DMARC reject they would get far too many false positives. In this case we have to suppress DMARC enforcement.

This completely justifies our reticence to enforce DMARC rejects at the connection level. Our customers have complex configurations, there are a lot of broken senders out there, and end users sometimes want to receive messages even if the message fails DMARC.

So in the end, out of 350 million messages that failed DMARC, about 3.5 million were marked as spam because of a DMARC reject record and nothing else – about 1%.

I know that doesn’t sound like a lot, but 3.5 million spoofing/phishing messages that might otherwise have gone undetected is pretty good.

Exchange Online is rolling out default DKIM-signing to everyone

$
0
0

If you are a customer of Office 365 (Exchange Online Protection, or EOP), you may have noticed, or will be noticing, that we are adding DKIM signatures to your outgoing email, even if you haven’t explicitly enabled DKIM-signing for your domain (see instructions here: http://blogs.msdn.com/b/tzink/archive/2015/10/08/manually-hooking-up-dkim-signing-in-office-365.aspx). We are gradually rolling this out to everyone.

If you haven’t enabled DKIM-signing, EOP will create a default signing policy for your domain and use those in the selector and d= field in the DKIM signature. For example, suppose the customer Contoso Organization signed up with an initial domain of contoso.onmicrosoft.com, and one of their provisioned domains is fabrikam.com. An outgoing message with a default signature will look like the following:

From: Second Example <second.example@fabrikam.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
s=selector1-fabrikam-com; d=contoso.onmicrosoft.com; t=1429912795;
    h=From:To:Message-ID:Subject:MIME-Version:Content-Type;
    bh=<body hash>;
    b=<signed field>;

Don’t worry if you see this, it means that DKIM is being affixed properly. Assigning DKIM signatures to your domain will help improve your deliverability around the Internet, and it also means that you can send email over IPv6 even if you don’t publish an SPF record. It also helps receivers roll up domain reputation for your domain, instead of having to live within shared outbound IP space.

For verifiers, there is an algorithm we use to formulate the selector/signing domain:

  1. IF you see d=*.onmicrosoft.com, AND
  2. IF you remove ‘selector1/2-‘, AND
  3. IF you replace any dots in the From: address with dashes (e.g., fabrikam.com –> fabrikam-com) [1] and it matches the unremoved parts in the selector (e.g., selector1-fabrikam-com)
  4. THEN you can consider the message implicitly DMARC aligned.

The idea here is we, as Office 365, know who are customers are and can properly attribute the message but our customer may not publish an SPF record, or may even publish an incorrect one. We don’t have access to their DNS so we can’t publish the required CNAMEs, but we can put information into the headers of a message that uniquely identifies their domain because they have to put something into their DNS records to enable themselves as a customer of ours. As a receiver, you can use the above to unwind it.

I call this an implicit DMARC alignment, somewhat similar to a DMARC Best Guess Pass.

Next, if you enable DKIM manually, the selector and d= domain will change so that it aligns with the From: address:

From: Second Example <second.example@fabrikam.com>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed;
s=selector1; d=fabrikam.com; t=1429912795;
    h=From:To:Message-ID:Subject:MIME-Version:Content-Type;
    bh=<body hash>;
    b=<signed field>;

Every domain gets their own DKIM keys, they are not shared among customers. They are not even shared among domains within the same customer with the exception of when we cannot attribute a message to a particular organization (i.e., you send from a non-provisioned domain and we don’t know it’s your organization).

This is all part of our ongoing quest to ensure the best security experience for all of our customers.


[1] You can also get this information by doing an MX record lookup and using the <domainGUID>. For example, if the domain where fabrikam.com:

fabrikam.com.    3600  IN  MX  5 fabrikam-com.mail.protection.outlook.com.

The <domainGUID> is the part before .mail.protection.outlook.com. It will not always be the same as the dot/dash replacement I say above because the algorithm works a little differently for domains with dashes in it.

The common types of spear phish we see today

$
0
0

As 2015 draws near to a close, I thought I’d write a blog post about the type of spear phishes we are seeing lately against our customer base. This is not general brand phish like someone spoofing Paypal, but instead a phisher trying to impersonate your domain, for example, if the domain under attack is woodgrovebank.com, we’re assuming the name of the CEO is John Doe.

  1. Spoofing your exact domain in the From: address

    From: John Doe <ceo@woodgrovebank.com>
    To: Jane Roe <cfo@woodgrovebank.com>

    These are the most insidious attacks because if you are using Exchange and Outlook, if the email address matches what is in the Global Address Book, Outlook will pull the picture of the person from Active Directory and display it in the message, fooling you even more that the spoofed sender is legitimate when it is not.
    This is an industry-wide problem and the official solution is for domains to publish SPF, DKIM, and DMARC records if you want to do it yourself. If you can’t, then Exchange Online Protection (EOP) is rolling out its antispoofing solution that works automatically in the absence of SPF, DKIM, and DMARC. This is not something you turn on, we are going to enable it for everybody automatically so your domain is protected from Day 1.

  2. Lookalike spoofing

    From: John Doe <ceo@woodgrövebank.com>
    To: Jane Roe <cfo@woodgrovebank.com>


    We have seen an increase in these lately, perhaps 1/10 of all spear phishes. The domain being spoofed is not actually the domain but instead contains a non-Roman alphabet character (or two, or three) in it. As a result, at a glance, it looks like the normal domain but in reality it is not.

    Phishers use this technique to get around SPF, DKIM, and DMARC because they know that the spoofed domain can’t possibly pre-register every single domain that looks like theirs. However, the drawback for phishers is that if the recipient hits reply, unless the phisher has the domain registered with an MX pointing somewhere, the message will go no where. In addition, since the sending domain and email address will not be in the Global Address Book if using Exchange + Outlook, the picture of the person won’t show up; that’s a level of deception that the phisher loses by using this technique.

    We’re aware of this problem in EOP and we our working on it. When we release protection for it, it will be wrapped into our existing antispoofing solution above.

  3. ‘Display From’ attacks

    From: John Doe <john.doe@freemailprovider.com>
    To: Jane Roe <cfo@woodgrovebank.com>

    Here, the phisher sends a message to the CFO using his free web mail account, e.g., Gmail, Yahoo, or Outlook.com. The CFO is supposed to interpret this as the CEO not being able to get to his work account and instead sends a message from his mobile device.

    Attacks like these are effective in the sense that it’s possible to create accounts like this fairly easily by scraping around the Internet, looking for LinkedIn profiles, and creating reasonably legitimate-looking accounts at a free web mail provider. The message will also pass SPF, DKIM, and DMARC and will be sent from an IP with good reputation. Using regular language and email accounts may fool some users into thinking it is the real thing.

    The drawback is similar to #2; the Global Address Book will not pull the CEO’s picture and display it in Outlook/Exchange if using those two pieces of software. Also, the full email address will probably be displayed to the end user as it is unlikely to be in the person’s address book.

    As above, we’re aware of this problem. Any solution will go into our existing antispoofing solution above.

    In the interim, what some of our customers have done is create Exchange Transport Rules (ETRs) that say “If the message comes from ‘John Doe’ in the message header (or as a sender property), set the SCL to 9 (or delete the message, or send to quarantine) except of the message header ‘Authentication-Results’ contains the text ‘dmarc=pass action=none header.from=woodgrovebank.com’.

    That is not the only way to do it, of course, and you have to set up a DMARC record; alternatively, you could set up an SPF record and look for the results in that header, too. Anyhow, the idea here is that if John Doe sends to your organization, the message will be marked as spam unless it comes from your own domain. That will stop all Display From spoofs. The drawback is that all of the other John Doe’s in the world will also not be allowed to send email into your organization.

Those are the big three types of phishing attacks were are seeing beyond the regular phishing attacks that have been around for years. As you can see, some of them are addressed by sender authentication, but others are a result of phishers being forced to move to other attacks in response to stronger auth.

Email authentication should work out of the box and we should not rely upon domain owners to do it themselves

$
0
0

This is going to be a long post. Sorry. I didn’t have time to write a shorter one.

Who should be responsible for setting up email authentication records?

For years, I have been discussing the virtues of publishing email authentication records including SPF, DKIM, and DMARC. There are plenty of tutorials and documentation on the web, and I have written many articles in support of it, explaining its basics as well, and how it works in Office 365.

SPF, DKIM, and DMARC can be challenging to set up for numerous reasons:

  1. SPF requires that you know who all of your senders are and keep them up-to-date. This is hard to do in large, decentralized service where any department can contract out another email sender to send on your domain’s behalf.

  2. SPF also has a maximum 10 DNS lookups. In the modern world, it’s easy to exceed that limit with only a handful of 3rd parties sending as you. Office 365 alone requires you to have a spare lookup limit of 3 DNS lookups.

  3. DKIM is tricky to set up, and even trickier to rotate keys. You have to download and install the software, and then keep the keys in sync across all those servers you are running. Then, you have to publish keys in DNS making sure to copy/paste correctly. Then, when it comes time to update the keys, you have to do it all over again – unless, of course, you are using Office 365 in which case there is slim chance of copy/paste errors and you never have to rotate keys.

  4. DMARC is challenging to set up because you have monitor it all yourself and then make the decision to be more or less aggressive on your mail stream.

For domain owners who are highly motivated to do this sort of thing – people like me and all my friends at MAAWG – this is time-consuming but most of us enjoy it and so it’s not too bad.

However, I don’t think we should expect small business owners to do it. Instead, what I think should happen is this:

If you provide email infrastructure on behalf of someone else, you are responsible for helping your customers set up DKIM/DMARC alignment, and you should rotate keys automatically.

For example, if you are a large bulk email provider bulksender.com, you should:

1. Ensure that your customer sets up DKIM and then you rotate their keys automatically

a) Tell your customers to publish two common keys in their DNS, and these should be predictable. For example, for the domain contoso.com:

HereIsMyFirstKey._domainkey.contoso.com IN TXT “v=DKIM1; k=rsa; p=<public key #1>

HereIsMySecondKey._domainkey.contoso.com IN TXT “v=DKIM1; k=rsa; p=<public key #2>

b) The above is the bare minimum you should do. Ideally, you should not ask your customers to publish TXT records at all for DKIM, but instead publish CNAMEs that point to your own infrastructure:

HereIsMyFirstKey._domainkey.contoso.com IN CNAME HereIsMyFirstKey._domainkey.bigsender.com

HereIsMySecondKey._domainkey.contoso.com IN CNAME HereIsMySecondKey._domainkey.bigsender.com

Ideally, everyone would get their own key. However, this may not be possible as not everyone can store and replicate multiple keys securely. In that case, you may be stuck with common keys.

c) Third, the big sender should sign with the first key and then 6-12 months later, flip over to the second key. The first key should remain in place but then updated to new values a week or two later. Then, 6-12 months later, repeat the process and flip back.

Doing it this way ensures that the customer has DMARC alignment based upon the d=domain in the DKIM signature, and their DKIM keys are rotated automatically without them having to do anything. If you ask your customer to update their DKIM keys, my guess is that perhaps 5% of them will ever do it.

So don’t make them do it. Do it for them automatically by putting them into a good state from the beginning in that their email from your infrastructure is DKIM/DMARC aligned. You can still have your own 5321.MailFrom be your own domain for the purposes of bounce-processing and passing SPF.

Here in Office 365, we’re working on modifying the signup process such that in addition to publishing a couple of DNS records to verify your domain, you also have to publish some CNAMEs to get your DKIM properly set up, too. We use the DKIM selectors selector1 and selector2.

2. If you are a domain registrar with access to the domain’s DNS settings, you should auto-populate their SPF record with options to manage it themselves

SPF records are hard to maintain yourself for a couple of reasons. The first is if you are a small business owner, you probably have better things to do than play around with DNS settings and updating your SPF record (unless you are an email geek like me; if you’re not an email geek you do what your email service tells you to do). I see plenty of small domains and big companies get their SPF record syntactically wrong.

The second reason SPF records are hard to maintain is because once your business grows, you have to keep maintaining a list of 3rd party senders who send on your behalf and keep adding them to your SPF record, but SPF has a cap of 10 DNS lookups. In 2016, it’s easy to exceed that limit. Office 365 takes up 3, Google Apps takes up 3, and SalesForce takes up 2. You’re now at 8. How many more bulk senders can you add? The normal solution is to break it up by subdomains so some senders send from email.contoso.com, others from e-mail.contoso.com, and so forth.

That’s a pain to maintain, too. Believe me, I know, because I’ve worked with teams internal to Microsoft to set it up. The process of setting up these subdomains and then making DNS changes through the change review board is not fast, and even I find it hard to juggle so many different teams’ requests.

A big innovation I see in this space is through the use of SPF macros. Macros allow you as a domain owner to publish a wildcard in your SPF record and instead of returning a static SPF record, it looks it up dynamically. For example, suppose contoso.com’s traditional SPF record looked like this:

contoso.com IN TXT “v=spf1 ip4:1.2.3.4 include:spf.protection.outlook.com include:_spf.google.com include:salesforce.com include:bigsender.com –all”

Contoso knows that it has gone over the limit, so instead it publishes a macro in its SPF record:

contoso.com IN TXT “v=spf1 include:%{i}._ip.{%h}._ehlo.{%d}._spf.contoso.com –all”

Huh?

What macros do is tell the receiving mail server to substitute the part in %{…} with the corresponding value from the email you are checking with the metadata from the email. In the above:

%{i} means the connecting IP
%{h} means the connecting mail server’s EHLO or HELO string
%{d} means the domain you are looking up

If a mail server were to look up the SPF record for contoso.com and see that weird syntax, it would then do a second lookup substituting in the parameters from the message. So, if the email comes from the IP 1.2.3.4 from a mail server whose EHLO string is mailserver.com, and checking the domain contoso.com, then substituting the corresponding parts it looks up the SPF record for 1.2.3.4._ip.mailserver.com._ehlo.contoso.com._spf.contoso.com. The response might be something like this:

contoso.com IN TXT “v=spf1 ip4:1.2.3.4 –all”

A mail server could then do a regular SPF evaluation to see if 1.2.3.4 is in that particular SPF record. In this case it is.

If instead a message arrived from a mail server with an EHLO string NAM02-CY1-obe.outbound.protection.outlook.com from the IP 65.55.159.61, the server would look up 65.55.159.61._ip.NAM02-CY1-obe.outbound.protection.outlook.com._ehlo.contoso.com._spf.contoso.com then the DNS server might return:

contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com –all”

The mail server then does a normal SPF lookup on this record to see if the IP is in there (it is).

Or, instead if a message arrived with an EHLO string of localhost from the IP 254.1.1.1, the server would get back the macro and then do a second lookup on 254.1.1.1._ip.localhost._ehlo.contoso.com._spf.contoso.com. The DNS server might return:

contoso.com IN TXT “v=spf1 –all”

Because 254.1.1.1 is not in your SPF record at all, the dynamic SPF says it’s not there and tells the verifier to stop checking.

So, rather than you having to publish a static DNS record, you have a backend database that parses out variables from the DNS query and dynamically returns the corresponding response.

Receiver –> DNS –> parse out DNS query –> look up in table —+
                                                              |
                                                              |
                                 +—————————-+
                                 |
                                 +
+——-+———+————————————————+

| IP    | EHLO    | SPF record to return                           |
+——-+———+————————————————+
| IP #1 | EHLO #1 | v=spf1 include:1.2.3.4 –all                    |
| IP #2 | EHLO #2 | v=spf1 include:spf.protection.outlook.com –all |
| IP #3 | EHLO #3 | v=spf1 include:_spf.google.com –all            |
| IP #4 | EHLO #4 | v=spf1 include:_spf.bigsender.com –all         |
+——-+———+————————————————+
                                |
                                |
                 Return corresponding SPF record
                                |
                                |
                  Receiver does normal SPF check

Instead of publishing all of your 3rd parties who send as you in DNS, you would use a backend database (preferably through a nice, pretty UX) to add all your senders (the fact that this publishes an SPF record is abstracted from you; you don’t even need to know what SPF is). Ideally, it would have a checkbox for the most common large senders but also let you manually enter IPs that are not associated with any common large sender.

Who do you use to send email?

_x_ Office 365
___ Google Apps
_x_ SalesForce
_x_ Big Sender
_x_ Click here to enter in IPs manually

You could enter in as many as you like because you aren’t worried about exceeding any DNS lookups, the lookup table parses it all out, looks it up in the table, and then dynamically returns the result depending upon the IP/EHLO combination.

It’s an elegant way around the 10 DNS lookups, helps prevent syntax errors, lets you stuff all of your senders into your root domain (@contoso.com instead of @email.contoso.com) and makes it easier to manage over the long term.



I, as an author of this blog and speaking of my own opinion and not necessarily on behalf of Microsoft, bring all this up and describe it this way because there is a small company that provides exactly this type of service called ValiMail (my examples above using macros copies their syntax).

I’m familiar with their service and I like what they do, it’s a great way to outsource your email authentication so you don’t have to do it yourself. I also like how they have a lot of automation involved when you set up DMARC records, your SPF record self-populates. That sets you up into a good state and keeps you there.

If you’re looking for a service to do this, you may want to see what they offer.


And this brings me back to the registrars.

It would be great if registrars provided a service like this for all their customers. When you registered a new domain, you would get a wildcard SPF record by default with the option to manually do it yourself. The registrar would also publish a default DMARC record for you with the option for you to manually do it yourself. In that way, if you don’t know how to do it yourself, your registrar takes care of it for you. And if you feel confident to do it yourself (or you’re a geek like me and enjoy doing it yourself) you could take over by picking the “I want to do it myself” option.

But in either case, your email is SPF/DMARC aligned and kept up-to-date with minimal inputs from you, right from the beginning of domain registration. I understand that registrars would then have to provide this service, but maybe they could contract that out.

Putting it all together

So you see, rather than making domain owners be responsible for their own email authentication, service providers should be responsible for helping their customers set it up:

  1. If your service sends email, make sure that your customer enables DKIM/DMARC alignment by telling them what CNAMEs to publish (or barring that, at least a TXT record) and ensure that they cannot complete signup without them.
  2. If your service is a registrar or can control DNS records, auto-populate their SPF record to get SPF/DMARC alignment with an option to do it themselves. Ideally, you would also publish a DMARC record. That’s a story for another post (or maybe when I get the motivation, come back and extend this post).

Even though there are many administrators today who set this up and take it as a cost of doing business, there are probably even more small business owners (and even medium and large) who don’t have the expertise to set it up and maintain.

I think that this is the best way to get email authentication compliance. Let’s make it simple so all a customer or domain owner has to do is answer a few questions and the backend takes care of the rest.

Forward!

Office 365 is expanding its DKIM-signing to our consumer brands plus adding default signatures to enterprise email traffic

$
0
0

Here at Office365 and Hotmail/outlook.com, we are making some changes with regards to our DKIM-signing in both services. We believe in sender authentication, especially with regards to DKIM, and plan to sign 100% of all email in both services.

1. First, email traffic from our consumer brands will all be DKIM-signed (eventually)

First, Outlook.com and Hotmail.com are going to start rolling out DKIM-signing for outbound traffic originating out of Office365 for users that have been migrated off the old outlook.com/Hotmail stack and onto the Office 365 stack. We are about 1/3 of the way finished that, so not every email from those domains will have DKIM signatures.

Initially, we will (a) add DKIM signatures only to a handful of domains, e.g., to @gmail.com and @yahoo.com and a couple of others, and (b) it will slowly ramp up from a handful of mail servers to 100% of mail servers. (c) Then, we will start signing for all outbound traffic to all domains. This is to prevent this traffic from going from 0 to millions of new DKIM-signatures overnight.

These domains will sign with d= domains in the DKIM-signatures that match the domains in the From: address (e.g., d=outlook.com, d=hotmail.com) which means they will be DKIM/DMARC aligned.

You can read more about the outlook.com to Office 365 migration here:

- New ways to get more done in Outlook.com
https://blogs.office.com/2015/05/21/new-ways-to-get-more-done-in-outlook-com/

2. Second, all outbound traffic out of Office 365 will add a second signature

Updated on Jan 27, 2016

a) All outbound email for our enterprise traffic in the Office 365 service will start getting a default DKIM signature with d=office365.com for email that originates out of our normal risk delivery pools (i.e., email that is not marked as spam). This means there will be a From: domain that does not align with the d= domain in the DKIM signature.

From: Regular User <user@contoso.com>
DKIM-Signature: s=selector1; d=office365.com

This second signature will be added because we use it to aggregate our outbound sender reputation at a service level rather than at an IP-level. We then use it to monitor our service’s spam/non-spam ratio and take action when we observe our users sending high volumes of spam. This helps us maintain good deliverability around the Internet.

This DKIM signature will not be DKIM/DMARC aligned, it is added for outbound spam prevention.

b) For email that originates out of our high risk delivery pools (i.e., email that is marked as spam [before we shut them down], NDRs, other questionable quality email), will have a different d= domain, we haven’t set it up yet but will probably be something like d=somethingAtMicrosoft.net.

From: Regular User <
user@contoso.com>

DKIM-Signature: s=selector1; d=somethingAtMicrosoft.net

You can read more about our outbound spam strategies here:

- Higher Risk Delivery Pool for Outbound Message
https://technet.microsoft.com/en-us/library/jj200746(v=exchg.150).aspx

- Configure the Outbound Spam Policy
https://technet.microsoft.com/en-us/library/jj200737(v=exchg.150).aspx

This DKIM signature will not be DKIM/DMARC aligned, it is added for the purpose of outbound spam prevention.

c) All traffic out of Office 365 should also have a second signature with either a From/d= aligned signature because the customer has set up DKIM signing, or a default signature we add on their behalf that looks like this:

From: Regular User <user@contoso.com>
DKIM-Signature: s=selector1; d=contoso.com

OR

From: Regular User <user@contoso.com>
DKIM-Signature: s=selector1-contoso-com; d=contoso.onmicrosoft.com

You can see we encode the From: domain into the selector in the event that a customer doesn’t set up DKIM in our UX. This is not yet deployed to our entire customer base but we are working on it (we have to update Active Directory for everyone and then publish DNS records and that takes time).

The custom DKIM signature will be DKIM/DMARC aligned. The default custom DKIM signature is not DKIM/DMARC aligned but with a simple adjustment of the DMARC algorithm (which we do internally), you can extract the domain from the DKIM signature and use it as de factor DKIM/DMARC alignment.
 
For more information:

- Exchange Online is rolling out default DKIM-signing to everyone
http://blogs.msdn.com/b/tzink/archive/2015/12/16/exchange-online-is-rolling-out-dkim-signing-to-everyone.aspx

- Manually hooking up DKIM signing in Office 365
http://blogs.msdn.com/b/tzink/archive/2015/10/08/manually-hooking-up-dkim-signing-in-office-365.aspx


3. Third, there are some things to be aware of with these DKIM signatures

a) Some of these DKIM-signatures will be broken because our customers route email through Office 365 and then through other systems which modify the message.

RFC 6376 – the RFC that defines DKIM – specifically says to treat a message with a broken DKIM signature as an unsigned message. This is because a DKIM signature may not survive forwarding along the Internet (headers get added, footers get added, messages get formatted, From: addresses get rewritten, etc.). For this reason, a message with a broken DKIM signature out of Office 365 should definitely not be rejected because there is a lot of legitimate email that may become modified.

DomainKeys Identified Mail (DKIM) Signatures
https://tools.ietf.org/html/rfc6376

6.1.  Extract Signatures from the Message

Survivability of signatures after transit is not guaranteed, and
signatures can fail to verify through no fault of the Signer.
Therefore, a Verifier SHOULD NOT treat a message that has one or more bad signatures and no good signatures differently from a message with no signature at all.

6.3.  Interpret Results/Apply Local Policy

In general, modules that consume DKIM verification output SHOULD NOT determine message acceptability based solely on a lack of any
signature or on an unverifiable signature; such rejection would cause severe interoperability problems.

If a receiver does choose to reject messages on an invalid DKIM signature, they can either disable that functionality, exclude it for messages originating out of Office 365, or accept the false positives associated with this configuration.

b) Finally, our consumer traffic may or may not have the second DKIM signature with d=office365.com.


These are the changes that are coming to both services. For the average user, the changes will be transparent to you, you won’t notice anything different. However, what we as an industry use them for are antispoofing checks; antispoofing failure mitigations (an email that originates out of outlook.com which is forwarded will fail SPF but pass DKIM and therefore DMARC); better sender authentication which means that 3rd parties can block more spam, that is, suspicious email “from” outlook.com; better reputation tracking for email originating out of Office 365; improved deliverability to 3rd party services; and finally, compliance with industry best practices.

Thanks for reading.

I don’t mean to name and shame, but I will

$
0
0

A few months ago, I made the mistake of signing up for a webcast that opted me in to getting continuous communicates from them about upcoming online seminars.

I was getting tired of all of these invites so I unsubscribed.

I kept getting more invites so I unsubscribed again.

I kept getting even more invites so I unsubscribed a third time!

And then I got more invites!

I got fed up with this sender – I won’t name them out loud but let’s just say that it rhymes with ‘RightTalk’. I finally ended up creating an Exchange Transport Rule (ETR) about two months ago:

PS C:\> Get-TransportRule -Identity “Block RightTalk” | fl Description

Description : If the message:
  sender’s address domain portion belongs to any of these domains: ‘mail01.righttalk.net’

Take the following actions:
  reject the message and include the explanation ‘Stop spamming me I already unsubscribed three times’ with the status code: ‘5.7.1’

As you can see, I am rejecting the message and bouncing it back to them (they send authenticated email so I am not concerned about sending backscatter) and I even tell them why I am doing this. Ever since I created that rule two months ago, I haven’t gotten any messages from them in my inbox.

<Relaxed sigh>

However, yesterday I decided to check whether or not they took the hint and actually unsubscribed me.

Nope.

PS C:\> Get-MailDetailTransportRuleReport -TransportRule “Block RightTALK”

Date      Subject
1/26/2016 1:31:32 PM Now on Demand: Take Control of Your 2016 Operations Budget with Fleet Management Technology
1/28/2016 7:49:00 PM On Demand: Take Control of Your 2016 Operations Budget with Fleet Management Technology

Even as recent as this past week, they are still sending me email. Neither unsubscribing nor bouncing their email has worked. As a customer of the Exchange Online service, I’m happy to not see these messages getting delivered to me. However, as a citizen of the Internet who fights spam for a living, I’m irritated that this sender does not unsubscribe me, nor do they do any sort of SMTP bounce processing.


Common errors in SPF records

$
0
0

The other day I was asked to come up with some common errors that we see when people set up SPF records as we want to start notifying our customers when they have these types of errors. I thought it would be a good idea to make this public and add to it as necessary.

1. You have too many DNS lookups

SPF specifies a maximum number of 10 DNS lookups (e.g., nested includes, A-record resolution, MX-record resolution, PTR record resolution, etc. are greater than 10). I talk about how to get around this here:

http://blogs.msdn.com/b/tzink/archive/2016/01/22/email-authentication-should-work-out-of-the-box-and-we-should-not-rely-upon-domain-owners-to-do-it-themselves.aspx

Most verifiers, if they can verify you before hitting the 10 DNS lookup limit, will pass if you haven’t hit 10 up to that point. But go over and this will generate a PermError.

2. Too many SPF records in DNS

This occurs when someone meant to add more IPs or 3rd parties to their SPF record and created a second TXT record instead of including them all into the first one:

contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com –all”
contoso.com IN TXT “v=spf1 ip4:1.2.3.4 –all”

Whereas it should be this:

contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com ip4:1.2.3.4 –all”

If this occurs, SPF should return a PermError. We here at Office 365 will cut you some slack and interpret two SPF records as a single one; that is, we detect your probable intent even though you didn’t do it correctly. Most other receivers are not nearly as forgiving.

3. Duplication of what you need for Office 365 to work

You only need to have include:spf.protection.outlook.com in your SPF record, you don’t need any of the following:

include:outlook.com
include:spf.messaging.microsoft.com
include:spf.frontbridge.com

While SPF still works if you include any of the others, you will get unnecessarily closer to the 10 DNS lookup limit (this prevents you from adding much more 3rd party services to send on your behalf).

As a matter of fact, if you have any of those in your SPF record, you should replace them with include:spf.protection.outlook.com.


4. Forgetting to include an ip4: and instead putting ip:

For example:

contoso.com IN TXT “v=spf1 ip4:1.2.3.4 ip4:5.6.7.8 ip:9.10.11.12 –all”

Office 365 will cut you some slack and interpret an ip: as ip4:. Other services aren’t quite so forgiving and may generate a PermError.

5. Not properly formatting the double quotes

You can put additional double quotes into the SPF record as long as you put spaces in the correct places:

contoso.com IN TXT “v=spf1″ “include:spf.protection.outlook.com” “ip4:1.2.3.4″ “include:_spf.google.com” “–all”

The above is created by someone who is trying to add additional 3rd parties to their SPF record by putting them in double quotes. However, this concatenates the record so it gets treated as this:

contoso.com IN TXT “v=spf1include:spf.protection.outlook.comip4:1.2.3.4include:_spf.google.com–all”

Email receivers will treat this as you not having an SPF record at all.

It should be this (better to not include any quotes at all because you are using up characters in your SPF record, and you only have 255 to spend):

contoso.com IN TXT “v=spf1″ ” include:spf.protection.outlook.com” ” ip4:1.2.3.4″ ” include:_spf.google.com” ” –all”


6. Exceeding 255 characters in an SPF record

If you are going over 255 characters, it should be separated by a space and max of two components:

contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com <bunch of others> ip4:1.2.3.4 ” “ip4:5.6.7.8 –all”

If you go over 255 characters, I think most receivers will return a PermError but it varies by receiver.

7. Forgetting to include key parts of the record indicating that it is an SPF record

These are all incorrect:

contoso.com IN TXT “include:spf.protection.outlook.com ip4:1.2.3.4 –all”
contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com ip4:1.2.3.4″
contoso.com IN TXT “include:spf.protection.outlook.com ip4:1.2.3.4″

The first one lacks v=spf1, the second does not have a failure policy, and the last one lacks both.

If you do this, most receivers will either return a PermError or will treat it as if there is no SPF record.


8. Having a failure policy of +all

It’s okay to have any of the following policies: -all, ~all, ?all. Don’t do +all:

contoso.com IN TXT “v=spf1 ip4:1.2.3.4 +all”

This allows the entire world to send email as you. I was talking to another (very) large receiver at M3AAWG this past week and we’re talking about tightening up this behavior by rejecting in SMTP if you have a +all in your SPF record.

So don’t do it.


9. Having large IP ranges in your SPF record

IP ranges larger than /16 usually are wider than you need:

contoso.com IN TXT “v=spf1 ip4:1.0.0.0/2 –all”

That allows far too many IPs to send on your behalf. This is another pattern that I was talking about with the large receiver at M3AAWG that we are looking to start rejecting in SMTP.


10. Including a PTR in your SPF record

This one… I am not so strict about:

contoso.com IN TXT “v=spf1 ptr ptr:contoso.com –all”

The SPF spec says not to do this and it is deprecated, but I can see the rationale for still using it. So while I recommend not to use it, I’m not so sure I’d want to reject on it.


Those are the most common errors that we see today, and how they are treated. As I think of or discover more, I will add to the list.

To test SPF records, I use the following two websites:

SPF Record Testing Tools:
http://www.kitterman.com/spf/validate.html

DMARCIAN’s SPF Surveyer
https://dmarcian.com/spf-survey/

Other than some Perl and shell scripts I wrote internally, that’s all I basically use.

How antispoofing protection works in Office 365

$
0
0

Exchange Online Protection (EOP), the email filtering component of Office 365, is rolling out, or has already rolled out, full antispoof protection for all of its customers. Most of our customers already have this protection, and now we are preparing to roll it out to everyone else.

What is the spoofing problem EOP is solving?

Over the past several months, EOP (along with the rest of the industry) has seen a massive increase in targeted spear phishing attacks where the attacker spoofs an employee high up in the organization such as the CEO, and emails another high ranking employee such as the CFO, asking them to perform a wire transfer, or similar behavior. There are no attachments or links in the message, only regular language instructions asking the target to comply with the demands.

For example:


From: Rudy Bosive (the CEO) <rudy@woodgrovebank.com>
To: Tom Amtir (the CFO) <tom@woodgrovebank.com>
Subject: Can you make this wire transfer for me?

Tom, we just closed on an acquisition of a new service but we’re trying to keep it quiet. Could you wire over $50,000 to them? The account number is below and we need to get this taken care of today.

Thanks.

Rudy

Sent from Outlook for iPhone



These types of messages are sent in low volume, and contain language that is grammatically correct without spelling errors, but are fraudulent. They also can cause a lot of financial damage to an organization if the targeted recipient is not aware that the message is malicious.

EOP already has antispoofing technology including SPF, DKIM, and DMARC [1]. However, many organizations don’t have either the expertise or resources needed to set these up or maintain them. Yet, they still need antispoofing protection – these types of messages must be detected and users protected even if you don’t have the required authentication records published.

This Exact Domain Spear Phishing Protection (name subject to change) is the new feature that EOP has rolled out for many of its existing customers, and will roll out for the rest over the next few weeks – the detection of spoofing for your domain even without the standard SPF, DKIM, or DMARC records.

You don’t have to do anything to receive this protection, you get it for free automatically. Furthermore, you can customize it to make it more or less strict by permitting others to send on your behalf.
 

How is EOP solving it?

First, EOP checks to see if the message is destined to your organization and comes from any of your provisioned domains, or a subdomain of any of your provisioned domains. For example, any of these may be potential spoofs:

From: Rudy Bosive (the CEO) <rudy@woodgrovebank.com>
To: Tom Amtir (the CFO) <tom@woodgrovebank.com>

From: Rudy Bosive (the CEO) <rudy@foo.woodgrovebank.com>
To: Tom Amtir (the CFO) <tom@woodgrovebank.com>

From: Rudy Bosive (the CEO) <rudy@woodgrovebank.com>
To: Tom Amtir (the CFO) <tom@bar.woodgrovebank.com>

From: Rudy Bosive (the CEO) <rudy@foo.woodgrovebank.com>
To: Tom Amtir (the CFO) <tom@bar.woodgrovebank.com>

If the From and To domains align, EOP checks to see if the message is legitimate. Is it a message that originated internally within the organization? That’s okay. Are they authorized to send email on your behalf? That’s also okay. Is it a known good bulk sender? That’s okay. EOP also uses the sending domain reputation (or lack of reputation), sending IP reputation, recipient reputation (how many messages do you receive from this sender? how is your email routed through the EOP service?), and also makes use of machine learning.

EOP uses of all of this data to make sure that the service marks malicious email as spoofs, and not any legitimate email.

How to tell if EOP thinks a message is a spoof

When EOP thinks a message “from” your domain to any of your domains is a spoof, it marks it as spam and adds the following header:

X-Microsoft-Antispam: …;SFTY:9.5

The X-Microsoft-Antispam header is already used by EOP to indicate various other spam filtering components. The SFTY:9.5 refers to the Safety Level of a message. This header property has other values but are reserved for internal use by EOP.

You can double check this by looking at the From: address and the To: address. Both domains will be the same, subdomains of each other, or part of your Accepted-Domains.

By the end of the second quarter of 2016, EOP will start adding Safety Tips to the message. Safety Tips are visual indicators letting you know that the message is fraudulent or may be a phishing scam. These Safety Tips are viewable when using Outlook on the web to view your email.

image

How much email will be marked as spam because of this change?

Note: This is an upcoming feature and is not yet available as of Feb 22, 2016

In order to view how much email this feature marks as spam, EOP will also give you the ability to see who is sending email with your domain in the 5322.From header (the one that shows up in the email client) to your organization’s domains. For example, suppose you had all of the following domains provisioned:

contoso.com
fabrikam.com
woodgrovebank.com

This report would tell you all combinations of:

From: To:
contoso.com  contoso.com 
contoso.com  foo.contoso.com 
contoso.com  fabrikam.com
foo.contoso.com  bar.woodgrovebank.com

And so forth.

To view this report using Powershell:

  1. Connect to Exchange Online using Powershell

  • Run the following cmdlet:

    Get-PhishFilterPolicy –SpoofAllowBlockList | Export-CSV “SpoofingSenders.csv”

    Or, to get more details:

    Get-PhishFilterPolicy –SpoofAllowBlockList –Detailed | Export-CSV “DetailedSpoofingSenders.csv”

    This cmdlet will export all data sending to your organization over the past 30 days.

  • Open and review the downloaded *.csv file. The column names are detailed below, with the ones that will be marked as spam bolded in purple:


    Column Description

    PSComputerName

    Used internally by EOP

    RunspaceId

    Used internally by EOP

    PsShowComputerName

    Used internally by EOP
    True Sender The organizational domain of the PTR record of the sending IP address that is spoofing your organization. If no domain is found, the sender IP is shown.

    Mail Volume

    Number of messages sent to your organization from this sender in the past 30 days

    User Complaints

    Number of junk submissions from your users corresponding to this sender in the past 30 days

    Allowed to Spoof

    Indicates whether this entity is allowed to spoof your organization. There are 3 possible values:

    Yes: All spoofed addresses from this spoofing sender will be allowed to spoof your organization

    No: No spoofed addresses from this spoofing sender will be allowed to spoof your organization. Messages from this sender will be marked as spam when the feature is enabled.

    Partial: Some spoofed addresses from this spoofing sender will be allowed to spoof your organization, the rest will be marked as spam. Add –Detailed parameter to see the specific addresses.

    Spoofed Sender
    (only included with 
    –Detailed parameter)

    The visible spoofed sender that the message appears to be sent from (in the From: address in your email client, also known has the header.from)

    Authentication Result (only included with –Detailed parameter)

    This shows Passed if the sender passed EOP sender authentication checks (such as SPF or DKIM), Failed if the sender failed EOP sender authentication checks, or Unknown if the result is not known

    Source of Allowed to Spoof (only included with –Detailed parameter)

    Indicates if the Allowed to Spoof verdict was automatically determined by the phish filter or set by the administrator.

  • Integration with the Office 365 Admin Portal is not available at this time, but will be by the end of Q2 2016. When that occurs, this blog post will be updated.


    How can you exclude certain senders from being marked as spam?

    Even though EOP attempts to exclude all legitimate senders from being marked as spam, it is possible that the service does not have enough history to make that determination.  This can happen if a new sender starts sending email as you, or the volume of email is too small to generate a positive reputation, or the sender comes a set of IPs that are known to send spam or phish. EOP does self-correct in cases where it has enough data to make a decision, but other corner cases may continue to be marked as spam.

    To eliminate the chance of false positives, we recommend administrators use one of the following four options to exclude senders from being marked spam by our antispoofing filter:

    1. Add the sender’s IP addresses to your SPF record, or have them authenticate with DKIM

      You can use either SPF or DKIM to have a 3rd party send email on your behalf (see note [2] at the end of this blog post for examples).

      Whichever method you choose, or ideally picking both methods, this will allow EOP to authenticate emails from these 3rd party services sending as your domain to your domain.

      However, it also allows others (for example, Yahoo, Gmail, Comcast, and so forth) to verify email to them sending as you. This is beneficial because it allows your customers to build trust with your domain no matter where their mailbox is located, and at the same time EOP will not mark a message a spam due to spoofing because it passes authentication of your domain.

      Advanced users: For more information on getting SPF records to work around the 10 DNS lookup limit, and automating your SPF records, see the blog post Email authentication should work out of the box and we should not rely upon domain owners to do it themselves.

    2. Use the spoof policy in Powershell or UX in the Exchange Admin Center to exclude the sending IP address from Antispoofing enforcement

      This functionality is not yet available as of Feb 22, 2016 but will be released by the end of Q2 2016.

      Alternatively, if you want to permit various senders to send email on your behalf to your domain (but still undergo spam filtering), you can explicitly set this up.

      To allow a specific IP address to send email as your domain to your domains using Powershell:

      a) Connect to Exchange Online using Powershell

      b) Download the spoofed senders list, as above, with either basic or detailed view.

      Get-PhishFilterPolicy –SpoofAllowBlockList | Export-CSV “SpoofingSenders.csv”

      Or

      Get-PhishFilterPolicy –SpoofAllowBlockList –Detailed | Export-CSV “SpoofingSenders.csv”


      c) Open up your favorite text editor (such as Notepad or Notepad++) and look for the IPs you want to specify overrides and type either Yes or No in the Allowed to Spoof column.

      - Specifying ‘Yes’ in the Standard view will allow all detected sender addresses from that IP address to send as your domain. Specifying ‘No’ will prevent it (that is, if EOP may believe it is a good sender but you can override it). Future sender addresses detected by the phish filter may or may not be blocked or allowed

      - Specifying ‘Yes’ or ‘No’ in the Detailed view will allow or block the detected sender address from the true sender sender spoofing your domain.

      - If there are spoofing senders you would like to proactively allow or block that do not currently appear in the Detailed view, you can specify the “True Sender”, “Spoofed Sender”, and the “Allowed to Spoof” verdict.

      Save the file with a new name UpdatedSpoofingSenders.csv.

      d) Update the changes you made in the “Allowed to Spoof” column by running the following two commands in this order:

      $UpdatedSpoofingSenders=Get-Content -Raw UpdatedSpoofingSenders.csv

      Set-PhishFilterPolicy –SpoofAllowBlockList $UpdatedSpoofingSenders

      Senders who you did not allow will be blocked.

      e) To verify your changes:

      Get-PhishFilterPolicy –SpoofAllowBlockList

      Or

      Get-PhishFilterPolicy –SpoofAllowBlockList –Detailed

      Integration with the Office 365 Admin Portal is not available at this time, but will be later on in 2016. When that occurs, this blog post will be updated.

      Bonus: Office 365 is planning to give you insights into Intelligence Data. This encompasses a whole range of signals from our various platforms. These spoofing senders – many of whom are malicious – will in this Intelligence Data set. It is not the only set of signals we will provide, but is an important one. This Intelligence Data will be available later this year (2016).

    3. Add the sender’s IP address to an IP Allow List in the Exchange Admin Center

      Alternatively, if you don’t want to add the sender’s IP addresses to your SPF record but also don’t want email to be marked as spam due to spoofing, you can add the IP or IPs to your IP Allow List. This will skip any spam filtering from these IP addresses and mark the message as SCL –1, which delivers the message to your inbox unless an Exchange Transport Rule (ETR) deletes or quarantines the message.

      If a message bypasses filtering due to an IP Allow list entry, you will see the following properties inserted into a header in the message:

      X-Forefront-Antispam-Report: IPV:CAL;…SFV:SKN;…SCL:-1

      The IPV:CAL means “IP Filtering Verdict: Customer Allow List”.

      For more information on managing IP Allow lists, see

      Configuring the connection filter policy in EOP.

    4. Add the sender’s IP addresses plus sending domain to an Exchange Transport Rule (ETR) which bypasses filtering

      Alternatively, if you don’t want to skip filtering on all email from a specific set of IP addresses into your organization because those IPs are shared by more than just your domain, you can create an ETR. The ETR should look for a combination of sending IP addresses and a sending domain. You should never create an ETR that only looks for your domain because this can be easily exploited by spammers.

      The ETR syntax should be:



      Name: Antispoofing Allow rule to bypass filtering

      If the message…
      sender ip addresses belong to one of these ranges: ‘1.2.3.0/24’
      and sender’s address domain portion belongs to any of these domains: ‘contoso.com’

      Do the following…
      Set the spam confidence level (SCL) to ‘-1’
      and set the header ‘X-ETR’ with the value ‘Antispoofing allow rule set SCL –1’

        

        
      You can add as many IP addresses as you want to the rule, you can add other domains to protect, you can modify the text of the header to stamp to make it more descriptive, and you can even specify Exceptions to the rule. 

      The result is that email from these IP ranges sending email as your domain will bypass spam filtering and get delivered to your users’ inboxes except if the user has the sender on their personal Blocked Sender list, and there are not other ETRs running at higher priority than this rule that take action on the message (for example, a rule that sends email to the Admin Quarantine).

      If a message bypasses filtering due to an ETR like the above, you will see the following header inserted into the message:

      X-Forefront-Antispam-Report: SFV:SKN;…SCL:-1
      X-ETR: Antispoofing allow rule set SCL -1

      For more information on ETRs, see

      Managing Transport Rules.

      Once the Allow/Block functionality above is released, you don’t need to do this ETR and instead should use that.

      Remember – is important to never, ever allow only your domain in an ETR by itself with no other criteria. If you do, a spammer can spoof your domain and it will get a free pass to the inbox.

    Can antispoofing protection be turned off?

    No, antispoofing protection cannot be disabled.

    Instead, if you get false positives, you should use any of the workarounds above to bypass filtering for these senders. This ensures that rather than leaving you exposed to phishing messages, you instead block the most dangerous types of spoofs and allow in only authorized senders.

    EOP has done a great job at minimizing potential false positives and so this change will cause minimal disruption to legitimate mailflow.

    Is there anything else I should know about?

    Yes.

    1. In order to get full antispoofing protection, your domain’s MX record must point to EOP as its primary MX record. If it does not, antispoof protection will not be implemented for your domain.

    2. Antispoof protection does not override IP Allow rules, ETR allow rules, or end user safe senders, all of which bypass spam filtering. This may change in the future.

    Conclusion

    These changes to EOP’s spam filtering will help protect your organization from some of the most malicious phishing messages that we see today. The feature is intended to minimize any false positives while also giving you the ability override the filter decisions when necessary.

    As always, let us know how it works for you so we can continue to improve it!


    Footnotes

    [1] EOP supports DMARC for inbound email, which is a technology to stop spoofing of the From: domain. The main difference between DMARC and EOP’s Antispoofing solution is that DMARC requires certain DNS records to be published, whereas EOP’s Antispoofing solution does not. The second difference is that DMARC sends failure reports back to the forensic and aggregate reporting email addresses, whereas for EOP’s Antispoofing that information is only available via Powershell or through the Exchange Admin Center.

    For more information, see these blog posts:


    [2] For setting up 3rd party services to send on your behalf (as your domain) and have it authenticate using the regular antispam authentication methods:

    a) SPF Authentication

    If you are sending from an on-premise mail server, or a cloud-hosted service, and you have a dedicated IP address, you should add them to your SPF record. For example, before updating your SPF record, it might look like this:

    contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com –all”

    After adding the dedicated IP address (or CIDR address range), your SPF record might look like this:

    contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com ip4:1.2.3.4 –all”

    If you are using a larger service like a bulk email service provider or software-as-a-service provider, they may ask you to include all of their servers which is much more manageable:

    contoso.com IN TXT “v=spf1 include:spf.protection.outlook.com ip4:1.2.3.4 include:bulkEmailProvider.com –all”

    If you do this, your email that sends as you from this service will now authenticate with SPF.

    For more information:

    b) DKIM authentication

    Some bulk email service providers, or software-as-a-service providers, let you set up DKIM keys for email originating out of their service. This requires co-ordination between yourself and them to set up the necessary DNS records, and it is dependent on the organization. This is also a useful method for email authentication.
    The message might look like the following:

    Return-Path: <communication@bulkemailprovider.com>
    From: <sender@contoso.com>
    DKIM-Signature: s=s1024; d=contoso.com
    Subject: Here is a message from Bulk Email Provider’s infrastructure, but with a DKIM signature authorized by contoso.com

    In this example, Bulk Email Provider gave Contoso a public DKIM key to publish in their DNS record, and then signs with the corresponding private key when sending email and putting the resultant DKIM signature into the message. Third parties can then authenticate against the DKIM d= domain and compare it to the domain in the From: address.

    For more information:

    * * * * * *

    With either SPF or DKIM, when sending to EOP, a header is inserted into the message with the authentication results, and either the smtp.mailfrom or the header.d should match the domain in the header.from. In this case, contoso.com in the header.d in the DKIM-Signature header aligns with the domain in the From: address. In the absence of a DMARC record, EOP stamps dmarc=bestguesspass.

    Authentication-Results: spf=pass (sender IP is 1.2.3.4)
      smtp.mailfrom=bulkemailprovider.com; contoso.com; dkim=pass 
      (signature was verified) header.d=contoso.com; dmarc=bestguesspass 
      action=none header.from=contoso.com;

    A Powershell script to help you validate your DKIM config in Office 365

    $
    0
    0

    One of our support engineers (not me, so let’s give credit where credit is due) wrote a script to help you, as a customer of Office 365, validate for DKIM configuration once you have enabled it in the Admin Portal. We’ve added a few more checks to make it more clear, but you can also use this.

    To verify your DKIM configuration:

    1. Copy/paste the below script into a file, Validate-DkimConfig.ps1

    2. Connect to Exchange Online using Powershell, making sure that the directory you are in is the same as where you saved the script above.

    3. Type the following command in Powershell:

    . .\Validate-DkimConfig.ps1.

    4. To check the config for a single domain, run the following command:

    Validate-DkimConfig <domain>

    To show the full signing config, use the –showAll switch:

    Validate-DkimConfig <domain> –showAll

    To validate all domains for your organization, run the following command:

    Validate-DkimConfig

    You will be able to see if anything is wrong because the output is color coded.


    function Validate-DkimConfig
    {
        [cmdletbinding()]
        Param(
            [parameter(Mandatory=$false)]
            [string]$domain,
            [parameter(Mandatory=$false)]
            [switch]$showAll
        )

        if ($domain) {
            $config = Get-DkimSigningConfig -Identity $domain
            Validate-DkimConfigDomain $config -showAll:$showAll
        }
        else {
            $configs = Get-DkimSigningConfig
            foreach ($config in $configs) { Validate-DkimConfigDomain $config -showAll:$showAll}
        }

    }

    function Validate-DkimConfigDomain
    {
        [cmdletbinding()]
        Param(
            [parameter(Mandatory=$true)]
            $config,
            [parameter(Mandatory=$false)]
            [switch]$showAll
        )

        # Display the configuration
        $domain = $config.Domain;
        $onmicrosoft = if ($domain.EndsWith(“onmicrosoft.com”)) { $true } else { $false }
        $actions = @()

        Write-Host “Config for $domain Found…” -ForegroundColor Yellow
        if ($showAll) {
            $config | fl
        }
        else {
            $config | Select Identity, Enabled, Status, Selector1CNAME, Selector2CNAME, KeyCreationTime, LastChecked, RotateOnDate, SelectorBeforeRotateonDate, SelectorAfterRotateonDate | fl
        }

        if (!$config.Enabled) {
            Write-Host “Config $($config.Name) Not Enabled” -ForegroundColor Yellow
            Write-Host
            $actions += “Config $($config.Name) needs to be Enabled”
        }

        # Get the DNS ENtries
        Write-Host “Locating DNS Entries…” -ForegroundColor Yellow
        $cname1 = “selector1._domainkey.$($domain)”
        $cname2 = “selector2._domainkey.$($domain)”
        $txt1 = $config.Selector1CNAME;
        $txt2 = $config.Selector2CNAME;

        $cname1Dns = Resolve-DnsName -Name $cname1 -Type CNAME -ErrorAction SilentlyContinue
        $cname2Dns = Resolve-DnsName -Name $cname2 -Type CNAME -ErrorAction SilentlyContinue
        $txt1Dns = Resolve-DnsName -Name $txt1 -Type TXT -ErrorAction SilentlyContinue
        $txt2Dns = Resolve-DnsName -Name $txt2 -Type TXT -ErrorAction SilentlyContinue

        # Validate Entries
        Write-Host “Validating DNS Entries…” -ForegroundColor Yellow   

        Write-Host   
        Write-Host “Config CNAME1 : $($config.Selector1CNAME)”
        if (!$onmicrosoft) {
            if ($cname1Dns) {
                Write-Host “DNS    CNAME1 : $($cname1Dns.NameHost)”
                Write-Host “TXT Hostname  : $($cname1)” 
                $match = if ($config.Selector1CNAME.Trim() -eq $cname1Dns.NameHost.Trim()) { $true } else { $false }
                if ($match) {
                    write-host “Matched       : $($match)” -ForegroundColor Green
                } else {
                    write-host “Matched       : $($match)” -ForegroundColor Red
                    $actions += “Publish CNAME TXT Entry $($cname1) with value $($txt1)”
                }
            }
            else {
                write-host “DNS NotFound  : $($cname1)” -ForegroundColor Red
                $actions += “Publish DNS CNAME Entry $($cname1) with value $($txt1)”
            }             
        }

        Write-Host
        Write-Host “Config CNAME2 : $($config.Selector2CNAME)”
        if (!$onmicrosoft) {
            if ($cname2Dns) {
                Write-Host “DNS    CNAME2 : $($cname2Dns.NameHost)”
                Write-Host “TXT Hostname  : $($cname2)”
                $match = if ($config.Selector2CNAME.Trim() -eq $cname2Dns.NameHost.Trim()) { $true } else { $false }
                if ($match) {
                    write-host “Matched       : $($match)” -ForegroundColor Green
                } else {
                    write-host “Matched       : $($match)” -ForegroundColor Red
                    $actions += “Publish DNS CNAME Entry $($cname2) with value $($txt2)”
                }
            }
            else {
                write-host “DNS NotFound  : $($cname2)” -ForegroundColor Red
                $actions += “Publish DNS CNAME Entry $($cname2) with value $($txt2)”
            }       
        }

        Write-Host
        Write-Host “Config   TXT1 : $($config.Selector1PublicKey)”
        if ($txt1Dns) {
            $key = $txt1Dns.Strings[0].Trim()
            Write-Host “DNS      TXT1 : $($key)”
            $match = if (Compare-PublicAndConfigKeys $key $config.Selector1PublicKey) { $true } else { $false }
            if ($match) {
                write-host “Key Match     : $($match)” -ForegroundColor Green
            } else {
                write-host “Key Match     : $($match)” -ForegroundColor Red
                $actions += “Public Key in TXT Entry $($txt1) needs to be republished…”
            }
        }
        else {
            write-host “DNS NotFound  : $($txt1)” -ForegroundColor Red
            $actions += “Microsoft TXT Entry $($txt1) not found so Signing Config needs to be recreated…”
        }

        Write-Host
        Write-Host “Config   TXT2 : $($config.Selector2PublicKey)”
        if ($txt2Dns) {
            $key = $txt2Dns.Strings[0].Trim()
            Write-Host “DNS      TXT2 : $($key)”
            $match = if (Compare-PublicAndConfigKeys $key $config.Selector2PublicKey) { $true } else { $false }
            if ($match) {
                write-host “Key Match     : $($match)” -ForegroundColor Green
            } else {
                write-host “Key Match     : $($match)” -ForegroundColor Red
                $actions += “Public Key in TXT Entry $($txt2) needs to be republished…”
            }       
        }
        else {
            write-host “DNS NotFound  : $($txt2)” -ForegroundColor Red
            $actions += “Microsoft TXT Entry $($txt2) not found so Signing Config needs to be recreated…”
        }

        # Write out neccessary Actions
        Write-Host
        if ($actions.Count -gt 0) {
            Write-Host “Required Actions…” -ForegroundColor Yellow
            foreach ($action in $actions) { write-host $action}
        }
    }

    function Compare-PublicAndConfigKeys([string] $publicKey, [string] $configKey)
    {
        $match = $false;

        if (![string]::IsNullOrWhiteSpace($publicKey) -and ![string]::IsNullOrWhiteSpace($configKey))
        {    
            $regex = “p=(.*?);”
            $foundPublic = $publicKey -match $regex
            $publicValue = if ($foundPublic) { $matches[1] } else { $null }
            $foundConfig = $configKey -match $regex
            $configValue = if ($foundConfig) { $matches[1] } else { $null }

            if ($foundPublic -and $foundConfig)
            {
                if ($publicValue.Trim() -eq $configValue.Trim())
                {
                    $match = $true;
                }
            }
        }

        $match;
    }


    I hope you find this useful.

    Why study art?

    $
    0
    0

    I’ve got an unusual topic to discuss today – art history.

    You may be wondering “Um, why are you talking about art history on a blog about cybersecurity?”

    To which I reply “It’s my blog and I like writing about what I want to write about.”

    So, without further ado, let’s talk about art.

    * * * * * * * * * * * * * * * * *

    A few weeks ago, for some reason I don’t fully understand, I started learning about European art history. My entire life I had zero previous interest in learning about the history of art. Sure, I enjoyed looking at images and I knew what I liked, but I couldn’t have cared less about the story behind it.

    But then I read about a book about European history and art. I didn’t think I would care about the art part, but I did. In fact, I soaked it up like a sponge and I’ve been reading about it and watching YouTube videos (some on Khan Academy) a lot of the past few weeks.

    I’ve noticed how my opinions have changed, even over the past few weeks. I used to not understand modern abstract art, and I thought it was stupid. Now I still don’t understand it but, but I no longer think it’s stupid.

    So why study it at all?

    I study it because it teaches me to be a good observer. Why do I like what I like? what details did the artist put into the picture? And why did he or she do it? Before, I couldn’t have told you. And I still can’t… not entirely, but I’m better than before. But by observing and paying attention to the details in a painting, it trains me to observe and pay attention to the details in real life. In many of my features that I work on (such as automatic DKIM key rotation) I had to pay attention to the details in implementation. I do that for many of my features. One influences the other.

    Thus, I study art to get the context of a painting, as knowing the full context trains me to be a better observer.

    Below is a picture entitled The Third of May, a painting by Francisco Goya of Spain, completed in 1814.

    You can see the original here.

    Now, before when I was just looking at the painting, I’d say “Hmm, well that seems interesting” and gaze at it for 10-15 seconds and then move on. But no more. What do I notice now?

    (Before I continue, I am cheating. This is one of the paintings they analyze on Khan Academy so I am remembering this off the top of my head).

    First, it’s important to understand the context. During the Napoleonic era, France had invaded Spain. Some Spanish rebels led a resistance and the next day, the French military put down the revolt. This painting commemorates that, and it is the time period in which the painting takes place. It takes place on May 3, one day after the rebellion.

    The next thing I notice is the structure of the painting itself:

    • It’s painted with depth. Prior to the Renaissance, paintings were “flat”, that is, in two dimensional space. You can go here for an example (I’m not putting the picture here because I don’t want to divert attention away from the focus of the above picture). By contrast, in this painting, we can see the town way off in the background in three-dimensional space, the artist is using a technique to give depth that was popularized during the Renaissance.
    • The soldiers on the right are depicted in the shadows while the victims being executed on the left are in the light.
    • The victims on the left of the painting are also trapped in front of a hill. There’s no where for them to run.
    • The scene is depicted as taking place at night.

    Next up is the social commentary that the artist is making:

    • The perpetrators in this scene are shown without their faces visible, a common motif in art to depict aggressors, a lot like this.
    • The look of fear is evident on the man whose arms are in the air. However, the artist is depicting him not as a victim in retaliation for leading or participating in a rebellion, but as an innocent victim – either he wasn’t involved or he was involved but the cause he was leading was a just one.How do we know?

      First, the man is dressed in white which is a another motif of innocence that is common in art.

      But second and more important, the man has his arms up in a pose that you would see on the image of Christ that you would see on crucifix. The theme in the Christian story is that Christ was an innocent sacrifice who was unjustly executed by an oppressive regime.

      And third, to cement the fact that Goya is drawing an exact parallel between the innocent Spanish resistance and that of the innocent Christ, if you look close-up on the palms of the man with his hands in the air you can see “holes” in the palms of his hands. This is similar to the image of Christ with holes in the palms of his hands on the crucifix.

    Francisco_Goya_hands

    Thus, it is clear from the picture of the innocent man in the Christ-like pose that Goya is drawing a parallel between the Spanish resistance and the French putting it down, likening it to a justifiable movement where the ringleaders were sacrificed unfairly.

    * * * * * * * * * * * * * * * * * *

    All of that analysis wouldn’t have been possible without learning about art history, and now whenever I see a picture I try to look at it and do a quick analysis (actually, it goes pretty slow because I have to consciously analyze it).

    As you can see, this doesn’t exactly map to cybersecurity which is much more deterministic in some things, but paying attention to detail matters – maybe even more so. But the one thing that does map is understanding the context of a technology. Just as how I wouldn’t have been able to appreciate the painting above without knowing the historical background of when and why it was painted, it’s important to understand the context of why spam filters push features like SPF, DKIM, DMARC, antispoofing, threat intelligence, and so forth.

    Every antispam technology is nuanced, too. SPF is for direct-path communication, it doesn’t work well for forwarded messages. DKIM is good for verifying the sender of the message independent of the path it took, but it doesn’t work well by itself for determining whether or not the message is spam. DMARC is good for detecting spoofing, but is hard for many domain owners to set up. Threat intelligence pushes up the signals, and then we have to interpret what it all means.

    And so forth.

    It’s my hope that learning about art gives me – and you – a more well rounded point of view and that what you see is often just scratching the surface of what’s really there.

    Abstract art and the Final Ultimate Solution to the Spam Problem

    $
    0
    0

    I think I finally thought of something relevant to cybersecurity and my last post on why we should study art. This may be a stretch, but read on and tell me what you think.

    Every once in a while, a newbie to the world of fighting spam comes into one of the discussion lists or conferences where I regularly participate, enters the conversation and proposes a new method for ending spam for good. Whether it is a variant on making senders pay for email, to requiring some form of challenge/response filtering, to blocking email from all unknown senders, I’ve seen it all. Typically, the proposal comes from someone who is relatively new to the industry. They haven’t seen all the previously applied solutions that haven’t worked and so without the benefit of experience, they lack the ability to see the short comings of their solution.

    It is usually at this point that one of the more experienced members of the list sarcastically posts a link to https://craphound.com/spamsolutions.txt, a list of solutions to spam that tongue-in-cheek (but not really) explain why something won’t work:

    ——————————————-

    Your post advocates a

    ( ) technical ( ) legislative ( ) market-based ( ) vigilante

    approach to fighting spam. Your idea will not work. Here is why it won’t work. (One or more of the following may apply to your particular idea, and it may have other flaws which used to vary from state to state before a bad federal law was passed.)

    ( ) Spammers can easily use it to harvest email addresses
    ( ) Mailing lists and other legitimate email uses would be affected
    ( ) No one will be able to find the guy or collect the money
    <snip>

    Specifically, your plan fails to account for

    ( ) Laws expressly prohibiting it
    ( ) Lack of centrally controlling authority for email
    ( ) Open relays in foreign countries
    and the following philosophical objections may also apply:

    ( ) Ideas similar to yours are easy to come up with, yet none have ever
    been shown practical
    ( ) Any scheme based on opt-out is unacceptable
    ( ) SMTP headers should not be the subject of legislation
    <snip>

    ——————————————-

    I do feel some sympathy for the antispam newbie for what is basically a flame directed at them, but at the same, sometimes newbies have a lot of power in their respective organizations and end up implementing something that reduces spam while greatly reducing the usability of email. Email is great because it interoperates outside of a walled garden, that’s the part we want to keep. Most new solutions to spam decrease usability or something-else-or-other, as shown by that list.

    Yet never forget – while most people reading this blog are not newbies to fighting spam, at one point all of us were. All of us non-newbies suffer from a problem called the curse of knowledge. It makes it difficult for us to put ourselves in the shoes of another, we can’t see it from their point of view. While they don’t know what they don’t know, we can’t remember what’s it like to not know.

    So let me give you an example of what it’s like to not know: Abstract art.

    If you don’t know what I mean, here’s a couple of examples. The first one is by the American artist Jackson Pollock painting around the middle of the 20th century:

    A painting by 20th century American artist Jackson Pollock

    Now, if you’re anything like me a few weeks ago, you’d probably look at the first painting and say to yourself “What’s that supposed to be? Is it some sort of those ‘magic eye’ and we’re supposed to find the hidden message? It looks like a bunch of random splatters of paint on the canvas. How is that art?”

    This one, White on White, is by Kazimir Malevich, a Russian artist who completed it in 1918:

    White on white, by Kazimir Malevich

    Once again, if you’re anything like me – without any background in art history – you would say “Well, I guess it’s a couple of white rectangles with one at an angle. So what? I don’t see the aesthetic appeal in it. I mean, it looks okay. But nothing special. I much prefer things that look like actual things.”

    If you said anything like that to either of the two pictures, that’s exactly what I would have thought. As I said in my other post, I used to not understand abstract art, and I thought it was stupid. Now, I still don’t quite understand it, but I no longer think it’s stupid.

    Why do I think that?

    Because now that I have studied art, I realize that art doesn’t exist in a vacuum, it’s relative to the period of time that came before it – at least in European art. Successive periods in art are defined by what came before it, and as a counter-reaction to what came before, and what’s going on in society at the time.

    So let’s go back to medieval art. Below is a picture that was common in the middle ages. Back in the day, nearly all art was produced because a wealthy patron wanted it. Most medieval art was two-dimensional on a flat surface, like below, and at least in western Europe it usually told passages from the bible because in those times, religion played a major role in people’s lives and most people were illiterate. Art was a tool for learning for people who couldn’t read.

    You can see in the below that the picture is not “realistic” compared to what comes later (i.e., later in this post).

    The Renaissance is where we see a revolution in the way scenes are depicted (the Renaissance is a term that refers to the rediscovery of classical times, that is, Greece and Rome). I don’t have time to go into it (mostly because even I don’t know all the techniques) but you’ll see how everything now exists in three dimensions.

    In Raphael’s The School of Athens, notice how the scene is depicted as more realistic. Figures in the foreground are interpreted as being closer to the viewer while people in the background are interpreted as being further away:

    The School of Athens, by Raphael

    Even though we take it for granted, the big shift you’re seeing is one of perspective. Renaissance artists would use a ‘vanishing point’ off in the distance and then draw people along it, moving them up or down to create depth. For example, look at my little stick figure diagram below. Which characters are closes to the audience? Which ones are further away? The pixels are the same distance away from your eyes but you can “clearly” see that the ones that are bigger are closer while the ones that are smaller are further away. Furthermore, from what angle are you, the viewer, supposed to see the picture from? You’re not center on, but looking at it while you are standing on the right with your eyes looking to the left:

    Simple_perspective

    To see what a shift this is, just go and do a simple web search for medieval art and then for Renaissance art. This perspective shift that I talk about above for medieval art (composed during that time, not after) is either missing or minimized.

    The next major art movement is Baroque Art, and it is a counter-reaction to the Reformation. It is characterized by much more emotion in the picture (from its human subjects depicted in it), as well as movement. Baroque art has other characteristics, but I don’t have time to get into them.

    The below image shows the story of Daniel in the lion’s den (from the book of Daniel in the Old Testament). Look at the raw emotion on Daniel’s face, he is clearly (?) in fear, or awe, or something or other. Whatever it is, he doesn’t have a straight face.

    Another characteristic of Baroque art is its lack of clearly-defined lines. Take a look at the background behind Daniel and among the lions. There are shadows everywhere and it tends to blur together the rocks, and even the lions in the back are less clearly defined. You don’t see that so much during the Renaissance. The term Baroque comes from a translation of a Portuguese term meaning ‘misshapen pearl’, and it wasn’t a term of endearment. But gradually people came to accept it.

    Skipping ahead many decades and a couple of smaller movements and we get to the Age of Enlightenment, and the Neoclassical Era. This is the time period where we find many of the great writers that the founders of the United States borrowed heavily from – John Locke (natural rights), Baron de Montesque (separation of powers), the need for a strong government (Thomas Hobbes). The Age of Enlightenment and the Neoclassical period was ushered in by the scientific revolution and a callback to the rational times of Greece and Rome, after the Renaissance did it the first time. But the neoclassical era’s art was focused on science and rationality.

    The Oath of Horatti by Jacques-Louis David is one of the most famous paintings in neoclassical art. It’s a throw-back to Roman times and the general spirit of the times is rational thought. This is the period of time where writers we associate with the Age of Reason – Voltaire, Beccudia (prohibition on cruel and unusual punishment) – were most active; the American Constitution borrows heavily from this period. The art also reflects this. This is also the period where the Industrial Revolution really took off in full force in Europe. England was first, and later on followed by France, Italy, and Germany (though not necessarily in that order).

    We in the west like to pride ourselves on how we use Reason, but the Age of Reason came to and end with a very strong counter-movement called the Romantic period. Romanticism is still popular today and we can see it everywhere, especially in art. The Romantics rebelled against the stale lack of taste characterized by the Industrial Revolution. Think about the movie Star Wars – the Empire is cold, mechanical, and emotionless and they are the Industrial side. Meanwhile, the rebel alliance is driven by ideals and they represent the Romantic side.

    Romantic art tends to minimize its human subjects and greatly play up ideals, emotion, and nature. The Industrial Revolution (and neoclassical era that accompanied it) tended to create new technology but degrade the countryside. Consequently, the Romantics sought to idealize nature.

    The Sea of Fog is one of the Romantic period’s more well-known paintings. It shows a person in the middle of the picture gazing out, but the person is not the focal point of the picture. Instead, he is gazing out to the true focal point – raw, untamed nature.

    Sea of Fog, by Caspar Friedrich

    This is a recurring theme in Romantic art, but it was a pretty big movement and nature was not the only characteristic of that time period. I’d say maybe 1/3 of the time period contained nature. It was a lot but by no means everything. Above all, Romanticism idealized emotion over reason. For all the progress the Age of Reason and the Enlightenment made, it was rejected strongly by the artists and society eventually followed.

    Whew.

    I go through all of these art history movements (more quickly than they deserve) to show you how art has evolved. Each movement is a reaction to its preceding movement, and they reflect what’s going on in society at the time as well. Art makes much more sense, or at least is more fun to appreciate, when you understand the context in which it was created.

    And one big thing that happened in the middle of the 19th century was the invention of photography. This was a turning point in art history. Whereas before artists used to create paintings that were realistic, after photography came along, artists knew that they could no longer compete with photographs. Realistic paintings were out, and using color to convey emotion was in. Artists decided to use color and created new art styles to convey the emotion of contemporary society, or of themselves.

    This shouldn’t come as a surprise. Even today, our artists use color to depict emotion:

    Leaving the present and coming back to the 19th century, two big movements in using color to depict emotion were Impressionism (which depicted an ‘impression’ of the actual image):


    … and Post-Impressionism (I don’t yet really know what the difference is, but I do know that Vincent Van Gogh was part of this era).

    You still know what’s there in the picture, but the artist has moved away from a realistic view of the world and is instead using color and painting technique to depict something else about the picture. “If you want realism, take a photograph” says the artist.

    20th century art movements continue this move away from realistic depictions of the subjects. Expressionism, Dada, and Cubism are all parts of this counter-movement that was itself a reaction to technology. The White-on-White painting above was painted during World War 1, a time of massive social upheaval. Russia was getting clobbered during the War, and then they experienced the Bolshevik Revolution. In response, the artist painted the picture of two white boxes. They are not the same shade of white, they are off white. They also don’t have clearly defined edges but instead are fuzzy edges (my term). The picture is not necessarily supposed to be something, but instead could be the artist’s reflection on the disjointedness and instability of Russian society during that time.

    I think.

    That’s the problem with abstract art, I don’t know if I’m reading into it something that’s not there or if I am getting it right. But maybe that’s the point?

    * * * *  * * * * * * * * * * * * * * * * * * * * * * * *

    Okay, so what does this have to do with FUSSP?

    Now that you’ve made it to the end of this post, you’ve seen a lot of different movements in art and how it has evolved. It used to be realistic but now it isn’t. And it isn’t realistic on purpose and now you know why it isn’t, and it actually make sense… from a certain point of view. So rather than thinking about abstract art as some weird thing that crazy artists do, you know the history behind it. Or at least what led up to it.

    But just think about this crash course – when you started reading this post you (probably) had no idea of what abstract art was all about. It’s not your fault, you just didn’t know any better. Think about the state you were at that time – you didn’t know what you didn’t know. You were completely unaware why those paintings were the way they were and you felt perfectly justified in holding the opinions that you formerly had.

    That feeling is the feeling newbies have when they first learn about spam and then the propose the FUSSP. You may not be able to empathize with them because of the curse of knowledge, but just think about what art history buffs thought about you when you dismissively said “Abstract art makes no sense.” But armed with some understanding, you can now look at things with a different perspective. Newbies can as well, it will just take longer because there is so much to learn when it comes with cybersecurity.

    And now that you’ve been enlightened, you can’t go back.

    Can TLS make the user experience better?

    $
    0
    0

    This blog post reflects only my opinion about encryption and the protection of sensitive personal information. It is not reflective of anyone else’s views and is only how I interpret the state of the industry. And it may be wrong. Read on at your own discretion.

    The other day, I was talking to a financial institution and they sent me an email, but it was one of those “secure” emails. If you haven’t seen them before, they send you an email to your regular email account. You then have to click the link, login to the site and possibly create a new username/password, and then the service exchanges messages with you that way.

    So here’s the flow, it starts with me getting the following message in my inbox:

    Secure_email_off_site

     

    I click the link, sign up for a new account, reply in that portal, etc. When the bank replies, I get an email notification saying that I got a reply and then I have to login again. Every communication goes down this route.

    This is quite common in the industry. Email is not considered secure, so banks and health care industry do not exchange sensitive data with you over normal channels. Instead, you have to use a secure messaging app which forces you, as a user, to go offsite and create a new account. This is common in the email security industry, and I think it has been pointed out as a drawback to the overall user experience. I know that the above was inconvenient for me. Some applications will send you an encrypted blob attachment but you still need to sign into a portal to read it.

    If I send my bank an email, they don’t email me back directly with a response. Instead, they send me an email saying “Please login to the website and check your feedback there.” That’s where they already send all my statements and tax forms. When I login to my bank and check the private messaging portal, frequently they don’t say anything there either. Instead, they send me a letter in the postal mail.

    I can understand why banks and health care companies don’t send email. It can be an insecure platform; banks and health care are highly regulated and they don’t want anyone snooping on anything in transit.

    So I was wondering if it would be possible to instead rely upon TLS as the solution? TLS encrypts something in transit and it is easy to figure out if a message was received over TLS. Here’s a Twitter notification sent to me, Gmail lets me know with a red lock in the rendering pane that it wasn’t sent over an encrypted connection.

    Not_sent_over_TLS_in_Gmail

     

    Okay, so that’s on the receiving side. What about the sending side? Gmail tells me that too, they keep track of who can receive email over TLS and bubble up that information at compose time:

    Cannot_send_over_TLS_in_gmail

     

    So, in theory, we can see it’s possible to detect if a recipient can receive over email (at least Google can do it). But it’s not just Google, either. The IETF recently published a draft called SMTP Strict Transport Security which proposes a standard to indicate whether or not the domain can receive email over TLS and whether or not it can be enforced. Thus, in theory, that could be used for a sender to decide how to send a message to a user. If the user can receive over TLS (by checking the necessary records in DNS), then send over regular email. If they can’t (or we don’t know if they can), then send using secure email apps or postal mail. This way, at least some of the user base will have the best experience.

    So would that work?

    No.

    Why not?

    Remember that banks and health care companies are highly regulated to protect highly sensitive data. When a bank or health care company sends you a message using a secure messaging application, the platform they are using (i.e., the corporation providing the messaging service) has entered into an agreement with the bank or health care company to similarly protect that data. Thus, the secure messaging app has said “I understand that you are highly regulated and I agree to also protect this data, meeting certain requirements.” The company owning the app says that, or the bank builds their own web platform and control it all themselves.

    Services like outlook.com, Office 365, and Gmail have not entered into that same agreement with the health care companies. Yes, they can send and receive data securely, but they have not agreed to the same restrictions around protecting that particular user’s data while the data is at rest. You may argue that these services do a good job of encrypting data at rest (maybe) but the fact remains that email providers are not under the same constraints. So while a user may want to view data using their regular email account, the bank doesn’t have an agreement with Microsoft or Google or whoever to let them manage that data indefinitely.

    So much for that.

    So while I would like a better user experience, it doesn’t look like it will happen if we just rely upon TLS.

     

     

     

    What Gmail’s changes in their web interface means to you as a customer of Office 365

    $
    0
    0

    A few weeks ago, Gmail made several changes to better reflect the security status of messages sending to its service. I am a user of Gmail and I appreciate what they are doing. If you’re a customer of Office 365, what does this mean for you?

    1. If you send email to Gmail and you don’t have SPF records nor sign with DKIM, Gmail puts a large gray question mark next to the sender, for example:
      .
      unauthenticated_sender
      .
      This is part of Google’s emerging No Auth, No Entry story. They are not denying entry, but they are degrading the experience to indicate that the sender is unknown. The sender is not necessarily spamming, but you as a user should be cautious. As an Office 365 customer, you are in control of your own DNS records and therefore responsible for setting up SPF records correctly.
      .
      Setting up SPF records in Office 365
      .
      However, make sure you watch out for some common errors, you may think you are publishing the correct SPF record but you may hit a common trap. If so, Gmail may not recognize it and you’ll get a gray question mark.Common errors in SPF records
      .
      For DKIM, Office 365 will automatically provision your DKIM signing for you and sign with the default settings, so you should never see that gray question mark in Gmail (unless you route your email through another server before sending to the Internet, and that server breaks our DKIM signature). Gmail requires that DKIM keys be at least 1024 bits; don’t worry, we know that and take care of that for you.

      For the best experience sending to Gmail [1] for how a message will render (see #2 below), you’ll want to enable DKIM with custom signatures (DKIM selectors) in Office 365. This requires publishing two records in your DNS zone for your domain.
      .

    2. If you send to Gmail but your From: address’s domain does not align with either the domain in the DKIM d= signature or domain that passes SPF, Gmail puts a ‘via’ in the From: address. Gmail also has a hierarchy of which one it prefers – if it is signed by DKIM, they show the d= signing domain. If it doesn’t exist, they show the SPF-validated domain. If neither pass, they show the gray question mark per above.This means that if you have SPF set up in Exchange Online Protection (EOP) with the default DKIM settings enabled for your domain, Gmail will show the ‘via’ tag in the UX:
      .
      sender_with_via
      .
      If you don’t want that to show up in Gmail, you should manually enable DKIM in Office 365, as per above. When you do, your message will look like the following:
      .
      sender_without_via
      .
    3. A couple months ago, Google announced that messages that weren’t sent over TLS would receive a red lock in the UX, and this past week they announced that they saw a 25% reduction in email not sent over TLS. Here’s what a message would look like:
      .
      unauthenticated_sender_without_TLS
      .
      Office 365 sends over opportunistic TLS by default, so any outbound message sent from EOP to Gmail will not see that red icon.

     

    Those are the big things to be aware of when sending email to Gmail out of Office 365.

    Hope this helps.


    [1] While Gmail strongly pushes DKIM for delivery, EOP also uses it for tracking our outbound reputation so we can stay off of others’ IP reputation lists. EOP also requires SPF or DKIM when receiving email over IPv6, so automatically enabling your domains for DKIM means that you can send to any of our IPv6-enabled customers even if you have misconfigured SPF and haven’t set up DKIM. Other receivers around the Internet similarly require SPF or DKIM when receiving email over IPv6.


    Understanding Safety Tips in Office 365

    $
    0
    0

    Exchange Online Protection (EOP) already protects you with industry-leading spam and malware prevention. However, these attacks are so well crafted that they look legitimate. Sometimes putting messages into the Junk Email folder isn’t enough. EOP will automatically verify the sender and add a Safety Tip within an email message to warn you about potentially harmful messages when you check your email in Outlook on the web [1]. You don’t need to do anything to enable this, we do it for you automatically (you’ll be in either the preview customer list and get it immediately, or if not in the preview list then a few weeks from now):

    • A red Safety Tip in an email means that the message you received contains something suspicious. We recommend deleting these types of email messages from your inbox without opening them.
      .
      Suspicious_Phish
      Suspicious_Fraud
      .
    • A yellow Safety Tip in an email means that the message has been marked as spam. If you don’t recognize and trust the sender of the message, don’t download any attachments or pictures and don’t click any links in the message. You can click the “It’s not spam” link in the yellow bar of a junk mail item to move the message to your inbox.
      .
      Unknown_Spam
      .
      We’ll also show the yellow Safety Tip when the message is in your Inbox even though it is spam, but it’s there because you’ve disabled moving spam to your Junk Email folder.

    In addition to unsafe messages, we’ll also tell you about valid messages from senders we trust:

    • A green Safety Tip in an email means that we checked the sender of the message and verified that it is safe. These senders, which are maintained by Microsoft, include financial organizations and others that are frequently spoofed.
      .
      Trusted_trusted
      .

    We’ll also tell you when we skipped filtering from senders you trust:

    • A gray Safety Tip means that the message was not filtered for spam, such as when your organization creates an Exchange Transport Rule to bypass filtering, or when the user puts the sender on their Safe Senders list.
      .
      Gray_ETR_nonspam
      .
      The gray Safety Tip also shows up when external images are blocked, that is, the message is in your Inbox and we don’t think it’s spam, but we don’t download the images unless you choose to download them. [2]

    Most messages in your inbox will have no Safety Tip when checking your email in Outlook on the web, we only add them when we have information we think you need.

    If you disagree with how we marked a message (that is, it’s not spam or it’s not legitimate), you can report them to us for analysis as described here: Report junk email and phishing scams in Outlook on the web. We take these samples and use them to make your experience better.

    Also, Safety Tips work best if you conform to our Mail flow best practices for Exchange Online.

    Safety Tips are an important tool in combating phishing scams and online fraud. We’ll continue to add more features to Safety Tips to ensure you have the best experience. As always, please let us know what you think. 


    [1] Outlook on the web was formerly known as Outlook Web Access, or OWA. You can use Outlook on the web to check your email if you are an Office 365 customer. Safety Tips have existed in Outlook.com/Hotmail for years, and now we are ensuring you get the same experience in both web portals.

    [2] The red, yellow, and green Safety Tips between Outlook.com and Outlook on the web (Office 365) are similar with a few minor differences. However, Office 365 makes much greater use of the gray safety bar compared to Outlook.com.

    Outlook.com DKIM signing done, now on to hotmail.com

    $
    0
    0

    A couple of months ago, I wrote a blog post that we were starting to roll out DKIM signing for our consumer email accounts sending from @outlook.com. These are for accounts that have been migrated from the old Hotmail/outlook.com infrastructure and onto our new Exchange Online infrastructure. Not all accounts have been migrated yet, so if you are an @outlook.com user and don’t see DKIM signatures on your messages, it’s probably because your account hasn’t been migrated yet. DKIM signatures are also sent to external recipients, that is, outlook.com-to-outlook.com messages are not signed (we have other ways of authenticating for this type of direct traffic).

    We finished outlook.com last week.

    Next up is signing DKIM for migrated @hotmail.com users. Hopefully that goes a little faster.

    Taking the hassle out of email authentication

    Seven things to know about Safety Tips

    $
    0
    0

    As I posted on this blog a couple of months ago, and as we posted on the Office blog last month, Office 365 is going to be releasing Safety Tips over the next few weeks. In this blog post, I go into more about how they work.

    1. Outlook on the web has more Safety Tips than regular Outlook

    When you check your email using Outlook on the web (formerly known as Outlook Web Access, or OWA), there are four tips:

    Trusted_trusted

    Suspicious_Phish

    Unknown_Spam

    Gray_ETR_nonspam

    However, when checking your email in Outlook, you’ll only see the red (Suspicious) Safety Tip:

    Suspicious_Phish

    We have plans to also roll out the yellow Safety Tip a few weeks from now.
     
    2. It doesn’t matter which version of Outlook you’re using, you’ll get Safety Tips automatically

    Safety Tips in Outlook doesn’t depend on what version of Outlook you’re using because we crack open and insert the Safety Tip directly into the message body. That means that whatever email client you’re using will show the Safety Tip. It’s done at the email filter level and not rendered at the mail client level.

    And that means that not only does it show up in any version of Outlook, it shows up in any email client. But for the purposes of this blog post, I will call them as checking safety tips in Outlook.
     
    3. Think of Safety Tips in Outlook as “Safety Tattoos”

    We think of safety tips, when checking them in Outlook, as safety “tattoos” because they stick with your message forever (almost). If you drag them from the Junk folder to the Inbox, or vice versa, the color of the tip doesn’t change.

    The only way to delete the tip is to open up the message, highlight it, and delete it.

    In this blog post, Safety Tattoos means only the version that we insert inline into the message. Safety Tips refers to both Safety Tattoos and Safety Tips in Outlook on the web, I use it as a general name for the entire feature.
     
    4. The tattoo doesn’t show up in Outlook on the web

    You may be thinking “If you’re inserting the safety tip directly into the message, and you’re rendering it in Outlook on the web, does this mean that if I check it in Outlook on the web that I’ll see it twice?”

    No.

    Outlook on the web is smart enough to suppress the safety tattoo and instead show a rendered version of it that you can interact with. For example, the Outlook on the web one may say “Click here to enable blocked content.” The safety tattoo does not because it does not have access to that part of the email client’s rendering capabilities.
     
    5. As an administrator, you can disable Safety Tattoos

    The Exchange Admin Center lets you turn on or off Safety Tattoos, they will be on by default for both new and existing customers. We don’t recommend it, but you can disable them by logging in and changing that setting.

    You can’t disable them in Outlook on the web, however. They also show up.

     

    6. Safety Tips work best if your MX record points to Exchange Online Protection (EOP)

    Not all customers point their MX records at EOP. Many point at a 3rd party service and then relay the email onto us. If you do that, Safety Tips won’t work as well; they may be suppressed entirely. This robs your users of the additional safety experience of visual clues that something is wrong with the message.
     
    7. The red (Suspicious) Safety Tip’s criteria is continually expanding

    The red safety tip has lots of different criteria that we are rolling out. Some of it is the same as in outlook.com, but much of it is unique to EOP. As we find more criteria that we think is suspicious — phishing — we’ll add it to the tip. We don’t plan to document every scenario for which we assign a red tip.

     


    Those are the items I can currently think of. As we get more, I’ll add them. Not sure about what to do with the title of the blog post when that happens, though.

    Why does my email from Facebook, that I forward from my outlook.com account, get rejected?

    $
    0
    0


    Why is my (your) email bouncing when I (you) forward it?

    Recently, many people have been asking me why their email from Facebook, that they forward from their outlook.com or Hotmail account to another account, bounces after they forward it? That is:

    Facebook -> outlook.com (forward) -> Hotmail/Yahoo/Gmail -> bounces back

    Why does this happen? How can you fix it?
     
    It’s because of the way we are migrating accounts, plus some older behavior designed to protect the mailbox

    A few months ago we announced that Hotmail/outlook.com accounts were being migrated to Exchange Online (EOP). This is not an instantaneous process, it takes a long time to move all of those accounts over and we are not complete. But, this issue of bounced email that is forwarded only happens if your account has been migrated. A migrated account first goes through the old Hotmail infrastructure and then lands in the Exchange Online environment where it re-uses the spam filter verdict that the old Hotmail infrastructure stamped .

    Exchange server has a feature wherein it “fixes up” content in a message. This has been around for many years and it’s to prevent malformatted data from going into your mailbox where it could cause a corruption problem. So, if messages arrive in a certain way, it converts it to a format it does expect. For example:

    Before being fixed: Joe Sender <joesender@example.com>

    After being fixed: Joe Sender <joesender@example.com>

    There are good reasons for doing this, especially in enterprise environments, that I won’t get into here. To a human, the message looks identical. But the problem with doing this is that if a message has a DKIM signature, then fixing up the message will break the DKIM signature – even doing something as small as adding quotes around the Display Name breaks the existing DKIM-Signature.

    Normally this isn’t a problem because Exchange Online verifies SPF, DKIM, and DMARC and stamps the results in the Authentication-Results header [1]. But if the message is forwarded to another mailbox who reverifies the message, it won’t pass SPF (because the IP address of the forwarder won’t be in the original sender’s SPF record [2]), it won’t pass DKIM because the message headers and body have changed slightly and the hash doesn’t verify, and therefore it can’t pass DMARC since the From: domain won’t align with either the domain that passed DKIM or SPF.
     
    Non-migrated Hotmail/outlook.com account

    2015_05_19_Hotmail_nonmigrated_and_forwarded_intact

    Migrated Hotmail/outlook.com account

    2015_05_19_Hotmail_migrated_and_forwarded_broken

    This wouldn’t normally be a problem except when it comes from domains that publish a DMARC record with p=reject or p=quarantine. Most email doesn’t but there are a few large senders that do including Facebook (facebookmail.com), LinkedIn, Yahoo, AOL, and soon-to-be Gmail. Since they don’t align with SPF/DKIM/DMARC, messages from those domains will bounce if you forward them because they fail DMARC.

    :(
     
    So what can I (you) do?

    The good news is that we are fixing this – that is, we are fixing the “fixing up of messages.” We are introducing a change in Exchange Online that will stop modifying the contents of messages when they are forwarded out of the system. SPF will still fail, but DKIM will pass. This means that you’ll be able to forward email to your heart’s content and it will pass DMARC just fine.

    I don’t have a timeline on this because we’ve tested this before and had to pull it back because we found problems. But hopefully over the next few weeks this fix will go out worldwide and then this will no longer be a problem.

    We ask that you sit tight until then.

    Thanks.

     


    [1] For a migrated mailbox, Hotmail/outlook.com does its authentication checks and stamps the result in the Authentication-Results header. When the message is sent to EOP, it does its own authentication checks and stamps the result, but they are not used. Instead, Hotmail/outlook.com pushes its authentication checks into its spam/non-spam verdict which EOP re-uses.

    [2] Some systems rewrite the 5321 MailFrom so that it does pass SPF, e.g.,

    Original SMTP MAIL FROM: <sender@example.com>
    Original SMTP RCPT TO: <receiver@contoso.com>

    Forwarded SMTP MAIL FROM: <some_hash_receiver_contoso-com@contoso.com>
    Forwarded SMTP RCPT TO: <receiver@something.com>

    This does pass SPF at the forwarded-to receiver, but since the domain in the SMTP MAIL FROM does not match the From: address – which is not rewritten – it still fails DMARC.

    Viewing all 243 articles
    Browse latest View live


    <script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>