OS X Server: Microsoft Outlook clients might not be able to send mail due to spam filter settings
Microsoft Outlook users might not be able to connect to an email server on OS X Server if your spam filter is configured in a certain way.
Users who use Microsoft Outlook as their email client and who connect to an email server hosted from Mac OS X Server v10.6 or later might not be able to send mail if the spam filter is configured in a certain way. The following (or similar) alert is returned to Outlook clients:
The message could not be sent because one of the recipients was rejected by the server. The rejected email address was "recipient@example.com" is subject "example", account: "mail.example.com" , server "mail.example.com", protocol: SMTP, server response: "504 5.5.2 < hostname > : Helo command rejected: need fully qualified host name", port: 25, secure (SSL): no, server error: 504, error number: 0x800CCC79.
This issue happens when the OS X Server Mail service (postfix) is configured to require a fully qualified hostname from SMTP clients. This setting is enabled when you turn on spam filtering in Mac OS X Server v10.6 or later.
The fully qualified hostname requirement can be configured separately from other spam filter settings by editing a configuration file. Any modification of a security-related setting should be evaluated prior to making the change.
Once you have evaluated the change, you can use the following steps to implement it. Use these guidelines when editing a configuration file.
For OS X Server (Mountain Lion and later)
In /Library/Server/Mail/Config/postfix/main.cf, locate the smtpd_helo_restrictions setting.
Remove "reject_non_fqdn_helo_hostname" from the list of settings.
Restart the Mail service.
For OS X Lion Server and Mac OS X Server v10.6.x
In /etc/postfix/main.cf, locate the smtpd_helo_restrictions setting.
Remove "reject_non_fqdn_helo_hostname" from the list of settings.
Restart the Mail service.