AMaViS: Filtering spam and viruses

Two annoying issues when using email are spam and viruses. Fortunately you can fight both using AMaViS (A Mail Virus Scanner). AMaViS is an interface between Postfix, SpamAssassin (famous for its Bayesian spam filtering capabilities) and optionally a virus scanner. AMaViS contains the spam filtering part but has no virus scanner built in. You have installed ClamAV for that purpose. It is a high quality and free virus scanner that gets updated frequently.

The AMaViS configuration is spread across several files in the /etc/amavis/conf.d directory. Fortunately the virus scanner "ClamAV" is already configured by default, though it will need enabling in the file /etc/amavis/conf.d/15-content_filter_mode by removing the '#' from the @bypass_... lines so that spam and virus filtering gets enabled. (It's confusing that enabling the bypass option actually enables spam and virus scanning.)

I suggest you also take a closer look at /etc/amavis/conf.d/20-debian_defaults. Don't change this file though. If you want to change settings then put them into the /etc/amavis/conf.d/50-user file.

  • $sa_spam_subject_tag: if this line stays uncommented then every email that AMaViS suspects to be spam get this string inserted into the email's subject. If you do not want to alter the subject you should set "$sa_spam_subject_tag=undef". Users can still find out if AMaViS flagged the email as spam by checking the X-Spam-Status header.
  • $sa_tag_level_deflt: spam with a score that is greater or equal to this level will get spam headers added. Debugging is easier if you always add the headers so you should consider setting it to some very low score (e.g. "undef" - which is the lowest possible level - so that headers are always added.
  • $sa_tag2_level_deflt: emails with a spam score greater or equal to this level will be marked as spam. Usually this default is reasonable. Lower it if too much spam gets through. Raise it if you get too many regular mails caught as spam.
  • $sa_kill_level_deflt: should be set to the same value as $sa_tag2_level_deflt
  • $final_spam_destiny: the default D_BOUNCE setting is plain stupid here. The usual approach is to tag email as spam. But if you bounced them you would hit some innocent person because spam mails never contain the correct sender address. Just let the user decide what to do with spam. So it is strongly recommended that you set this to D_PASS.
  • $spam_quarantine_to: where to send quarantined spam email to. As spam emails are just tagged and then passed through they should not be quarantined. So the recommended setting here is undef.
  • $banned_filename_re: carefully check this list because these patterns tell AMaViS when to bounce an email because it contains data that you do not like to receive in an email

So a sensible "50-user" file would be:

$sa_spam_subject_tag = undef;
$spam_quarantine_to  = undef;
$sa_tag_level_deflt  = undef;
$final_spam_destiny  = D_PASS;
1;  # ensure a defined return

Restart AMaViS if you have made changes to the config files:

$> /etc/init.d/amavis restart

Make sure that AMaViS is listening on TCP port 10024:

$> netstat -nap | grep 10024

You should get this output:

tcp  0   0 127.0.0.1:10024     0.0.0.0:*    LISTEN   12345/amavisd

If you get such a line then AMaViS is running and waiting for incoming SMTP sessions. Otherwise check your /var/log/mail.log file - perhaps you have made a mistake in the configuration files.

A few words on how AMaViS will be plugged into Postfix. If a person sends you an email from the internet it will be received by Postfix (the smtpd process) on TCP port 25 (SMTP). If Postfix accepts the email it will be forwarded to AMaViS on TCP port 10024 (SMTP). And if AMaViS is happy with the email's contents it will be sent back to Postfix on TCP port 10025 (SMTP). Postfix finally delivers the email to the actual recipient. See the chapter "The Big Picture" for an illustration.

Making Postfix forward emails to AMaViS is done by setting the content_filter setting. Also set the "receive_override_options" setting that will be explained later:

$> postconf -e content_filter=smtp-amavis:[127.0.0.1]:10024
$> postconf -e receive_override_options=no_address_mappings

We need to define the smtp-amavis service first in the /etc/postfix/master.cf. And we also need Postfix to listen on TCP port 10025 for emails that get sent back from AMaViS. So add these sections to your /etc/postfix/master.cf:

smtp-amavis unix -      -       n     -       2  smtp
    -o smtp_data_done_timeout=1200
    -o smtp_send_xforward_command=yes
    -o disable_dns_lookups=yes
    -o max_use=20

127.0.0.1:10025 inet n  -       -     -       -  smtpd
    -o content_filter=
    -o local_recipient_maps=
    -o relay_recipient_maps=
    -o smtpd_restriction_classes=
    -o smtpd_delay_reject=no
    -o smtpd_client_restrictions=permit_mynetworks,reject
    -o smtpd_helo_restrictions=
    -o smtpd_sender_restrictions=
    -o smtpd_recipient_restrictions=permit_mynetworks,reject
    -o smtpd_data_restrictions=reject_unauth_pipelining
    -o smtpd_end_of_data_restrictions=
    -o mynetworks=127.0.0.0/8
    -o smtpd_error_sleep_time=0
    -o smtpd_soft_error_limit=1001
    -o smtpd_hard_error_limit=1000
    -o smtpd_client_connection_count_limit=0
    -o smtpd_client_connection_rate_limit=0
    -o receive_override_options=no_header_body_checks,no_unknown_recipient_checks
    -o local_header_rewrite_clients=

Restart Postfix first:

$> postfix reload

Now two settings here deserve an explanation. First the receive_override_options are set to no_address_mappings. This disables all address mappings. Your virtual aliases for example are not considered at first. Then the email is sent to smtp-amavis and in the end returned to the 127.0.0.1:10025 service that sets a lot of options. One of those options is the receive_override_options again. But this time the no_address_mappings setting is left out. So at this stage Postfix finally checks your virtual aliases. Sounds complicated? Well, it has to be done like this or otherwise your aliases would be evaluated twice which leads to mails sent twice. The other options are used to disable checks that Postfix has already done during the first stage.

Note: receive_override_options=no_address_mappings makes Postfix not consider aliases any more. So if you wish to disable AMaViS as a content filter then you must disable this parameter, too!

Another tiny caveat is that the user "clamav" must be a member of the system group "amavis" so that the two services are allowed to talk to each other:

$> adduser clamav amavis
$> /etc/init.d/clamav-daemon restart

And another issue to take care of: AMaViS tries to find out whether a certain email is incoming (sent from the internet to your domains) or outgoing (sent from your system to the internet) by looking at the @acl_local_domains setting. You need to tell AMaVis where to check if a certain domain is one of your destination domains. The reason is that you usually don't want to scan your outgoing emails. Imagine that an email is accidentally deemed to be spam and your customer gets warned of your emails. You don't need that.

So edit the /etc/amavis/conf.d/50-user file and before the "1;" enter these lines:

@lookup_sql_dsn = (
    ['DBI:mysql:database=mailserver;host=127.0.0.1;port=3306',
     'mailuser',
     'mailuser2009']);

$sql_select_policy = 'SELECT name FROM virtual_domains WHERE CONCAT("@",name) IN (%k)';

The @lookup_sql_dsn defines how AMaVis can access your database. And the $sql_select_policy sets the SQL query that is run when AMaVis wants to determine if the destination domain of the currently scanned email is one of your virtual domains. The %k is a list of strings that AMaVis expects to find. The actual query will look like this:

SELECT name
FROM virtual_domains
WHERE CONCAT("@",name)
IN (
    'john@example.com',
    'john',
    '@example.com',
    '@.example.com',
    '@.com',
    '@.')

This may look a bit weird. But in the end the string '@example.com' is searched for. Restart AMaVis:

$> /etc/init.d/amavis restart

Now try sending an email to john@example.com. If you examine the email headers of that mail you should find lines that got added by AMaVis:

X-Virus-Scanned: Debian amavisd-new at mymailserver
X-Spam-Score: 0
X-Spam-Level:
X-Spam-Status: No, score=0 tagged_above=-9999 required=6.31 tests=[none]

Your users will be able to filter out spam depending on this information. The X-Spam-Status line will be set to "Yes" if the spam score is above the required score ($sa_tag2_level_deflt). The X-Spam-Level line contains a number of "*" that desc the spam score. If your email program looked for a X-Spam-Level: ***** header then it would catch spam with at least a score of 5. A real-life example of caught spam:

X-Spam-Status: Yes, hits=16.0 tagged_above=-9999.0 required=6.31
    tests=BAYES_99, FORGED_MUA_OUTLOOK, MSGID_FROM_MTA_ID,
    RCVD_IN_BL_SPAMCOP_NET, UNDISC_RECIPS, URIBL_OB_SURBL, WORK_AT_HOME
    X-Spam-Level: ***************
    X-Spam-Flag: YES

All incoming emails should now be tested for viruses and spam. Try that. SpamAssassin comes with a spam sample containing the GTUBE (Generic Test for Unsolicited Bulk Email) that contains a signature that is supposed to trigger spam filters - just like EICAR.COM for virus scanners. Send John that spam email:

$> sendmail john@example.com < /usr/share/doc/spamassassin/examples/sample-spam.txt

Among other information your /var/log/mail.log will now contain a line being written by AMaViS:

amavis[13001]: (13001-02) Passed SPAM, <...> -> <john@example.com>, ...

So the email was detected as spam and passed through to John. Very well. Finally, fix the permissions of the AMaViS configuration files so that no unauthorized user can read the database password:

$> chmod o= /etc/amavis/conf.d/50-user