There is a newer issue of thie ISPmail guide available if you are using Debian Jessie!
(I have not tested this kind of AMaViSd setup in production thoroughly. Proceed with prejudice. Please comment if everything worked for you. Thanks.)
(This section is completely optional. If you don’t want to filter incoming email for malware then feel free to skip it. Note that using AMaViSd and ClamAV as described here will increase your RAM usage by up to 500 MB and will considerably lower the througput of emails.)
In a previous section we already dealt with restrictions during the SMTP dialog. You learned about real-time black lists (RBLs) which help you block most of the incoming spam. There is another piece of software that can help you fight back internet goons: AMaViSd (A Mail Virus Scanner daemon). It has two main features:
- reject spam emails or send them to a quarantine
- reject malware emails
Fighting spam with AMaViSd is arguably not optimal. Quarantined emails seem to be confuse users and create work for system administrators. And the spam detection rate appears to be rather low compared to bayesian filters. You would have to train AMaViSd (which then uses SpamAssassin for spam detection) using serious amounts of ham (good) and spam (bad) emails. And the typical user won’t be able to tell spam from the mailing list newsletter they subscribed to but can’t remember. Spam detection is a very individual thing so it should be left to users and their email programs. Feed a mail client like Thunderbird one month of ham and spam emails and you will get a nearly 100% accuracy of spam detection. So to cut it short: this guide will not cover spam detection using AMaViSd. Sorry.
However malware is still a problem. Windows is losing market share thanks to MacOS and more user-friendly Linux desktops. But there are still many Windows users around and as MacOS is gaining shares it will probably get attacked, too. AMaViSd can fortunately help here. It makes use of any virus scanner that has a command-line interface. And there is even a free virus scanner called ClamAV that detect computer viruses, trojan horses and even phishing emails. Let’s make use of that.
Installation
Install AMaViS and its suggested packages:
$> apt-get install amavisd-new clamav-daemon clamav-testfiles clamav-freshclam
This setup is a tad complex and can go wrong. So if you are adding content filtering to your live server I strongly suggest you enable the soft bounce feature in Postfix. If Postfix intends to reject or bounce an email then this feature will rather keep the mail in the queue and try again later. Whenever you do major changes in your mail server setup I recommend you enable it:
$> postconf -e soft_bounce=yes
$> postfix reload
Alright – so how does AMaViSd plug into Postfix? Basically AMaViSd runs permanently as a daemon process and listens on a TCP port (10024) for incoming SMTP connections. Postfix will send all incoming email to that port. AMaViSd then runs all configured malware tests and decides what to do with the email. If it consider the email clean it will pass it back to Postfix – but on a different TCP port (10025) where Postfix will accept any email for delivery.
However by default Postfix doesn’t listen on port 10025. Neither does it know of AMaViS. So first you need to edit /etc/postfix/master.cf and add these two services:
amavis unix – – n – 5 smtp
-o smtp_data_done_timeout=1200
-o smtp_send_xforward_command=yes
-o smtp_tls_note_starttls_offer=no127.0.0.1:10025 inet n – n – – smtpd
-o content_filter=
-o smtpd_delay_reject=no
-o smtpd_client_restrictions=permit_mynetworks,reject
-o smtpd_helo_restrictions=
-o smtpd_sender_restrictions=
-o smtpd_recipient_restrictions=permit_mynetworks,reject
-o smtpd_data_restrictions=reject_unauth_pipelining
-o smtpd_end_of_data_restrictions=
-o smtpd_restriction_classes=
-o mynetworks=127.0.0.0/8
-o smtpd_error_sleep_time=0
-o smtpd_soft_error_limit=1001
-o smtpd_hard_error_limit=1000
-o smtpd_client_connection_count_limit=0
-o smtpd_client_connection_rate_limit=0
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
-o local_header_rewrite_clients=
-o smtpd_milters=
-o local_recipient_maps=
-o relay_recipient_maps=
Note that all lines but the first lines of each service need to be indented by spaces.
(You can find detailed information on these services and the general configuration in the documentation in the file /usr/share/doc/amavisd-new/README.postfix.gz on your mail server.)
Restart Postfix to make it pick up these two new services:
postfix reload
Now you need to tell Postfix to use the “amavis” service as a content filter. Making Postfix forward emails to AMaViS is done by setting the content_filter setting. Also set the “receive_override_options” setting that will be explained later by running these shell commands:
postconf -e content_filter=amavis:[127.0.0.1]:10024
postconf -e receive_override_options=no_address_mappingsservice postfix restart
The purpose is probably obvious. All email is forwarded to the “amavis” service defined in the /etc/postfix/master.cf using TCP port 10024 on the IP address 127.0.0.1 (localhost).
Why filter outbound email?
You may wonder why AMaViSd even scans outgoing email. But nowadays I would indeed recommend to allow scanning any email because it helps determine a virus infestation of a local PC. Ultimately you decide. There are different approaches described in the AMaViS documentation.
Configuring AMaViS
AMaViS is configured using different files in /etc/amavis/conf.d. You may want to check what is set there. Above all you need to change the 15-content_filter_mode file to enable virus and spam scanning. Confusingly the default settings are:
#@bypass_virus_checks_maps = (
# \%bypass_virus_checks, \@bypass_virus_checks_acl, \$bypass_virus_checks_re);#@bypass_spam_checks_maps = (
# \%bypass_spam_checks, \@bypass_spam_checks_acl, \$bypass_spam_checks_re);
To enable scanning you have to remove the “#” signs in front of these four lines. It sounds like it would then bypass the scanning but instead it enables it.
The default scanner is ClamAV which is a free and decent anti-virus software. To use it you will have to get a group membership right:
adduser clamav amavis
/etc/init.d/clamav-daemon restart
The configuration files in /etc/amavis/conf.d are run in sorted order. A setting in “50-user” will override the same setting in “01-debian”. So if you feel you want to override any seeetings please put them into “50-user”. Make sure that your 50-user file ends with “1;” or else AMaViS will not start up properly.
Restart AMaViS if you have made changes to the config files:
/etc/init.d/amavis restart
Make sure that AMaViS is listening on TCP port 10024:
netstat -nap | grep 10024
You should get this output:
tcp 0 0 127.0.0.1:10024 0.0.0.0:* LISTEN 12345/amavisd
If you get such a line then AMaViS is running and waiting for incoming SMTP sessions. Otherwise check your /var/log/mail.log file – perhaps you have made a mistake in the configuration files.
This should get you started. For a more detailed documentation please see the README.Postfix that provided by AMaViS.
Testing
Now that everything is set up you will want to test your spam scanning process. As you have installed the “clamav-testfiles” package you can use those samples to test ClamAV.
First try to scan the sample files in /usr/share/clamav-testfiles:
$> clamdscan /usr/share/clamav-testfiles/
ClamAV will then scan each of the sample files by querying the ClamAV daemon running in the background. Using the daemon (“clamdscan” instead of “clamscan”) saves you loading the malware database every time. You should see:
/usr/share/clamav-testfiles/clam.pdf: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.chm: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.sis: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.impl.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-nsis.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.tar.gz: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bz2.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.7z: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-yc.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ole.doc: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.newc.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-aspack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ppt: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ea05.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.ea06.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.html: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-upack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.bz2: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bin-be.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.cab: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.mbox.base64: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.arj: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.binhex: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-fsg.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.mbox.uu: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.odc.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_cache_emax.tgz: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.tnef: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-mew.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-upx.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-pespin.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_ISmsi_ext.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.mail: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-wwpack.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_IScab_ext.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_ISmsi_int.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam-petite.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.szdd: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.d64.zip: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam_IScab_int.exe: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.bin-le.cpio: ClamAV-Test-File FOUND
/usr/share/clamav-testfiles/clam.exe.rtf: ClamAV-Test-File FOUND
———– SCAN SUMMARY ———–
Infected files: 44
Time: 1.478 sec (0 m 1 s)
If your output looks like this then ClamAV has successfully found the malware patterns in the test files. If you instead get “ERROR: Can’t connect to clamd: No such file or directory” the please make sure that the ClamAV daemon is running by executing “service clamav-daemon start”.
Now let’s try running an infested email through Postfix using the swaks tool to see if AMaViSd and ClamAV are used correctly:
swaks -t john@example.org –attach – –server localhost –suppress-data < /usr/share/clamav-testfiles/clam.exe
In addition to many lines in your /var/log/mail.log file dealing with “postfix” you should see one line dealing with “amavis”. It will look like this:
29 23:16:23 sirius amavis[4558]: (04558-02) Blocked INFECTED (ClamAV-Test-File) {DiscardedOutbound,Quarantined}, LOCAL [::1]:44226 [::1] <root@mailserver.example.org> -> <john@example.org>, quarantine: C/virus-ClS0k87Qzm4Q, Queue-ID: 6608ED6B, Message-ID: <20131029221623.6608ED6B@sirius.localdomain>, mail_id: ClS0k87Qzm4Q, Hits: -, size: 1547, 128 ms
This proves that the ClamAV malware scanner is working and found a test pattern. Also the email was moved to /var/lib/amavis/virusmails/C/virus-ClS0k87Qzm4Q for your inspection. To learn how to deal with quarantined emails please see the AMaViS documentation. In a nutshell you could release this potentially dangerous file from the quarantine and deliver it nonetheless using “amavisd-release C/virus-ClS0k87Qzm4Q“.
Nearly done
Did you enable soft bounce earlier? Everything works? If the
mailq
command shows that there are still emails in the queue that need to be delivered then re-queue them first:
postsuper -r ALL
Re-scheduling means that Postfix reconsiders what to do with every email in the queue. Transport (routing) and content filtering information stick to an email. So even if you reconfigure Postfix then emails in the queue would not pick that up until you re-queue them.
Now flush the queue
postfix flush
and the emails should get delivered. The /var/log/mail.log will give you information on what happens.
If everything works as you expect then switch off soft bounce mode again and you are done:
postconf -e soft_bounce=no
Want more?
If you seriously want to run AMaViS on a production mail server then please spend time with its documentation. It’s a complex piece of software that can integrate a lot of third-party software like virus scanners. It can be tweaked a lot. It can automatically add DKIM signatures on outgoing email. It has a quarantine system so you can moderate spam emails. It just offers many additional features outside of the scope of this tutorial. If you believe you have a valuable addition to this tutorial then please send a comment on this page. Thank you.
Just one word on spam scanning. SpamAssassin (as being used by AMaViS) uses different rules to look for spam but can also learn to distinguish ham from spam emails. To do that properly it needs to scan an equal amount of several hundred ham and spam emails before it can really do its job. And as far as I know there is no way to learn spam per-user. So whatever you teach SpamAssassin is good or bad will apply for all your users.
/etc/postfix/master.cf:
amavis unix – – n – 2 smtp
=> This is not working on my server (error-message “warning: connect to transport private/smtp-amavis: No such file or directory”).
I looked in my old Debian Squeeze configuration (created after the workaround tutorial) and there the above line is different
smtp-amavis unix – – n – 2 smtp
^^^^^
Hi Matt!
You have to adapt the “main.cf” file too:
…
# -> Amavis as Content Filter
content_filter = smtp-amavis:[127.0.0.1]:10024
#——————–^^^^^^^^^^^^^^
receive_override_options = no_address_mappings
…
Sorry, have over seen the lines
…
“postconf -e content_filter=amavis:[127.0.0.1]:10024”
service postfix restart
….
in the new tutorial. In this case you have to use (master.cf)
…
amavis unix – – n – 2 smtp
…
as Christoph wrote.
In most cases “2” amavis servers is quiet low. If your server has some resources left, you can increase the number amavis servers like
…
amavis unix – – n – 10 smtp
…
This step also requires the adaptation of the “/etc/amavis/conf.d/50-user” file:
Add / or change the line
…
$max_servers=11;
…
I personally increase the number of servers in amavis to a higher value as in postfix, so that amavis has a small space to the top.
Configuring/Tweaking ClamAV:
By default ClamAV is also configured to scan archive files as well. However for this to work you will have to installed unzip and unrar on system.
# apt-get install unrar unzip
“SelfCheck 3600” is too long time for clamav Virus definitions integrity I prefer to set it to 600, i.e.
…
SelfCheck 600
…
For Desktop use clamav default config is pretty good. However for servers its good to raise to up MaxThreads
(need some RAM and other ressourses):
…
MaxThreads 30
…
Is it just me or is the section on configuring Amavis is repeated twice here ?
Nick
Yes a first time we can read :
“The default scanner is ClamAV which is a free and decent anti-virus software. To use it you will have to get two group memberships right:
adduser clamav amavis
adduser amavis clamav
service clamav-daemon restart
service amavis restart”
and a second time :
“The default scanner is ClamAV which is a free and decent anti-virus software. To use it you will have to get a group membership right:
adduser clamav amavis
/etc/init.d/clamav-daemon restart”
It’s a little bit confusing. It seems a copy-paste problem or a part of an old version of the document. The second version seems more logical. Christoph when you have a few time could you fix that ?
And thanks for such amount of work, so usefull !
I noticed that the number of processes Amavisd was increased from 2 (default) to 5
amavis unix – – n – 5 smtp
but I do not see the configuration file / etc/amavis/conf.d/50-user where you should have:
$ max_servers = 5;
is that right?
You are correct. I had to do this as well. With only 2 instances running Amavis was scanning about 2 emails every 4 seconds, which is entirely too slow if you have a big deployment. I upped it to 8 (to support around 3000 mailboxes) and it delivers immediately. Without increasing this directive mail will flow through your system very slowly and fill up your postfix queue, which delays sending.
Be aware that this takes quite a bit of horsepower.
Why “smtp_tls_note_starttls_offer” is set to no?
In the parameters of configuration of Postfix is already disabled by default
smtp_tls_note_starttls_offer (default: no)
I am configuring Amavis to run on a different server than Postfix, but am having trouble when sending a test email with “swaks –to john@example.org –server localhost”
I see this in the Postfix log:
Mail-A postfix/error[11035]: 69BB7160136: to=, relay=none, delay=0.02, delays=0.02/0.01/0/0, dsn=4.4.1, status=deferred (delivery temporarily suspended: connect to 10.10.10.87[10.10.10.87]:10024: Connection refused)
It seems as though Amavis is rejecting the connection from Postfix.
I told Postfix to use amavis with this command:
postconf -e content_filter=amavis:[10.10.10.87]:10024
If I can provide any more information, please let me know. Thanks!
The problem was Amavis was listening on the loopback interface (127.0.0.1) and needed to be listening on the interface with the ip address 10.10.10.87. I resolved it by adding these two lines to the 50-user config file for Amavis:
@inet_acl = qw(10.10.10.90); #### Allow SMTP from the mailserver only #####
$inet_socket_bind = ‘10.10.10.87’; ##### Listen on IP address instead of socket ###
If you want to use a dedicated machine for this setup you need to tell it to listen to connection requests outside of itself.
You will see a message:
Starting amavisd: The value of variable $myhostname is “debian1”, but should have been
a fully qualified domain name; perhaps uname(3) did not provide such.
You must explicitly assign a FQDN of this host to variable $myhostname
in /etc/amavis/conf.d/05-node_id, or fix what uname(3) provides as a host’s
network name!
(failed).
invoke-rc.d: initscript amavis, action “start” failed.
WARNING: Starting amavisd-new failed. Please check your configuration.
So edit /etc/amavis/conf.d/05-node_id and replace
#$myhostname = “mail.example.com”;
with
$myhostname = “mail.yourmailserv.com”;
(you have just to remove the # at the beginning of the line)
Hi, May be it already have answered somewhere else but as I followed this tutorial, I would like to request you to check my questions. My server virus scanning is working fine, But I want amavisd to send all virus tagged and spam mail to user’s separate mail box, I don’t want them delete the mail. I don’t know how do I enable spam box. Also I don’t know where amavisd keep the quarantine mails, I actually need the quarantine mail If user want it.
Sorry for repeating question.
Hi, May be it already have answered somewhere else but as I followed this tutorial, I would like to request you to check my questions. My server virus scanning is working fine, But I want amavisd to send all virus tagged and spam mail to user’s separate mail box, I don’t want them delete the mail. I don’t know how do I enable spam box. Also I don’t know where amavisd keep the quarantine mails, I actually need the quarantine mail If user want it.
Hello,
what about the sieve filter for moving all mail tagged as SPAM into spam folder?
Thanks
Hello,
i’m not sure why but spamassassin don’t scan mails 🙁
I activated everything and amavisd is working (X-Virus-Scanned: Debian amavisd-new at…) but there is no X-SPAM Header.
Thanks
Hello,
same problem here.
Seems like SA does not check incoming e-mail.
Thanks.
Please check the file
/etc/amavis/conf.d/20-debian_defaults and adapt this values:
$sa_tag_level_deflt = undef; # add spam info headers if at, or above that level
if you put “undef” all headers will be added automatically to all emails.
X-Spam-Flag: NO
X-Spam-Score: -0.799
X-Spam-Level:
X-Spam-Status: No, score=-0.799 required=6.31 tests=[DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, FREEMAIL_FROM=0.001,
HTML_MESSAGE=0.001, RCVD_IN_DNSWL_LOW=-0.7, SPF_PASS=-0.001] autolearn=ham
Uncommenting all lines in amavis’ config filter asks for spamassassin to be installed.
If I do not uncomment spam-checking, everything runs fine.
As I do not need it, this is just a little hickup in the neat description you did here.
Hi all,
I have found a solution how to run spamassassin on all domains, no matter how many you have.
Just open /etc/amavis/conf.d/05-domain_id
and change
@local_domains_acl = ( “.$mydomain” );
to
@local_domains_acl = qw(.);
To search for viruses inside rar files as well (if you feel the need to)…
apt-get install libclamunrar6
too (from non-free).
Hallo Christoph,
mit Hilfe Deiner Anleitung habe ich Schritt für Schritt einen Mail Server aufgesetzt. Ich habe schon für viele andere Installationen howtos und Tipps im Internet gesucht, doch jedes Mal war das ein Puzzlespiel aus verschiedenen Quellen. Deine strukturierte Anleitung mit Hintergrundinformationen und Zwischentest ist eine wahre Perle, Chapeau! Ich werde mich gerne erkenntlich zeigen.
An einer Stelle bin ich aber noch etwas unsicher, da das Zusammenspiel von Amavis und ClamAV bei mir nicht sofort zu funktionieren schien. Beim Suchen in ich auf eine Anleitung meines Providers df.eu gestossen, dort wird für die Datei ‘master.cf’ folgendes
smtp-amavis unix – – n – 2 smtp
statt
amavis unix – – n – 5 smtp
und für die Datei ‘main.cf’
#ContentFilter:
content_filter = smtp-amavis:[127.0.0.1]:10024
receive_override_options = no_address_mappings
vorgeschlagen. Danach wurden erst bei mir die Virus-Mails erkannt.
Habe ich etwas übersehen?
Nochmals Danke für die tolle Anleitung, Olaf
Es wundert mich, dass der AMaViS bei dir gar nichts getan hat. Ob der Dienst "smtp-amavis" oder "amavis" genannt wird, ist egal, solange die Bezeichnung einheitlich benutzt wird.
Die "receive_override_options" sollte man setzen, damit es keine Probleme mit verdoppelten Emails gibt. Das passiert ansonsten, wenn Aliase doppelt ausgeführt werden. Ich habe die Zeile anscheinend vergessen, mit ins Tutorial aufzunehmen und habe die Seite gerade mal korrigiert. Danke für den Hinweis.
is betther configure a house keeping
find /var/lib/amavis/virusmails -type f -ctime +6 -exec rm ‘{}’ \;
is my option. betther see what is better to you!
Another option if you just want a milter to run Antivirus checks on incoming mail is ClamSMTP. Runs much leaner than Amavis.
Wheezy Howto: http://www.server-world.info/en/note?os=Debian_7.0&p=mail&f=6
* If you use other milters like OpenDKIM, you need to also add the following line at the end of the ClamSMTP entry in /etc/postfix/master.cf (After ‘-o smtpd_authorized_xforward_hosts=127.0.0.0/8’):
-o receive_override_options=no_header_body_checks,no_unknown_recipient_checks,no_milters
This prevents OpenDKIM from re-submitting the email to ClamSMTP for a second unnecessary scan (waste of time & resources).
Thanks for your guide, Christoph! Looking forward to your take on Jessie