What is server-based filtering?
You already have a fully functioning mail server that even tags spam. But so far you left the actual task of sorting out the tagged spam to the user. We can do better by setting up server-side filters. The task ahead is to move all spam emails to a user's "Spam" folder. We are using the Sieve feature of Dovecot which is a mail filter like procmail (which does not work for virtual mailboxes). Sieve has a simple scripting language that allows us to forward all emails that are tagged by SpamAssassin to a "Spam" folder automatically. At the end of the /etc/dovecot/dovecot.conf file you will find a "plugin" section. Within there you can define a "sieve_global_path" which points to a default filter file that takes effect if the recipient user hasn't defined any per-user filtering rules.
By the way the term "server-side filtering" means that the rules are executed on the server automatically. As opposed to "client-side filtering" which is configured in the mail program on the user's computer. Apparently server-side filtering is preferred as it happens even if the user is offline. And it gives you interesting options such as a vacation autoresponder or filtering based on header fields.
The global sieve filter script
A basic recipe will do the "Spam" sorting. Create a /var/vmail/globalsieverc file containing these lines:
require ["fileinto"];
# Move spam to spam folder
if header :contains "X-Spam-Flag" ["YES"] {
fileinto "spam";
stop;
}
Make sure that the globalsieverc file is readable by the vmail user:
$> chown vmail /var/vmail/globalsieverc
Define this file as a global filtering file within the "plugin { }" section of your /etc/dovecot/dovecot.conf file:
sieve_global_path = /var/vmail/globalsieverc
and restart Dovecot:
$> /etc/init.d/dovecot restart
Send John another spam email:$> sendmail john@example.com < /usr/share/doc/spamassassin/examples/sample-spam.txt
Then watch the /var/vmail/dovecot-deliver.log file. The last line should look like:
deliver(john@example.com): 2009-07-01 01:00:22 Info: msgid=<GTUBE1.1010101@example.net>: saved mail to spam
This says that the email was saved into the "spam" folder. If you fetch email you should see the email show up in that folder automatically.
Note that different folders only work when IMAP is used to fetch emails. POP3 doesn't have a notion of folders and will only show emails in the inbox. And the user will have to subscribe to the "Spam" folder manually. Alternatively you can add "spam" to the "/var/vmail/example.com/john/Maildir/subscriptions" file which contains a list of folders that John wants to see in his mail program.Managesieve server
We were talking about per-user filter scripts. How does a user manage such a script? This is done by a low-level interface called "managesieve". Previously additional software like "pysieved" was needed to offer "managesieve". In Lenny's Dovecot this is already supported out of the box.
To enable it you'll have to add "managesieve" to the "protocols" line in your /etc/dovecot/dovecot.conf file. In its "protocol managesieve" section the line "sieve=~/.dovecot.sieve" should already be set. This means that a user's sieve filtering script will be stored in his virtual "home directory". John's sieve script would end up in "/var/mail/example.com/john/.dovecot.sieve".
Managesieve client
Your users probably don't want to learn the sieve filtering language. So they need a comfortable way to administer their filtering rules. This can be accomplished by the "avelsieve" plugin to the "squirrelmail" web mail software. It's assumed that you have set up squirrelmail as decribed in its chapter. Install the "avelsieve" plugin:
$> aptitude install avelsieve
Fortunately that's all that needs to be done. If you login via squirrelmail you will see a new link at the top called "Filters". Click on it. You can now add rules of all kinds.
Note that avelsieve in Debian Lenny currently has a bug (Debian #516198 , Avelsieve #247) that can make existing rules disappear when using Dovecot and TLS. I have prepared a patched avelsieve Debian package that you can install with "dpkg -i" if you encounter the same problem.
31 Comments
Filter if not wrking
Submitted by Anonymous (not verified) on
Dear,
Thanks for the nice tutorial. I have followed properly but my filtering not working. I can see filter link in squirrelmail but when click on that it say:
//##
Could not log on to timsieved daemon on your IMAP server localhost:2000.
Error Encountered: 111 Connection refused
##//
Any advice...?
/Touhid
check in
Submitted by Anonymous (not verified) on
check in dovecot.conf
protocols = imap imaps pop3 pop3s managesieve
Horde/Ingo
Submitted by Anonymous (not verified) on
Horde Ingo also offers managesieve-support (with tls), wich works like a charme, with this configuration in /pathtohorde/ingo/config/backends.php
/* Sieve Example */ $backends['sieve'] = array( 'driver' => 'timsieved', 'preferred' => 'imap.example.com', 'hordeauth' => full, 'params' => array( // Hostname of the timsieved server 'hostspec' => 'imap.example.de', // Login type of the server 'logintype' => 'PLAIN', // Enable/disable TLS encryption 'usetls' => true, // Port number of the timsieved server 'port' => 2000, // Name of the sieve script 'scriptname' => 'ingo', // The following settings can be used to specify an administration // user to update all users' scripts. If you want to use an admin // user, you also need to disable 'hordeauth' above. You have to use // an admin user if you want to use shared rules. // 'username' => 'cyrus', // 'password' => '*****', ), 'script' => 'sieve', 'scriptparams' => array(), 'shares' => false );hooks.php
Submitted by allchemic (not verified) on
u need also change config/hooks.php for standard debian packages
class Ingo_Hooks
{
public function transport_auth($driver)
{
switch ($driver) {
case 'timsieved':
$ob = $GLOBALS['registry']->call('mail/imapOb');
return array(
'password' => $ob->getParam('password'),
'username' => $ob->getParam('username')
);
}
return true;
}
}
thst's brecause default settings return only username, without domain, and u need whole user@example.com to authentication
Autoresponder with some sieve-magic
Submitted by Anonymous (not verified) on
I really would appreciate, if someone (Christoph?) could add some info, on how to implement an autoresponder (also known as vacation-feature) using sieve.
Simple :)
Submitted by Christoph Haas on
That's simpler than you may think. If you have squirrelmail and avelsieve installed then just create a new rule and tick the "Vacation" box.
Sieve Filter not working
Submitted by Anonymous (not verified) on
hello,
first of all, thanks for your great work. this tutorial isactually one of the best ever read!!! thanks for that!
but still i have a problem with the sieve filtering: dovecot seems to totally ignore the rule. if i set a rule with -avelsieve- the rule is correctly saved into userdir/sieve/phpscript.sieve. the symlink is also set correctly userdir/.dovecot.sieve => userdir/sieve/phpscript.sieve.
if i logout of squirrelmail and then login again.... then avelsieve/squirrelmail does not find any filter rules (no filter defined). the filterfile itself still lies in the correct folder... as mentioned above...
when an email is sent to an adress with a filter defined... nothing happens... the mail is delivered and the filter is ignored (i did not use tls/ssl)
in the dovecot config i checked the protocols settings... but all seems ok to me:
protocols = imap imaps pop3 pop3s managesieve
[ ... ]
protocol managesieve {
sieve=~/.dovecot.sieve
sieve_storage=~/sieve
}
i hope i provided all informations needed to detect the mistake i made ?!?!?
is there any other setting i have to take care of (in addition to the things i mentioned)??
thanks for any idea // greetings from cologne
chris
Bug?
Submitted by Christoph Haas on
Did you read the last sentence on this page regarding my bugfixed version of avelsieve for Lenny? ;)
sieve is working now :) - avelsieve still blind...
Submitted by Anonymous (not verified) on
hello again,
i wrote you a message just a few minutes ago... telling you that my dovecot ignores all sieve filters and that squirrelmail tells me that there are no filters after a logout.
well at least the filter work generally. the mistake lay in the dovecot.conf
protocol lda {
log_path = /var/vmail/dovecot-deliver.log
auth_socket_path = /var/run/dovecot/auth-master
postmaster_address = admin@cre8.info
mail_plugins = cmusieve quota
# mail_plugins = quota
}
originally i had the plugins defined in two separate lines
mail_plugins = cmusieve
mail_plugins = quota
and this makes dovecot ignore the first definition / replace it with -quota-
now i wonder why avelsieve still does not notice the filter placed in the correct folder??
any advice??
Fatal error: Allowed memory size of 268435456 bytes exhausted
Submitted by Anonymous (not verified) on
Im spanish and my english is too poor :-( Sorry.
When I click on Filters I have this response:
"Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 1025 bytes) in /usr/share/squirrelmail/plugins/avelsieve/include/managesieve.lib.php on line 873"
I have Lenny:
matrix:~# dpkg -l avelsieve
Desired=Unknown/Install/Remove/Purge/Hold
| Estado=No/Instalado/Config-files/Desempaquetado/Fallo-config/Medio-inst/espera-disparo/pendiente-disparo
|/ Err?=(ninguno)/Retenido/Requiere-reinst/X=ambos problemas (Estado,Err: mayúsc.=malo)
||/ Nombre Versión Descripción
+++-==============-==============-============================================
ii avelsieve 1.9.7-6 Sieve mail filters plugin for SquirrelMail
matrix:~# dpkg -l squirrelmail
Desired=Unknown/Install/Remove/Purge/Hold
| Estado=No/Instalado/Config-files/Desempaquetado/Fallo-config/Medio-inst/espera-disparo/pendiente-disparo
|/ Err?=(ninguno)/Retenido/Requiere-reinst/X=ambos problemas (Estado,Err: mayúsc.=malo)
||/ Nombre Versión Descripción
+++-==============-==============-============================================
ii squirrelmail 2:1.4.15-4+len Webmail for nuts
matrix:~#
What is it?
Thanks
Pepe
I answer to my same one (Resolved mistake) ;-)
Submitted by Anonymous (not verified) on
Resolved mistake.
It seems to be that one had given an infinite curl, due to a wrong configuration of /etc/dovecot/dovecot.conf.
With the following configuration it works well:
##
## MANAGESIEVE specific settings
##
protocol managesieve {
# Login executable location.
#login_executable = /usr/lib/dovecot/managesieve-login
# MANAGESIEVE executable location. See IMAP's mail_executable above for
# examples how this could be changed.
#mail_executable = /usr/lib/dovecot/managesieve
# Maximum MANAGESIEVE command line length in bytes. This setting is
# directly borrowed from IMAP. But, since long command lines are very
# unlikely with MANAGESIEVE, changing this will not be very useful.
#managesieve_max_line_length = 65536
# Specifies the location of the symlink pointing to the active script in
# the sieve storage directory. This must match the SIEVE setting used by
# deliver (refer to http://wiki.dovecot.org/LDA/Sieve#location for more
# info). Variable substitution with % is recognized.
#sieve=~/.dovecot.sieve
# This specifies the path to the directory where the uploaded scripts must
# be stored. In terms of '%' variable substitution it is identical to
# dovecot's mail_location setting used by the mail protocol daemons.
#sieve_storage=~/sieve
# If, for some inobvious reason, the sieve_storage remains unset, the
# managesieve daemon uses the specification of the mail_location to find out
# where to store the sieve files (see explaination in README.managesieve).
# The example below, when uncommented, overrides any global mail_location
# specification and stores all the scripts in '~/mail/sieve' if sieve_storage
# is unset. However, you should always use the sieve_storage setting.
# mail_location = mbox:~/mail
# To fool managesieve clients that are focused on timesieved you can
# specify the IMPLEMENTATION capability that the dovecot reports to clients
# (default: dovecot).
#managesieve_implementation_string = Cyrus timsieved v2.2.13
}
##
## LDA specific settings
##
protocol lda {
log_path = /var/vmail/dovecot-deliver.log
# Address to use when sending rejection mails.
postmaster_address = postmaster@xxxxxxx.org
# Hostname to use in various parts of sent mails, eg. in Message-Id.
# Default is the system's real hostname.
#hostname =
# Support for dynamically loadable plugins. mail_plugins is a space separated
# list of plugins to load.
#mail_plugins =
#mail_plugin_dir = /usr/lib/dovecot/modules/lda
# Binary to use for sending mails.
#sendmail_path = /usr/lib/sendmail
# UNIX socket path to master authentication server to find users.
auth_socket_path = /var/run/dovecot/auth-master
# Enabling Sieve plugin for server-side mail filtering
mail_plugins = sieve
# global_script_path = /var/vmail/globalsieverc
plugin {
# Path to a global sieve script file, which gets executed ONLY if
# user's private Sieve script doesn't exist. (e.g. /etc/dovecot/default.sieve)
# (v1.0.1 and older called this setting "global_script_path")
sieve_global_path = /var/vmail/globalsieverc
# The include extension fetches the :personal scripts from this
# directory. When ManageSieve is used, this is also where scripts
# are uploaded.
sieve_dir = ~/sieve
# The location of the user's active script:
sieve = ~/.dovecot.sieve
# Directory for :global include scripts (for v1.1 include extension)
sieve_global_dir = /var/vmail/sieve
sieve_maxscriptsize = 128
}
}
It seems to be that I had put in two places simultaneously the same judgments; putting the symbol "#" in front of the duplicated ones, resolved problem, at the moment.
Regards
default filtering fules with horde's ingo
Submitted by Anonymous (not verified) on
I've written a HowTo on creating a default rule set that work with sieve and horde's ingo. You can find it on http://wiki.horde.org/DefaultFilterRules
user sieve filtering overrides global filtering
Submitted by Anonymous (not verified) on
Thank you for this excellent tutorial Christoph!
One small note:
I think you should mention somewhere, that if a user filter file is present (ie ~/.dovecot.sieve) the global filters won't be executed!
It took me quite a while to figure out, why my spam filtering suddenly stopped working ^^
Actually...
Submitted by Christoph Haas on
Actually it's there. Quoting from above: "Within there you can define a "sieve_global_path" which points to a default filter file that takes effect if the recipient user hasn't defined any per-user filtering rules." Maybe that can be rephrased to make it more clearly?
avelsieve Bug in Lenny
Submitted by Anonymous (not verified) on
sieve location
Submitted by Anonymous (not verified) on
protocol managesieve { sieve=/var/vmail/%d/%n/.dovecot.sieve sieve_storage=/var/vmail/%d/%n/sieve } plugin { ... sieve=/var/vmail/%d/%n/.dovecot.sieve sieve_dir=/var/vmail/%d/%n/sieve }Sieve - Setup in general!?
Submitted by Anonymous (not verified) on
NVM, for sure this howto was
Submitted by Anonymous (not verified) on
The driver said: (QUOTA) “Quota exceeded”
Submitted by Anonymous (not verified) on
Please help:
My ingo (Horde frontend) by some users says when they trying to put some extra emails in whitelist or blacklist:
There was an error activating the script. The driver said: (QUOTA) “Quota exceeded”
The rest I got working. I see that the size of the .dovecot.sieve file is 99.865 bytes. I don’t use quota for Maildir. Please let me know how to increase that quota. I was searching everywhere – haven’t found.
# Error Driver (QUOTA) "Quota exceeded"
Submitted by Anonymous (not verified) on
I have
$conf['storage']['params']['driverconfig'] = 'horde';
$conf['storage']['driver'] = 'sql';
in /var/www/html/horde/ingo/config/conf.php
I have 2,146 records for one of the users in ingo_lists table of horde DB.
The size of the script (.dovecot.sieve file) is 99.965 bytes.
When trying to activate script I get the following
# Changes saved.
# Error There was an error activating the script. The driver said:
(QUOTA) "Quota exceeded"
Please let me know where I can change that size of QUOTA
global sieve filter ignored
Submitted by Anonymous (not verified) on
global sieve filter ignored
Submitted by Anonymous (not verified) on
Sieve namespace
Submitted by Anonymous (not verified) on
I had to change "fileinto 'spam'" to "fileinto 'INBOX.spam'", or it'd throw an error in dovecot-deliver.log about not finding the namespace for the mailbox.
Move SPAM to spam
Submitted by Anonymous (not verified) on
Actualy there are an error in the tutorial
If you follow the tutorial and install&configure AMAVIS the amamavis is marking the spam as
X-Spam-Status: Yes
The other things you should take care: permission of the sieve rc and location
If you want that spam to be in Spam modify accordingly (spam -> Spam)
Learning Spam and Ham
Submitted by Anonymous (not verified) on
Most of our users use Apple's Mail application so I sort out Spam into a folder called Junk. This can be synchronised with the Mail-app. If the user gets a Spam delivered into his Inbox, he can kick it into the Junk-Folder just by klicking the "Werbung"-Button (german for Ad). If Mail is configured to sync this folder back to the server, there is a folder full of qualified Junk at the server. It would be consequent to use the Junk-Folder as a Spam-Learning source. This is no problem as this Spamassassin-Feature is well documented on the web. But how do I manage to use this on variable foldernames? Imagine, another user sortes his Spam into a folder called "Spam" (via managesieve), because his E-Mail Client does so…
Do you have an idea, how to get this working? Sebastian
re: Learning Spam and Ham
Submitted by Anonymous (not verified) on
If you are using Dovecot, look into another plugin called "antispam". It will call sa-learn for you.
Sieve reply from address
Submitted by Anonymous (not verified) on
When using sieve successfully with vacation auto-reply I have a problem: the reply message has "from" address in the form: username@dovecot.server.hostname not users.mail.address@our.domain
The dovecot.conf lda-section postmaster address is configured as postmaster@our.domain so i thought it would send the reply message with that as from address - or at least with the same address it got the mail (users.mail.address@our.domain).
Port 1490 for avelsieve
Submitted by Anonymous (not verified) on
I had to change the port in /etc/squirrelmail/avelsieve-config.php (auth failure) :
$sieveport = 2000;
I do not know why Debian changed the default port?
Filter not working
Submitted by Anonymous (not verified) on
I am using debian lenny box. All the configuration is ok, no error message but filter is not working. If i set any filter, nothing happen.
where to check for errors??
/Borua
can sieve filters cause server rejection (like exim can)?
Submitted by matt (not verified) on
how would you weild this on cPanel linux webhosts?
sieve script to rewrite the headers
Submitted by Kiran Reddy (not verified) on
Hi,
How can i write sieve script to rewrite headers.
Example : X-Original-To rewrite header to Delivered-To using sieve script.
Thanks.