Server-side filters: Sieve
What is server-based filtering?
You already have a fully functioning mail server that even tags spam. But so far you left the actual task of sorting out the tagged spam to the user. We can do better by setting up server-side filters. The task ahead is to move all spam emails to a user's "Spam" folder. We are using the Sieve feature of Dovecot which is a mail filter like procmail (which does not work for virtual mailboxes). Sieve has a simple scripting language that allows us to forward all emails that are tagged by SpamAssassin to a "Spam" folder automatically. At the end of the /etc/dovecot/dovecot.conf file you will find a "plugin" section. Within there you can define a "sieve_global_path" which points to a default filter file that takes effect if the recipient user hasn't defined any per-user filtering rules.
By the way the term "server-side filtering" means that the rules are executed on the server automatically. As opposed to "client-side filtering" which is configured in the mail program on the user's computer. Apparently server-side filtering is preferred as it happens even if the user is offline. And it gives you interesting options such as a vacation autoresponder or filtering based on header fields.
The global sieve filter script
A basic recipe will do the "Spam" sorting. Create a /var/vmail/globalsieverc file containing these lines:
require ["fileinto"];
# Move spam to spam folder
if header :contains "X-Spam-Flag" ["YES"] {
fileinto "spam";
stop;
}
Make sure that the globalsieverc file is readable by the vmail user:
$> chown vmail /var/vmail/globalsieverc
Define this file as a global filtering file within the "plugin { }" section of your /etc/dovecot/dovecot.conf file:
sieve_global_path = /var/vmail/globalsieverc
and restart Dovecot:
$> /etc/init.d/dovecot restart
Send John another spam email:
$> sendmail john@example.com < /usr/share/doc/spamassassin/examples/sample-spam.txt
Then watch the /var/vmail/dovecot-deliver.log file. The last line should look like:
deliver(john@example.com): 2009-07-01 01:00:22 Info: msgid=<GTUBE1.1010101@example.net>: saved mail to spam
This says that the email was saved into the "spam" folder. If you fetch email you should see the email show up in that folder automatically.
Note that different folders only work when IMAP is used to fetch emails. POP3 doesn't have a notion of folders and will only show emails in the inbox. And the user will have to subscribe to the "Spam" folder manually. Alternatively you can add "spam" to the "/var/vmail/example.com/john/Maildir/subscriptions" file which contains a list of folders that John wants to see in his mail program.
Managesieve server
We were talking about per-user filter scripts. How does a user manage such a script? This is done by a low-level interface called "managesieve". Previously additional software like "pysieved" was needed to offer "managesieve". In Lenny's Dovecot this is already supported out of the box.
To enable it you'll have to add "managesieve" to the "protocols" line in your /etc/dovecot/dovecot.conf file. In its "protocol managesieve" section the line "sieve=~/.dovecot.sieve" should already be set. This means that a user's sieve filtering script will be stored in his virtual "home directory". John's sieve script would end up in "/var/mail/example.com/john/.dovecot.sieve".
Managesieve client
Your users probably don't want to learn the sieve filtering language. So they need a comfortable way to administer their filtering rules. This can be accomplished by the "avelsieve" plugin to the "squirrelmail" web mail software. It's assumed that you have set up squirrelmail as decribed in its chapter. Install the "avelsieve" plugin:
$> aptitude install avelsieve
Fortunately that's all that needs to be done. If you login via squirrelmail you will see a new link at the top called "Filters". Click on it. You can now add rules of all kinds.
Note that avelsieve in Debian Lenny currently has a bug (Debian #516198 , Avelsieve #247) that can make existing rules disappear when using Dovecot and TLS. I have prepared a patched avelsieve Debian package that you can install with "dpkg -i" if you encounter the same problem.
Comments
Filter if not wrking
Dear,
Thanks for the nice tutorial. I have followed properly but my filtering not working. I can see filter link in squirrelmail but when click on that it say:
//##
Could not log on to timsieved daemon on your IMAP server localhost:2000.
Error Encountered: 111 Connection refused
##//
Any advice...?
/Touhid
check in
check in dovecot.conf
protocols = imap imaps pop3 pop3s managesieve
Horde/Ingo
Horde Ingo also offers managesieve-support (with tls), wich works like a charme, with this configuration in /pathtohorde/ingo/config/backends.php
/* Sieve Example */ $backends['sieve'] = array( 'driver' => 'timsieved', 'preferred' => 'imap.example.com', 'hordeauth' => full, 'params' => array( // Hostname of the timsieved server 'hostspec' => 'imap.example.de', // Login type of the server 'logintype' => 'PLAIN', // Enable/disable TLS encryption 'usetls' => true, // Port number of the timsieved server 'port' => 2000, // Name of the sieve script 'scriptname' => 'ingo', // The following settings can be used to specify an administration // user to update all users' scripts. If you want to use an admin // user, you also need to disable 'hordeauth' above. You have to use // an admin user if you want to use shared rules. // 'username' => 'cyrus', // 'password' => '*****', ), 'script' => 'sieve', 'scriptparams' => array(), 'shares' => false );Autoresponder with some sieve-magic
I really would appreciate, if someone (Christoph?) could add some info, on how to implement an autoresponder (also known as vacation-feature) using sieve.
Simple :)
That's simpler than you may think. If you have squirrelmail and avelsieve installed then just create a new rule and tick the "Vacation" box.
Sieve Filter not working
hello,
first of all, thanks for your great work. this tutorial isactually one of the best ever read!!! thanks for that!
but still i have a problem with the sieve filtering: dovecot seems to totally ignore the rule. if i set a rule with -avelsieve- the rule is correctly saved into userdir/sieve/phpscript.sieve. the symlink is also set correctly userdir/.dovecot.sieve => userdir/sieve/phpscript.sieve.
if i logout of squirrelmail and then login again.... then avelsieve/squirrelmail does not find any filter rules (no filter defined). the filterfile itself still lies in the correct folder... as mentioned above...
when an email is sent to an adress with a filter defined... nothing happens... the mail is delivered and the filter is ignored (i did not use tls/ssl)
in the dovecot config i checked the protocols settings... but all seems ok to me:
protocols = imap imaps pop3 pop3s managesieve
[ ... ]
protocol managesieve {
sieve=~/.dovecot.sieve
sieve_storage=~/sieve
}
i hope i provided all informations needed to detect the mistake i made ?!?!?
is there any other setting i have to take care of (in addition to the things i mentioned)??
thanks for any idea // greetings from cologne
chris
Bug?
Did you read the last sentence on this page regarding my bugfixed version of avelsieve for Lenny? ;)
sieve is working now :) - avelsieve still blind...
hello again,
i wrote you a message just a few minutes ago... telling you that my dovecot ignores all sieve filters and that squirrelmail tells me that there are no filters after a logout.
well at least the filter work generally. the mistake lay in the dovecot.conf
protocol lda {
log_path = /var/vmail/dovecot-deliver.log
auth_socket_path = /var/run/dovecot/auth-master
postmaster_address = admin@cre8.info
mail_plugins = cmusieve quota
# mail_plugins = quota
}
originally i had the plugins defined in two separate lines
mail_plugins = cmusieve
mail_plugins = quota
and this makes dovecot ignore the first definition / replace it with -quota-
now i wonder why avelsieve still does not notice the filter placed in the correct folder??
any advice??
Fatal error: Allowed memory size of 268435456 bytes exhausted
Im spanish and my english is too poor :-( Sorry.
When I click on Filters I have this response:
"Fatal error: Allowed memory size of 268435456 bytes exhausted (tried to allocate 1025 bytes) in /usr/share/squirrelmail/plugins/avelsieve/include/managesieve.lib.php on line 873"
I have Lenny:
matrix:~# dpkg -l avelsieve
Desired=Unknown/Install/Remove/Purge/Hold
| Estado=No/Instalado/Config-files/Desempaquetado/Fallo-config/Medio-inst/espera-disparo/pendiente-disparo
|/ Err?=(ninguno)/Retenido/Requiere-reinst/X=ambos problemas (Estado,Err: mayúsc.=malo)
||/ Nombre Versión Descripción
+++-==============-==============-============================================
ii avelsieve 1.9.7-6 Sieve mail filters plugin for SquirrelMail
matrix:~# dpkg -l squirrelmail
Desired=Unknown/Install/Remove/Purge/Hold
| Estado=No/Instalado/Config-files/Desempaquetado/Fallo-config/Medio-inst/espera-disparo/pendiente-disparo
|/ Err?=(ninguno)/Retenido/Requiere-reinst/X=ambos problemas (Estado,Err: mayúsc.=malo)
||/ Nombre Versión Descripción
+++-==============-==============-============================================
ii squirrelmail 2:1.4.15-4+len Webmail for nuts
matrix:~#
What is it?
Thanks
Pepe
I answer to my same one (Resolved mistake) ;-)
Resolved mistake.
It seems to be that one had given an infinite curl, due to a wrong configuration of /etc/dovecot/dovecot.conf.
With the following configuration it works well:
##
## MANAGESIEVE specific settings
##
protocol managesieve {
# Login executable location.
#login_executable = /usr/lib/dovecot/managesieve-login
# MANAGESIEVE executable location. See IMAP's mail_executable above for
# examples how this could be changed.
#mail_executable = /usr/lib/dovecot/managesieve
# Maximum MANAGESIEVE command line length in bytes. This setting is
# directly borrowed from IMAP. But, since long command lines are very
# unlikely with MANAGESIEVE, changing this will not be very useful.
#managesieve_max_line_length = 65536
# Specifies the location of the symlink pointing to the active script in
# the sieve storage directory. This must match the SIEVE setting used by
# deliver (refer to http://wiki.dovecot.org/LDA/Sieve#location for more
# info). Variable substitution with % is recognized.
#sieve=~/.dovecot.sieve
# This specifies the path to the directory where the uploaded scripts must
# be stored. In terms of '%' variable substitution it is identical to
# dovecot's mail_location setting used by the mail protocol daemons.
#sieve_storage=~/sieve
# If, for some inobvious reason, the sieve_storage remains unset, the
# managesieve daemon uses the specification of the mail_location to find out
# where to store the sieve files (see explaination in README.managesieve).
# The example below, when uncommented, overrides any global mail_location
# specification and stores all the scripts in '~/mail/sieve' if sieve_storage
# is unset. However, you should always use the sieve_storage setting.
# mail_location = mbox:~/mail
# To fool managesieve clients that are focused on timesieved you can
# specify the IMPLEMENTATION capability that the dovecot reports to clients
# (default: dovecot).
#managesieve_implementation_string = Cyrus timsieved v2.2.13
}
##
## LDA specific settings
##
protocol lda {
log_path = /var/vmail/dovecot-deliver.log
# Address to use when sending rejection mails.
postmaster_address = postmaster@xxxxxxx.org
# Hostname to use in various parts of sent mails, eg. in Message-Id.
# Default is the system's real hostname.
#hostname =
# Support for dynamically loadable plugins. mail_plugins is a space separated
# list of plugins to load.
#mail_plugins =
#mail_plugin_dir = /usr/lib/dovecot/modules/lda
# Binary to use for sending mails.
#sendmail_path = /usr/lib/sendmail
# UNIX socket path to master authentication server to find users.
auth_socket_path = /var/run/dovecot/auth-master
# Enabling Sieve plugin for server-side mail filtering
mail_plugins = sieve
# global_script_path = /var/vmail/globalsieverc
plugin {
# Path to a global sieve script file, which gets executed ONLY if
# user's private Sieve script doesn't exist. (e.g. /etc/dovecot/default.sieve)
# (v1.0.1 and older called this setting "global_script_path")
sieve_global_path = /var/vmail/globalsieverc
# The include extension fetches the :personal scripts from this
# directory. When ManageSieve is used, this is also where scripts
# are uploaded.
sieve_dir = ~/sieve
# The location of the user's active script:
sieve = ~/.dovecot.sieve
# Directory for :global include scripts (for v1.1 include extension)
sieve_global_dir = /var/vmail/sieve
sieve_maxscriptsize = 128
}
}
It seems to be that I had put in two places simultaneously the same judgments; putting the symbol "#" in front of the duplicated ones, resolved problem, at the moment.
Regards
default filtering fules with horde's ingo
I've written a HowTo on creating a default rule set that work with sieve and horde's ingo. You can find it on http://wiki.horde.org/DefaultFilterRules
user sieve filtering overrides global filtering
Thank you for this excellent tutorial Christoph!
One small note:
I think you should mention somewhere, that if a user filter file is present (ie ~/.dovecot.sieve) the global filters won't be executed!
It took me quite a while to figure out, why my spam filtering suddenly stopped working ^^
Actually...
Actually it's there. Quoting from above: "Within there you can define a "sieve_global_path" which points to a default filter file that takes effect if the recipient user hasn't defined any per-user filtering rules." Maybe that can be rephrased to make it more clearly?
avelsieve Bug in Lenny
"Note that avelsieve in Debian Lenny currently has a bug (Debian #516198 , Avelsieve #247) that can make existing rules disappear when using Dovecot and TLS. I have prepared a patched avelsieve Debian package that you can install with "dpkg -i" if you encounter the same problem."
I think this bug has been closed in the stable release (lenny) at 5th Sep. 2009. See also the following bugreport Debian Bug 542539
sieve location
protocol managesieve { sieve=/var/vmail/%d/%n/.dovecot.sieve sieve_storage=/var/vmail/%d/%n/sieve } plugin { ... sieve=/var/vmail/%d/%n/.dovecot.sieve sieve_dir=/var/vmail/%d/%n/sieve }Sieve - Setup in general!?
Hi there,
I have just recently setup my mailserver completely new after this tutorial...
I am running a virtual "Debian GNU/Linux 4.0" here, with:
* Dovecot 1.0.rc15
* Postfix 2.3.8
my managesieve plugin is telling me that it's not able to connect to sieveserver and on enabling the managesieve protocol in the dovecot onfig file on restarting dovecot it fails because the protocol managesieve is unknown.
Will I have to compile dovecot from scratch with sieve included?
NVM, for sure this howto was
NVM, for sure this howto was for LENNY ...
got it working for now!