[Phpgroupware-developers] hello to Sieve

[Tony (Angles) Puglisi]

Welcome to your new proposed filtering scheme: Sieve.
(no code committed yet - don't worry :)

Sieve has been in development for at least 6 years and describes a common 
scheme for
filtering mail. I don't  like to write docs nor reinvent the wheel, so I choose
Sieve, as much has already been written about it (ex. rfc3028).

Grabbing headers, unfolding them, and examining them is relatively easy, even 
the
regex is not that big of a deal. Writing a hard coded filter system is, 
therefor,
not that hard. However, writing something that is portable, flexible, and
standardized is hard.

Sieve scripts can be used as is on the server side if Sieve is supported there, 
such
as with Cyrus Imap. Additionally, since it is standardized, it should provide as
some sort of lingua franca that can be translated into other filtering schemes,
server side or not (maildrop, procmail, etc...).

Sieve is extensible, as my summary shows my intent to include php's POSIX and 
perl
regex in our implementation.

I'll probably start with a small subset of Sieve, so that if you'all decide that
Sieve blows, the code will still be applicable. For example, instead of 
"discard"
I'll probably make you "fileto" some sort of trash folder, we don't want to 
delete
important mail during the beta phase, do we :)

Sieve docs are long on words and short on "action" so I paste my summary below. 
If
you like to read, you may find some Sieve docs at:

Sieve Home Page
http://www.cyrusoft.com/sieve

Sieve Whitepaper
http://www.cyrusoft.com/sieve/sievewhitepaper.pdf

RFC 3028
http://www.ietf.org/rfc/rfc3028.txt

BEGIN SUMMARY: (you may wish to use the "view unformatted" option for this)

=====  String Comparison  =====

Three kinds
        :is                     exact match

        :contains       substring match
                :comparator "i;ascii-casemap"   case-insensitive comparison
                        (which treats uppercase and lowercase characters in
                        the ASCII subset of UTF-8 as the same)

                :comparator "i;octet"           case-sensitive comparison
                        (simply compares octets)

                        NOTES:
                        - default comparator if unspecified is "i;ascii-casemap"
                        - when specifying a name of a header line (ex. "From"),
                                the comparator is always "i;ascii-casemap" case 
insensitive
                        - Comparators other than i;octet and i;ascii-casemap 
must
                                be declared with require, as they are 
extensions.

        :matches        wildcard glob-style match
                using the characters "*" and "?".  "*" matches zero or more
                characters, and "?" matches a single character.

        ANGLES PLANNED EXTNSIONS:
        :ereg           php POSIX regex
        :preg           php Perl regex


=====  Addresses Comparisons  =====

        :localpart              acts on the local-part (left-side of the @ sign)
        :domain                 acts on the domain part (right-side of the @ 
sign)
        :all                    acts on the whole address


=====  Blocks  =====

Blocks are sets of commands enclosed within curly braces.  Blocks are
supplied to commands so that the commands can implement control
commands.

A control structure is a command that happens to take a test and a
block as one of its arguments; depending on the result of the test
supplied as another argument, it runs the code in the block some
number of times.

With the commands supplied in this memo, there are no loops.  The
control structures supplied--if, elsif, and else--run a block either
once or not at all.  So there are two arguments, the test and the
block.

=====  Control Structures: If, Require, Stop  =====

Syntax:   if
Syntax:   elsif
Syntax:   else

        There is no sub if-then blocks, just linear if--else(if)*--(end)

Syntax:   stop

   The "stop" action ends all processing.  If no actions have been
   executed, then the keep action is taken.

Syntax:   require

   The require command, if present, MUST be used before anything other
   than a require can be used.  An error occurs if a require appears
   after a command other than require.

   Example:  require ["fileinto", "reject"];

   Example:  require "fileinto";
             require "vacation";


=====  Action Commands  =====

Five available: keep, fileinto, redirect, reject, discard

Syntax:   keep  =========================================

   The "keep" action is whatever action is taken in lieu of all other
   actions, if no filtering happens at all; generally, this simply means
   to file the message into the user's main mailbox.  This command
   provides a way to execute this action without needing to know the
   name of the user's main mailbox, providing a way to call it without
   needing to understand the user's setup, or the underlying mail
   system.


Syntax:   fileinto   =========================================

   The "fileinto" action delivers the message into the specified folder.
   Implementations SHOULD support fileinto, but in some environments
   this may be impossible.

   The capability string for use with the require command is "fileinto".

   In the following script, message A is filed into folder
   "INBOX.harassment".

   Example:  require "fileinto";
             if header :contains ["from"] "coyote" {
                fileinto "INBOX.harassment";
             }


Syntax:   redirect   =========================================

   The "redirect" action is used to send the message to another user at
   a supplied address, as a mail forwarding feature does.  The
   "redirect" action makes no changes to the message body or existing
   headers, but it may add new headers.  The "redirect" modifies the
   envelope recipient.

   The redirect command performs an MTA-style "forward"--that is, what
   you get from a .forward file using sendmail under UNIX.  The address
   on the SMTP envelope is replaced with the one on the redirect command
   and the message is sent back out.  (This is not an MUA-style forward,
   which creates a new message with a different sender and message ID,
   wrapping the old message in a new one.)

   A simple script can be used for redirecting all mail:

   Example:  redirect "address@hidden";

   Implementations SHOULD take measures to implement loop control,
   possibly including adding headers to the message or counting received
   headers.  If an implementation detects a loop, it causes an error.


Syntax:   reject  =========================================

   The optional "reject" action refuses delivery of a message by sending
   back an [MDN] to the sender.  It resends the message to the sender,
   wrapping it in a "reject" form, noting that it was rejected by the
   recipient.  In the following script, message A is rejected and
   returned to the sender.

   Example:  if header :contains "from" "address@hidden" {
                reject "I am not taking mail from you, and I don't want
                your birdseed, either!";
             }

   A reject message MUST take the form of a failure MDN as specified  by
   [MDN].    The  human-readable  portion  of  the  message,  the  first
   component of the MDN, contains the human readable message  describing
   the  error,  and  it  SHOULD  contain  additional  text  alerting the
   original sender that mail was refused by a filter.  This part of  the
   MDN might appear as follows:

   ------------------------------------------------------------
   Message was refused by recipient's mail filtering program.  Reason
   given was as follows:

   I am not taking mail from you, and I don't want your birdseed,
   either!
   ------------------------------------------------------------

   The MDN action-value field as defined in the MDN specification MUST
   be "deleted" and MUST have the MDN-sent-automatically and automatic-
   action modes set.

   Because some implementations can not or will not implement the reject
   command, it is optional.  The capability string to be used with the
   require command is "reject".


Syntax:   discard  =========================================

   Discard is used to silently throw away the message.  It does so by
   simply canceling the implicit keep.  If discard is used with other
   actions, the other actions still happen.  Discard is compatible with
   all other actions.  (For instance fileinto+discard is equivalent to
   fileinto.)

   Discard MUST be silent; that is, it MUST NOT return a non-delivery
   notification of any kind ([DSN], [MDN], or otherwise).

   In the following script, any mail from "address@hidden" is thrown
   out.

   Example:  if header :contains ["from"] ["address@hidden"] {
                discard;
             }

   While an important part of this language, "discard" has the potential
   to create serious problems for users: Students who leave themselves
   logged in to an unattended machine in a public computer lab may find
   their script changed to just "discard".  In order to protect users in
   this situation (along with similar situations), implementations MAY
   keep messages destroyed by a script for an indefinite period, and MAY
   disallow scripts that throw out all mail.


=====  Test Commands  =====

   Tests are used in conditionals to decide which part(s) of the
   conditional to execute.

Available tests:
        address
        allof
        anyof
        envelope
        exists
        false
        header
        not
        size
        true


Syntax:   address [ADDRESS-PART] [COMPARATOR] [MATCH-TYPE]

   The address test matches Internet addresses in structured headers
   that contain addresses.  It returns true if any header contains any
   key in the specified part of the address, as modified by the
   comparator and the match keyword.


Syntax:   allof   =========================================

   The allof test performs a logical AND on the tests supplied to it.

   Example:  allof (false, false)  => false
             allof (false, true)   => false
             allof (true,  true)   => true

   The allof test takes as its argument a test-list.


Syntax:   anyof   =========================================

   The anyof test performs a logical OR on the tests supplied to it.

   Example:  anyof (false, false)  => false
             anyof (false, true)   => true
             anyof (true,  true)   => true


Syntax:   envelope [COMPARATOR] [ADDRESS-PART] [MATCH-TYPE]

   The "envelope" test is true if the specified part of the SMTP (or
   equivalent) envelope matches the specified key.

   If one of the envelope-part strings is (case insensitive) "from",
   then matching occurs against the FROM address used in the SMTP MAIL
   command.

NOTE (ANGLES):
        I THINK THIS IS ONLY AVAILABLE IN CERTAIN SITUATIONS,
        SUCH AS DURING AN SMTP TRANSACTION, THIS IS TH ONLY
        TIME WHEN THIS SPECIFIC "RCPT TO" DATUM IS EXPOSED,
        I.E. A CLIENT FEEDING AN OUTGOING MESSAGE TO A SMTP SERVER,
        PERHAPS OTHER OCCASIONS SUCH AS DURING SMTP DESTINATION
        FINAL DELIVERY. I MAY BE WORNG, THOUGH.


Syntax:   exists   =========================================

   The "exists" test is true if the headers listed in the header-names
   argument exist within the message.  All of the headers must exist or
   the test is false.

   The following example throws out mail that doesn't have a From header
   and a Date header.

   Example:  if not exists ["From","Date"] {
                discard;
             }


Syntax:   false   =========================================

   The "false" test always evaluates to false.


Syntax:   header [COMPARATOR] [MATCH-TYPE]   =============

   The "header" test evaluates to true if any header name matches any
   key.  The type of match is specified by the optional match argument,
   which defaults to ":is" if not specified, as specified in section
   2.6.

   Like address and envelope, this test returns true if any combination
   of the string-list and key-list arguments match.

   If a header listed in the header-names argument exists, it contains
   the null key ("").  However, if the named header is not present, it
   does not contain the null key.  So if a message contained the header

           X-Caffeine: C8H10N4O2

   these tests on that header evaluate as follows:

           header :is ["X-Caffeine"] [""]         => false
           header :contains ["X-Caffeine"] [""]   => true


Syntax:   not    =========================================

   The "not" test takes some other test as an argument, and yields the
   opposite result.  "not false" evaluates to "true" and "not true"
   evaluates to "false".


Syntax:   size <":over" / ":under">    =============================

   The "size" test deals with the size of a message.  It takes either a
   tagged argument of ":over" or ":under", followed by a number
   representing the size of the message.

   If the argument is ":over", and the size of the message is greater
   than the number provided, the test is true; otherwise, it is false.

   If the argument is ":under", and the size of the message is less than
   the number provided, the test is true; otherwise, it is false.

   Exactly one of ":over" or ":under" must be specified, and anything
   else is an error.

   The size of a message is defined to be the number of octets from the
   initial header until the last character in the message body.

   Note that for a message that is exactly 4,000 octets, the message is
   neither ":over" 4000 octets or ":under" 4000 octets.


Syntax:   true   =========================================

   The "true" test always evaluates to true.



======   Extended Example   ======

   The following is an extended example of a Sieve script.  Note that it
   does not make use of the implicit keep.

    #
    # Example Sieve Filter
    # Declare any optional features or extension used by the script
    #
    require ["fileinto", "reject"];

    #
    # Reject any large messages (note that the four leading dots get
    # "stuffed" to three)
    #
    if size :over 1M
            {
            reject text:
    Please do not send me large attachments.
    Put your file on a server and send me the URL.
    Thank you.
    .... Fred
    .
    ;
            stop;
            }
    #
    # Handle messages from known mailing lists
    # Move messages from IETF filter discussion list to filter folder
    #
    if header :is "Sender" "address@hidden"
            {
            fileinto "filter";  # move to "filter" folder
            }
    #
    # Keep all messages to or from people in my company
    #
    elsif address :domain :is ["From", "To"] "example.com"
            {
            keep;               # keep in "In" folder
            }

    #
    # Try and catch unsolicited email.  If a message is not to me,
    # or it contains a subject known to be spam, file it away.
    #
    elsif anyof (not address :all :contains
                   ["To", "Cc", "Bcc"] "address@hidden",
                 header :matches "subject"
                   ["*make*money*fast*", "*university*dipl*mas*"])
            {
            # If message header does not contain my address,
            # it's from a list.
            fileinto "spam";   # move to "spam" folder
            }
    else
            {
            # Move all other (non-company) mail to "personal"
            # folder.
            fileinto "personal";
            }

======  Example: hillen-sieve-script-001.txt   ======

require "fileinto";
        if header :is "X-Mailinglist" "suse-linux" {
                fileinto "INBOX.Listen.suse-linux";}
        elsif header :contains "Mailing-List" "reiserfs" {
                fileinto "INBOX.Listen.reiserfs";}
        elsif address :contains :all ["to", "cc", "bcc"] "free-clim" {
                fileinto "INBOX.Listen.free-clim";}
        elsif header :contains "List-Id" "gnupg-users.gnupg.org" {
                fileinto "INBOX.Listen.gnupg";}
        elsif header :is "X-loop" "isdn4linux" {
                fileinto "INBOX.Listen.isdn4linux";}
        elsif header :contains  "Mailing-list" "address@hidden"{
                fileinto "INBOX.Listen.qmail";}
        elsif allof (header :contains "Sender" "address@hidden",
                     address :contains :localpart ["to", "cc", "bcc"] 
"info-cyrus"){
                fileinto "INBOX.Listen.info-cyrus";}
        elsif header :contains "Sender" "address@hidden"{
                fileinto "INBOX.Listen.ntbugtraq";}
        elsif header :is "list-id" "<ietf-mta-filters.imc.org>"{
                fileinto "INBOX.Listen.sieve";}
        elsif header :contains "From" "address@hidden"{
                fileinto "INBOX.Newsletter.securityportal";}
        elsif address :contains :all ["from"] "address@hidden"{
                fileinto "INBOX.Newsletter.ebay";}
        elsif address :contains :all ["to", "cc", "bcc"] "address@hidden"{
                        fileinto "INBOX.Listen.allegro-cl";}
        elsif address :contains :all ["to", "cc", "bcc"] "address@hidden"{
                       fileinto "INBOX.Listen.plob";}
                else {
                         fileinto "INBOX";


======  Example: maro-sieve-script-001.txt   ======

##############################################
# Submitted by Tony Maro
# http://www.maro.net/tony
# Use and abuse this script as you will
# I'm not responsible for what it does...
#
# Save this script in your home directory.
# Install this script with the following command,
# replacing "thisfile" and "yourname" with the appropriate
# information:
#
# installsieve -i thisfile -m PLAIN -u yourname localhost
#
#
require "fileinto";
require "reject";
#
# Circle MUD mailing list list
# All mail from the list has "[CIRCLE]" in the subject
# Place all these in the "Circle List" folder
# I could filter on the mail list senders e-mail as it's always
# the same, but this way I even catch personal replies that come
# directly from a user to me
if header :contains "Subject" "[CIRCLE]" {
        fileinto "INBOX.Circle List";
}
#
# "shockwave" e-mail virus - just reject it
#
if header :contains "Subject" "A great Shockwave flash movie" {
        reject "Possible virus?  Check your system!";
}
#
# Get a lot of junk from dial-up uu.net accounts
# Make sure you create a Junk folder under your INBOX
# I like this one because it catches them even if they
# relay their crap through some international open
# mail relay
#
if header :contains "Received" ".da.uu.net" {
        fileinto "INBOX.Junk";
}
#
# If the mail is listed as TO someone at bigfoot.com
# Then just reject it because it's spam (my experience anyway)
#
if header :contains "To:" "@bigfoot.com" {
        reject "Yeah, right.  Bugoff, hosier!";
}
#
# If the mail is not directed to me put in the junk folder
# be sure to replace address@hidden with the
# appropriate information
# Took me a while to figure out how to do NOT statements... :-}
#
if anyof ( not address :all :contains ["To", "Cc", "Bcc"] "address@hidden" ) {
        fileinto "INBOX.Junk";
}
#
# Everything that makes it to here ends up in the INBOX
########################################################
--
that's "angle" as in geometry