Set Up a Mail Server on Linux

A mail server is a computer on the network that acts as a virtual post office for emails. This article explains how to set up an email server on Gentoo Linux using Postfix and Dovecot. You can choose any other Linux distro for this purpose, provided you m

Postfix is a Mail Transfer Agent (MTA). It waits for mail from Mail User Agents (MUA) from the local network or the outside world. SMTP is a complex protocol that is used both for sending and receiving mails. Dovecot is an IMAP/POP server that can be used to store and retrieve mails. IMAP stands for Internet Message Access Protocol and is offered by many existing email services like Gmail, Windows Live Mail, etc. POP (POP3) is the Post Office Protocol that was used before IMAP came into existence.

The primary difference between IMAP and POP is that, in the former, you need not download all messages to the client and you can manage multiple folders on the server. In case of POP, you have just one folder — the inbox—and the client must download all messages. This doesn't mean you can't download messages using IMAP, though. IMAP is just more convenient than POP when it comes to handling mails.

The flow of email

Let me explain the flow of email shown in Figure 1: The Internet, obviously, is the public facing side of your server. Postfix operates on two protocols, namely Submission and SMTP (Simple Mail Transfer Protocol). Submission is the same as SMTP, the only difference being that SMTP operates by default on TCP port 25 while Submission operates on TCP port 587. This enables naive users to send emails via the Submission ports using MUAs because many ISPs block port 25. There's also SMTPS— the SSL variant of the SMTP protocol that usually operates on TCP port 465. SMTP is a clear text protocol, and in this case, just the data is encrypted before transmission and decrypted after transmission at the server. There are a couple of reasons why the Submission port exists. You can read more about it at Wikipedia - http://en.wikipedia.org/wiki/ Mail_submission_agent LMTP (Local Mail Transfer Protocol): This is used by Postfix to transfer mails to the particular account on Dovecot. Amavsid, ClamAV and SpamAssassin: ClamAV and SpamAssassin are virus and spam filters, respectively, but in order to use both simultaneously, we need Amavisd, which acts as a forwarder. PostgreSQL: This is the database used for storing various

mail related data like accounts, aliases, quota information, etc. It's not mandatory to use PostgreSQL. You can use anything else that's supported. I chose it because of my previous experience with it and my future plans. Consider a typical case of when you are trying to send a mail using a mail client like Thunderbird. After you have supplied the account information to Thunderbird, when you hit the Send button, it will contact Postfix to send the mail. Before sending a mail, you must be authenticated. This authentication is usually accomplished using SASL (Simple Authentication and Security Layer).

Postfix has the ability to use Dovecot as an authentication backend, so that it can verify the username and password. Once that is done, Postfix will let you send the messages. With each message, it will check whom it is addressed to. There is a provision in Postfix to reduce the possibility of spam by allowing authenticated users to send mails only from their own addresses or any of the aliases they own.

Now, Postfix will forward the mail to Amavisd via SMTP on a separate port. Amavisd then calls ClamAV (or any other virus scanner) and SpamAssassin to verify that the mail doesn't contain a virus or any spam. A mail server sending spam to other servers can be blacklisted and this has serious repercussions—getting a server whitelisted again is a difficult task.

On completing verification by Amavisd, it will send back the mail to Postfix on a separate port using the SMTP protocol, or quarantine/discard the mail (depending on the configuration).

If Amavisd forwards the mail back to Postfix, it is queued for delivery. Postfix looks up the domain name's MX (Mail Exchange) DNS records. The mail exchange records simply specify the servers that would accept mail for the domain, and their priorities (used for redundancy). In case the MX record is not found for the target domain, the A record is used.

Postfix will try each server listed in the MX records one by one to deliver mail to them. If all the servers refuse to accept mail, the mail just lies in the queue and Postfix keeps trying periodically till the queue expires (by default, the queue expiry time is five days—for undelivered messages, it can be changed).

Coming to Dovecot, it is a powerful IMAP/POP server with loads of features and is designed for performance.

In traditional UNIX mail systems, there used to be just one MTA (something like Postfix), which would take care of delivering mail to a Maildir or MBOX file on the local system. Users would use their shell accounts to read mail using the command line clients.

The traditional model is quite inconvenient and you need to give shell accounts to each user, which is not a good idea. Dovecot can be combined with Postfix to set up Virtual Domain email hosting, by which a particular server can accept and store/retrieve emails for any number of domains (well, not really unlimited—it depends on the hardware).

Configuring the PostgreSQL database

As said earlier, PostgreSQL is used for storing accounts, aliases and related data. There are a couple of tools like postfixadmin and the like, which let you manage your mail server.

These control panels or tools propose their own database schema (structure). I did try them out but I wasn't satisfied with the features they offered. So I set out to use my own schema and write something like a control panel later.

Table: domains

Our domains table has three fields —ID, name and active. ID is the domain number, used as reference in other tables. Name is the domain’s name. Active is a Boolean stating whether the server should accept mail for the domain or not. create table domains ( id bigserial primary key, name character varying (50), active boolean, unique(name));

View: active_domains

We created a domains table earlier, but for performance reasons and to simplify queries in configuration files, use a view that lists only active domains:

Table: users

The users table holds information about users in your mail system - for every domain: create table users ( id bigserial primary key, domain_id integer not null references domains (id) on update restrict on delete cascade, name character varying (50) not null, password character(106) not null, active boolean not null default true, quota_kbytes integer not null default 0, unique(name, domain_id) );

The password field holds the SHA512-CRYPT hash produced by Dovecot as we’ll see later when configuring Dovecot. The hash is exactly 106 characters long.

The quota_kbytes is used for deciding the amount of IMAP storage the user gets in kilobytes.

We need two views for the users table, the actual purpose of which will be understood when we configure Dovecot.

View: active_users_passdb

SELECT 999 AS userdb_uid, 992 AS userdb_gid,

'/var/vmail/' || d.name || '/' || u.name::text AS userdb_ home, '*:storage=' || u.quota_kbytes AS userdb_quota_rule, u.name || '@' || d.name AS "user", u.password FROM users u JOIN domains d ON u.domain_id = d.id AND u.active = true AND d.active = true;

999 is the ID of the user that will store all virtual domains data. 992 is the ID of the group to which users storing all data belong.

That will change depending on your distribution, so before creating this you need to create your vmail user and check for the uid and gid.

The quota rule will be explained in Dovecot configuration.

View: active_users_userdb

SELECT 999 AS uid, 992 AS gid, '/var/vmail/' || d.name || '/' || u.name AS home, u.name || '@' || d.name AS "user" FROM users u JOIN domains d ON u.domain_id = d.id AND u.active = true AND d.active = true;

Table: user_aliases

This table is used for storing aliases to user accounts: create table user_aliases ( id bigserial primary key, domain_id integer not null references domains(id) on update restrict on delete cascade, name character varying(50) not null, destination character varying(50) not null, active boolean not null default true, unique(name, domain_id), foreign key(destination, domain_id) references users(name, domain_id) on update restrict on delete cascade);

We have two triggers here to avoid conflicting alias and usernames, and to check that a user exists when an alias is created. In PostgreSQL, the procedures that must be executed by a trigger are created first: CREATE OR REPLACE FUNCTION check_duplicate_user_for_alias_ name() RETURNS trigger LANGUAGE plpgsql AS $function$begin perform 1 from users where name = NEW.name and domain_id = NEW.domain_id; if found then raise exception 'User with name % already exists for domain % and therefore cannot create alias', NEW.name, NEW.domain_id; end if;

return NEW; end; $function$ CREATE OR REPLACE FUNCTION check_user_existence_for_user_ alias() RETURNS trigger LANGUAGE plpgsql AS $function$begin perform 1 from users where name = NEW.destination and domain_id = NEW.domain_id; if not found then raise exception 'User % not found for creating alias %', NEW.destination, NEW.name; end if;

return NEW;

end; $function$

Trigger creation code: CREATE OR REPLACE TRIGGER check_duplicate_user_for_alias_name BEFORE INSERT OR UPDATE OF name, domain_id ON user_aliases FOR EACH ROW EXECUTE PROCEDURE check_duplicate_user_for_ alias_name(); CREATE OR REPLACE TRIGGER check_user_existence BEFORE INSERT OR UPDATE OF destination, domain_id ON user_aliases FOR EACH ROW EXECUTE PROCEDURE check_user_existence_for_user_alias();

Table: remote_aliases

This table is used for storing aliases that are not forwarded to local users. That is, if foo@example.com is an alias and it's forwarded to abc@gmail.com, then that data is stored here: create table remote_aliases ( id bigserial primary key, domain_id integer not null references domains(id) on update restrict on delete cascade, name character varying(50) not null, destination character varying(50) not null, active boolean not null default true, unique(name, domain_id));

Again, a trigger is needed: CREATE OR REPLACE TRIGGER check_duplicate_user_for_alias_name BEFORE INSERT OR UPDATE OF name, domain_id ON remote_aliases FOR EACH ROW EXECUTE PROCEDURE check_duplicate_user_for_ alias_name();

Table: quota

create table quota ( username character varying(100) primary key, bytes bigint not null default 0, messages integer not null default 0);

The trigger for the table is: CREATE OR REPLACE FUNCTION merge_quota() RETURNS trigger LANGUAGE plpgsql AS $function$ BEGIN IF NEW.messages < 0 OR NEW.messages IS NULL THEN

-- ugly kludge: we came here from this function, really do try to insert IF NEW.messages IS NULL THEN

NEW.messages = 0; ELSE

NEW.messages = -NEW.messages; END IF; return NEW; END IF; LOOP UPDATE quota SET bytes = bytes + NEW.bytes, messages = messages + NEW.messages WHERE username = NEW.username; IF found THEN

RETURN NULL; END IF; BEGIN IF NEW.messages = 0 THEN INSERT INTO quota (bytes, messages, username)

VALUES (NEW.bytes, NULL, NEW.username); ELSE INSERT INTO quota (bytes, messages, username)

VALUES (NEW.bytes, -NEW.messages, NEW.username); END IF; return NULL; EXCEPTION WHEN unique_violation THEN

-- someone just inserted the record, update it END; END LOOP; END; $function$ CREATE OR REPLACE TRIGGER mergequota BEFORE INSERT ON quota FOR EACH ROW EXECUTE PROCEDURE merge_quota();

The above trigger code was taken from the Dovecot wiki http://wiki2.dovecot.org/Quota/Configuration

The quota table is used by Dovecot to track the amount of storage used.

Table: expires

create table expires ( username character varying(100) primary key, mailbox character varying(255) not null, expire_stamp integer not null);

Trigger for the table - CREATE OR REPLACE FUNCTION merge_expires() RETURNS trigger LANGUAGE plpgsql AS $function$ BEGIN

UPDATE expires SET expire_stamp = NEW.expire_stamp

WHERE username = NEW.username AND mailbox = NEW.mailbox; IF FOUND THEN

RETURN NULL; ELSE

RETURN NEW; END IF; END; $function$ CREATE OR REPLACE TRIGGER mergeexpires BEFORE INSERT ON expires FOR EACH ROW EXECUTE PROCEDURE merge_expires(); The above trigger code was taken from Dovecot Wiki http:// wiki2.dovecot.org/Plugins/Expire Expires table is used for deleting old mail from mailboxes. Particularly Trash and Spam, like every 30 days or so.

Function: sender_bcc_map

This function is optional. In my set-up, Postfix is configured to send a BCC to <sender>+Sent@<domain> for every mail sent as an authenticated user. This later gets classified as ‘Sent' mail in Dovecot. If you don't want the server generating a sent-mail copy automatically, you don't need this. CREATE OR REPLACE FUNCTION sender_bcc_map(sender character varying) RETURNS SETOF character varying LANGUAGE plpgsql AS $function$declare username text; domainname text; dom_id domains.id%type := 0; s_username text; begin username := split_part(sender, '@', 1); domainname := split_part(sender, '@', 2); perform 1 from users where name = username and domain_id = (select id from domains where name = domainname); if found then s_username := username || '+Sent'; return next s_username || '@' || domainname; end if; select destination || '+Sent' into s_username from user_ aliases where name = username and domain_id = (select id from domains where name = domainname); if found then return next s_username || '@' || domainname; end if;

return;

end;$function$

Function: sasl_login

This function is used by Postfix to check whether a sender is authorised to send the message at the address specified by the user. The function returns a list of addresses that the particular authorised user may send: CREATE OR REPLACE FUNCTION vmail.sasl_login(sender character varying) RETURNS SETOF character varying LANGUAGE plpgsql AS $function$declare s_username vmail.users.name%type; s_domainname vmail.domains.name%type; begin perform 1 from vmail.active_users_userdb t where t.user = sender; if found then return next sender; end if; select ua.destination, d.name into s_username, s_domainname from vmail.user_aliases ua join vmail.domains d on ua.domain_id = d.id where ua.name = split_part(sender, '@', 1) and d.name = split_part(sender, '@', 2); if found then return next s_username || '@' || s_domainname; end if; return; end;$function$

We're now done with the PostgreSQL setup. That is a lot of SQL code. But, there is still something missing—we need to delete aliases to a user when a user is deleted, as well as check for an existing alias when a new user is created. Both of these are left for readers to try out as an exercise. :-)

In the next part, we'll look at how to configure the other aspects of a mail system starting with Dovecot.