Using Majordomo with qmail

Russ Allbery <eagle@eyrie.org>
Last modified July 1, 2014

              Maintained by Russ Allbery <eagle@eyrie.org>
==========================================================================

If you're reading this on Usenet, this FAQ is formatted as a minimal digest, so if your news or mail reader has digest handling capabilities you can use them to navigate between sections. In rn variants, you can use Ctrl-G to skip to the next section; in Gnus, press Ctrl-D to break each section into a separate article.

The latest version of this FAQ is available on the web at <http://www.eyrie.org/~eagle/faqs/mjqmail.html>. Please send any comments, suggestions, or updates to eagle@eyrie.org.

Contents

  1. Introduction and Overview
    1.1. Why qmail is different
    1.2. Potential problems
    1.3. Stages of setup

  2. Handling incoming list mail
    2.1. The simple approach
    2.2. Eliminating Majordomo's wrapper program
    2.3. Using virtual domains
    2.4. Using qmail-users
    2.5. Cutting down on .qmail files

  3. Sending outgoing list mail
    3.1. The simple approach
    3.2. Privacy concerns with the simple approach
    3.3. Talking to qmail directly
    3.4. Handling digests and archives

  4. Other resources

1. Introduction and Overview

This FAQ is no longer actively maintained. Its author no longer uses either qmail or Majordomo. It is preserved in case anyone still finds it useful, but further updates are not being made and some of the information below is probably out of date.

This FAQ describes how to get Majordomo 1.94 working under a pure qmail system (one without qmsmac or some other emulation of sendmail's /etc/alias handling). A standard Majordomo installation relies heavily on sendmail aliases, and while it is possible to do the obvious thing and set up Majordomo using .qmail files in ~alias instead, there are actually considerably better ways.

This FAQ assumes that you understand how to do a stock installation of Majordomo. This FAQ is not aimed at the Majordomo novice, but rather at someone who has set up Majordomo before under non-qmail systems and now wants to set it up on a qmail system (or upgrade an existing non-qmail system running Majordomo to qmail). For basic questions on how to install Majordomo, see the documentation that comes with it.

It is expected that Majordomo 2.x will handle qmail directly, so this FAQ is only interim documentation. A release date for for Majordomo 2.x has not yet been set.

This FAQ owes a great deal to the documentation and scripts written by Giles Lean and Chael Hall, which I used as a starting point for my own experimentation. Giles also provided valuable feedback, corrections, and rewordings. Information on digests is largely from Lindsay Haisley.

1.1. Why qmail is different

In setting up Majordomo with qmail, there are two classes of things to think about:

qmail is more than just a different MTA (mail transport agent). It also has a fundamentally different philosophy than sendmail in a lot of respects. It's possible, using the information contained in this FAQ, to adjust Majordomo to fit into qmail's philosophy, which in my opinion will result in a much better overall list management system than just making a few minor changes to a Majordomo installation to get it working.

1.2. Potential problems

Before getting into the technical details of setting up Majordomo, you should be aware of some general potential problems with using qmail and Majordomo together.

One of the advantages that qmail has over sendmail is that it's a much faster and more efficient MTA. It receives mail faster and, more to the point, it sends mail faster. Furthermore, in order to support VERP (where every mail message sent to the list has a different envelope sender so that bounces can be easily associated with a subscribed address), it may send each list message as a separate mail message. sendmail under some circumstances does more blocking, sending one message with multiple envelope recipients if all recipients are at the same host.

This can result in qmail putting a much higher load on remote mail servers, particularly if you have a large number of list subscribers at one remote host, and particularly if that remote host is not also running qmail. If you're running large lists on a qmail mail server, you may wish to seriously consider decreasing control/concurrencyremote to something lower than the default of 20 so that qmail won't open 20 simultaneous connections to that remote host. (On the other hand, if the remote hosts can handle it, you may want to consider increasing concurrencyremote so that list mail goes out faster.)

1.3. Stages of setup

There are basically two things one has to worry about to get Majordomo working: routing incoming mail to the correct Majordomo program, and letting Majordomo send outgoing mail to the mailing lists. The remainder of this document describes how to set up each part.

The routing of incoming email to the majordomo programs requires changes, since qmail doesn't use the /etc/aliases method described in the Majordomo installation instructions. The minimal changes required to get Majordomo working are to create .qmail files in ~alias for every list and administrative address. (At this point, experienced Majordomo installers will be wondering how to hide the outgoing alias; how to either do this or protect it from external mail is described later.)

A stock Majordomo installation can send outgoing mail just fine under qmail (with the addition of a .qmail file linked to the list address file so that resend has an address to send to). For reasons of speed, bounce handling and loop detection, you may wish to have Majordomo send mail using a program that talks more directly to the qmail system rather than through qmail's sendmail emulation.

2. Handing incoming list mail

For each mailing list run on your system, you need to be able to accept mail at the list address and an associated owner address (either list-owner or owner-list) at a bare minimum. You may also want to accept mail at list-request (for administrative requests) and list-approval (for subscription and unsubscription notification and approval). The standard Majordomo addresses (majordomo and majordomo-owner) should also work as expected.

This section deals with configuring qmail so that mail to those addresses is routed to the correct Majordomo program.

A default installation of Majordomo uses a wrapper program (written in C and installed setuid root) to change users to the Majordomo user.

2.1. The simple approach

The easiest method to use is qmail's alias mechanism. Just create .qmail files for each of the list administrative and submission addresses for each of your lists in ~alias. The files needed for each list are:

    .qmail-LIST
    .qmail-owner-LIST
    .qmail-LIST-approval

where LIST is the name of the mailing list. These files should contain the standard Majordomo aliases; in other words, .qmail-LIST should have:

    |/path/to/majordomo/wrapper resend -l LIST LIST-outgoing

.qmail-owner-LIST should contain the e-mail address of the owner of the mailing list, and so forth. This is the direct qmail analog of setting up the standard Majordomo aliases in /etc/aliases. (Keep in mind that qmail specifies that all e-mail addresses in .qmail files should be fully qualified.) You may also want to add a .qmail-LIST-request file containing the standard request program invocation.

Finally, create ~alias/.qmail-majordomo containing the standard Majordomo invocation and ~alias/.qmail-majordomo-owner containing the Majordomo administrator's e-mail address.

Chael Hall's mj+qmail package contains full instructions and some scripts to automate this. Look for the latest version of mj+qmail in:

<http://chaos.x-philes.com/pub/development/>

2.2. Eliminating Majordomo's wrapper program

While the above method will certainly work, it still uses Majordomo's wrapper program to switch users from alias to the Majordomo user. One could just not use the wrapper program and let Majordomo run as alias, but this is probably a bad idea from a security standpoint (the alias user may be doing a variety of other things).

qmail already has a facility built in for changing UIDs to the user whose mail is being delivered, and in fact probably does so more robustly than Majordomo's wrapper does. It makes a great deal of sense to eliminate Majordomo's wrapper program entirely under qmail and let qmail do the user switching.

There are two basic ways to do this. The simplest is with the virtual domain system, described in section 2.3, but this requires that you have the ability to set up a DNS MX record for a separate host name that will only be handling mailing list mail. A more complicated solution that doesn't require that is to use the qmail-users mechanism as described in section 2.4.

Under either approach, you'll need to set up .qmail files to handle the incoming mail, which will now be delivered according to .qmail files in the Majordomo user's home directory and with the UID and GID of the delivery program already set to the appropriate user and group. The simplest way to handle this is to create .qmail files in the Majordomo user's home directory similar to those described in section 2.1, except leaving off the wrapper. For example, if Majordomo were installed in /var/qmail/majordomo, .qmail-LIST would just contain:

    |/var/qmail/majordomo/resend -l LIST LIST-outgoing

Also create a .qmail file containing:

    |/var/qmail/majordomo/majordomo

and a .qmail-owner file containing the e-mail address of the Majordomo administrator.

There are two ways to deal with the fact that the standard Majordomo sender address for list mail is owner-LIST instead of the more natural (at least for qmail) LIST-owner. One is to create then create .qmail-owner-LIST files for each list (and add a line to users/append as mentioned in the previous section if you're using qmail-users). Another, more natural approach is to change Majordomo to use LIST-owner. To do that, go through the .config files for each of your mailing lists and change the "sender" setting to LIST-owner instead of the default. You'll also want to install a small to majordomo to make that the default for new lists and for times when the configuration file couldn't be read. You can get that patch from:

<http://www.eyrie.org/~eagle/software/mjqmail/>

The last issue to be aware of in eliminating the wrapper is that the wrapper sets a variety of environment variables which will now no longer be set. The only one that actually causes problems is MAJORDOMO_CF. Since it won't be set, all of the Majordomo programs will fall back on /etc/majordomo.cf, which probably isn't correct. If it's not, edit all Majordomo programs you're running from aliases (resend, majordomo, request-answer, etc.), search for MAJORDOMO_CF, and change the fallback path to the correct path for majordomo.cf.

2.3. Using virtual domains

A virtual domain allows you to delegate an entire domain to a particular user. Using virtual domains requires that you have sufficient control over your DNS to set up a particular hostname just for your mailing lists. You'll need to be able to pick an address, like lists.example.com, and set up an MX record for it pointing to your mail server. If you're not able to do this, use the qmail-users mechanism described in the next section.

Choose the subdomain that you're going to give to Majordomo (such as lists.example.com). Add a line to /var/qmail/control/virtualdomains that looks like:

    lists.example.com:lists

replacing lists.example.com with your virtual domain and lists with your Majordomo user. All addresses in that domain will now be handled by .qmail files in the Majordomo user's home directory (well, after qmail is restarted or qmail-send gets a HUP signal). See the qmail-send man page for more details.

It's possible to use the virtual domain system under qmail 1.03 or later to delegate individual addresses rather than whole domains to a particular user. This would potentially allow one to use virtual domains even without a separate hostname and MX record for mailing lists. That would require listing every individual list address in the virtualdomains file, though, so using the qmail-users mechanism is probably simpler for that case.

2.4. Using qmail-users

The qmail-users man page, along with the documentation for qmail-pw2u and qmail-newu, contains the full details on how the qmail-users mechanism works, but basically qmail-pw2u is run on the password file to generate a list of addresses mail is accepted at, what UID and GID to switch to when delivering mail for a user, and where to deliver the mail (the user's home directory).

Assuming that Majordomo is installed in /var/qmail/majordomo, create /var/qmail/users/mailnames if it doesn't exist and add the following:

    lists:lists:majordomo

(replacing lists with the name of your Majordomo user). This will be used for the standard majordomo and majordomo-owner addresses. Then, for each list, add a line to /var/qmail/users/subusers of the form:

    LIST:lists:LIST:

(again, replacing lists with the name of your Majordomo user and replacing LIST by the name of the list). After this is done and you run qmail-pw2u, all incoming mail to LIST and LIST-anything will be controlled by .qmail files in Majordomo's home directory.

(Note that you may not want the home directory of the Majordomo user to be the same as the Majordomo installation directory, for various reasons. One reason to make them different is that qmail-pw2u likes to have the home directory of a user be owned by that user and you may not want to have the directory containing the main Majordomo scripts owned by the Majordomo user for security reasons.)

If you're not going to patch Majordomo to use LIST-owner addresses instead of owner-LIST addresses, you may also want to add the line:

    +owner-:lists:401:201:/var/qmail/majordomo:-:owner-:

to /var/qmail/users/append, replacing the user, UID, GID, and home directory as appropriate for your installation, so that all mail addressed to owner-anything will be handled by the Majordomo user.

Note that you may be able to avoid using the qmail-users mechanism for all of your addresses by not including the alias lines in users/assign and letting qmail-lspawn fall back on reading /etc/passwd. If you choose to do this, you can just put the lines for Majordomo in users/assign directly and don't need to use qmail-pw2u. See the man page for more details.

2.5. Cutting down on .qmail files

Using the above instructions for setting up the incoming mail addresses, one ends up with three or four .qmail files per list, most of whose contents are algorithmically determinable from the name of the list. If you're like me, that seems like a lot of clutter, and it may be worthwhile to use a program to resend incoming mail to the right place depending on the address rather than using .qmail files and having qmail do it.

One script for doing this for the common cases of -owner, -request, -approval, and the list submission address itself is available from:

<http://www.eyrie.org/~eagle/software/mjqmail/>

Patches and pointers to additional scripts that can handle more cases are welcome.

Using a script like this, one simply creates a .qmail-default file in the Majordomo directory that pipes incoming mail to the script. Note that by doing it that way, if there are any particular addresses which shouldn't go to the script, it's trivial to override them just by creating a separate .qmail file for those special cases. .qmail-default is only checked if no more specific .qmail files exist.

3. Sending outgoing list mail

Once the Majordomo system has received the mail, it has to send it out again to the mailing list. Normally this is done using the Majordomo program resend. (It's possible to have the list submission alias point directly at the file of addresses, but it's not recommended since the envelope sender isn't changed and none of the other things resend does can be done.)

resend by default uses /usr/lib/sendmail to send outgoing mail, which actually works just fine on a qmail system due to qmail's sendmail emulation. resend sends mail to another alias on the system, though, so an address for resend to send mail to has to be created or the way resend sends mail has to be changed.

3.1. The simple approach

The easiest way to get resend working under qmail is to create a .qmail-LIST-outgoing file and link it to the list file that Majordomo maintains (you have to use a symbolic link, since Majordomo will break hard links in the process of updating the address file). In order for this to work, your lists have to be set to strip comments from subscribed addresses (because qmail will complain about addresses with comments). Note also that qmail won't like any addresses which begin with non-alphanumeric characters, which may be a problem in some circumstances.

A patch to Majordomo that causes it to reject any addresses that don't start with alphanumeric characters was available as part of Chael Hall's mj+qmail package at:

<http://chaos.x-philes.com/pub/development/>

3.2. Privacy concerns with the simple approach

The problem with doing things this way is that qmail will insert a Delivered-To header pointing at the address for the raw list file into each outgoing message, and someone will be able to send mail directly to the raw list address and that mail will bypass resend. This is just an annoyance if you're running fairly open lists; this is a fatal problem for lists that are moderated or which use extensive spam filtering.

There are a couple of ways to deal with this problem. One is to add a filter as the first subscriber to every mailing list that checks incoming mail and rejects it if it wasn't sent by Majordomo. A good filter which does this is available from Giles Lean's majordomo-inject page (see the reference below).

Another, perhaps better, solution is to tell Majordomo to use something other than qmail's sendmail emulation to send outgoing mail. Details on how to do that are in the next section.

In addition to the problem with the exposed raw list address, one may wish to use something other than the sendmail emulation so that VERP envelope senders can be added to outgoing mail. It's possible to convince qmail to add VERP using the simple approach by creating .qmail-LIST-outgoing-owner and .qmail-LIST-outgoing-owner-default, but using a script to do it is cleaner in some ways and ensures that the envelope sender is set to what resend wants, just with VERP added, rather than LIST-outgoing-owner.

Another reason to use a separate mailing script is so that you aren't forced into having Majordomo strip comments from the list files; a sufficiently intelligent injection script could do that.

3.3. Talking to qmail directly

I have an mjinject script which can be used as the outgoing mailer for resend. It talks to qmail-queue directly (which is considerably more efficient than talking to the sendmail emulation or to qmail-inject), and optionally adds VERP to outgoing messages. It also adds a Delivered-To header to outgoing messages pointing to the submission address for the list, which should hopefully be effective in preventing loops. It's available at:

<http://www.eyrie.org/~eagle/software/mjqmail/>

This script is a merger of Giles Lean's and my work, and he contributed substantially to it.

As an aside, while VERP can be extremely handy in tracking down where mailing list bounces are coming from, some defective MTAs can't handle VERP addresses. In particular, I've received a report that WatchGuard firewalls reject them with "mailbox name not allowed." So if you try turning on VERP and suddenly start getting bounces from your mailing list, you may want to see if any of your subscribers are behind one of those broken MTAs.

3.4. Handling digests and archives

First of all, when doing digests under qmail, you will almost certainly want to handle digests as another Majordomo list. The stand-alone method of configuration would require a lot of porting to qmail, and why go to the effort when you've already ported Majordomo proper?

Digests and archives are both usually created just by adding another recipient to outgoing list mail, so one obvious way of dealing with them is to add them as additional addresses subscribed to the mailing list. To avoid confusing Majordomo (and mjinject), however, you want them to be simple addresses, which means that you would have to have potentially exposed addresses on your system going into the digest and archive scripts which would have to be protected.

If that isn't likely to be a problem, then creating .qmail-LIST-archive and .qmail-LIST-digestify files in Majordomo's home directory and then adding LIST-archive and LIST-digestify to the mailing list will cause digests and archives to work about the same as they would work under a sendmail Majordomo setup.

Another option is to add the invocation of digest to a .qmail-LIST file in majordomo's home directory (creating one to override mjdispatch or a similar program if you're using one), in addition to the invocation of resend. This will cause incoming messages as seen by digest to have not gone through resend, which may or may not be what you want. (It almost certainly isn't what you want if you have a moderated list.)

A third option is to use the digest and archive support in mjinject (see above). If you're using it, and create a LIST.programs file in the lists directory containing the command lines to run archive and/or digest, mjinject will pipe the message (coming from resend) through those programs as well as mailing it out to the list. You can then use your standard way of periodically sending out digests.

Note that the digest script itself uses the mailer setting from majordomo.cf, so if you've modified that to use qmail-inject, digest will as well, which in practice will require that digest be sending to a regular Majordomo list.

4. Other resources

qmail itself, as well as links to many other packages that work with it or add onto it and quite possibly links to other ways of making it work with Majordomo, is at:

<http://www.qmail.org/>

The current version of Majordomo is available from:

<http://www.greatcircle.com/majordomo/>

I've benefited much from the discussions of qmail and how to make it work in various situations and with various packages that have taken place on the qmail mailing list. It's fairly high volume, but has a lot of good information. You can subscribe by sending mail to:

<qmail-subscribe@list.cr.yp.to>

If you're starting fresh and setting up a mailing list manager for the first time, you may wish to use a system that's more "native" to qmail. ezmlm (written by the same person who wrote qmail) is available at:

<http://cr.yp.to/ezmlm.html>

with an extended version, ezmlm-idx, available from:

<http://www.ezmlm.org/>

Converted to XHTML by faq2html version 1.32