Previous   Next   Contents       (Exim 4.10 Specification)

25. The appendfile transport

The appendfile transport delivers a message by appending it to a file in the local file system, or by creating an entirely new file in a specified directory. Single files to which messages are appended can be in the traditional Unix mailbox format, or optionally in the MBX format supported by the Pine MUA and University of Washington IMAP daemon, inter alia. When each message is being delivered as a separate file, ``maildir'' format can optionally be used to give added protection against failures that happen part-way through the delivery. A third form of separate-file delivery known as ``mailstore'' is also supported. For all file formats, Exim attempts to create as many levels of directory as necessary, provided that create_directory is set.

The code for the optional formats is not included in the Exim binary by default. It is necessary to set SUPPORT_MBX, SUPPORT_MAILDIR and/or SUPPORT_MAILSTORE in Local/Makefile to have the appropriate code included.

Exim recognises system quota errors, and generates an appropriate message. Exim also supports its own quota control within the transport, for use when the system facility is unavailable or cannot be used for some reason.

If there is an error while appending to a file (for example, quota exceeded or partition filled), Exim attempts to reset the file's length and last modification time back to what they were before. If there is an error while creating an entirely new file, the new file is removed.

appendfile is most commonly used for local deliveries to users' mailboxes. However, it can also be used as a pseudo-remote transport for putting messages into files for remote delivery by some means other than Exim. ``Batch SMTP'' format is often used in this case (see the use_bsmtp option). appendfile is also used for delivering messages to files or directories whose names are obtained directly from alias, forwarding, or filtering operations. In these cases, $local_part contains the local part that was aliased or forwarded, while $address_file contains the name of the file or directory.

Before appending to a file, a number of security checks are made, and the file is locked. A detailed description is given below, after the list of private options.

25.1. Private options for appendfile


allow_fifo

Type:  boolean
Default:  false

Setting this option permits delivery to named pipes (FIFOs) as well as to regular files. If no process is reading the named pipe at delivery time, the delivery is deferred.


allow_symlink

Type:  boolean
Default:  false

By default, appendfile will not deliver if the path name for the file is that of a symbolic link. Setting this option relaxes that constraint, but there are security issues involved in the use of symbolic links. Be sure you know what you are doing if you set this. Details of exactly what this option affects are included in the discussion which follows this list of options.


batch_id

Type:  string, expanded
Default:  unset

See the description of local delivery batching in chapter 24.


batch_max

Type:  integer
Default:  1

See the description of local delivery batching in chapter 24.


check_group

Type:  boolean
Default:  false

When this option is set, the group owner of the file defined by the file option is checked to see that it is the same as the group under which the delivery process is running. The default setting is false because the default file mode is 0600, which means that the group is irrelevant.


check_owner

Type:  boolean
Default:  true

When this option is set, the owner of the file defined by the file option is checked to ensure that it is the same as the user under which the delivery process is running.


check_string

Type:  string
Default:  see below

As appendfile writes the message, the start of each line is tested for matching check_string, and if it does, the initial matching characters are replaced by the contents of escape_string. The value of check_string is a literal string, not a regular expression, and the case of any letters it contains is significant.

If use_bsmtp is set the values of check_string and escape_string are forced to ``.'' and ``..'' respectively, and any settings in the configuration are ignored. Otherwise, they default to ``From '' and ``>From '' when the file option is set, and unset when the directory option is set.

The default settings, along with message_prefix and message_suffix, are suitable for traditional ``BSD'' mailboxes, where a line beginning with ``From '' indicates the start of a new message. All four options need changing if another format is used. For example, to deliver to mailboxes in MMDF format:

  check_string = "\1\1\1\1\n"
  escape_string = "\1\1\1\1 \n"
  message_prefix = "\1\1\1\1\n"
  message_suffix = "\1\1\1\1\n"

create_directory

Type:  boolean
Default:  true

When this option is true, Exim attempts to create any missing superior directories for the file that it is about to write. A created directory's mode is given by the directory_mode option.


create_file

Type:  string
Default:  anywhere

This option constrains the location of files and directories that are created by this transport. It applies to files defined by the file option and directories defined by the directory option. In the case of maildir delivery, it applies to the top level directory, not the maildir directories beneath.

The option must be set to one of the words ``anywhere'', ``inhome'', or ``belowhome''. In the second and third cases, a home directory must have been set for the transport. This option is not useful when an explicit file name is given for normal mailbox deliveries. It is intended for the case when file names are generated from users' .forward files. These are usually handled by an appendfile transport called address_file. See also file_must_exist.


directory

Type:  string, expanded
Default:  unset

This option is mutually exclusive with the file option. When it is set, the string is expanded, and the message is delivered into a new file or files in or below the given directory, instead of being appended to a single mailbox file. A number of different formats are provided (see maildir_format and mailstore_format), and see section 25.3 for further details of this form of delivery.


directory_file

Type:  string, expanded
Default:  q${base62:$tod_epoch}-$inode

When directory is set, but neither maildir_format nor mailstore_format is set, appendfile delivers each message into a file whose name is obtained by expanding this string. The default value generates a unique name from the current time, in base 62 form, and the inode of the file. The variable $inode is available only when expanding this option.


directory_mode

Type:  octal integer
Default:  0700

If appendfile creates any directories as a result of the create_directory option, their mode is specified by this option.


escape_string

Type:  string
Default:  see description

See check_string above.


file

Type:  string, expanded
Default:  unset

This option is mutually exclusive with the directory option. It should not be set when appendfile is being used to deliver to files whose names are obtained from forwarding, filtering, or aliasing address expansions (by default under the instance name address_file), because in those cases the file name is already associated with the address. If file is set under these circumstances, its value is ignored.

Otherwise, the file option must be set unless the directory option is set. Either use_fcntl_lock or use_lockfile (or both) must be set with file. If you are using more than one host to deliver over NFS into the same mailboxes, you should always use lock files.

The string value is expanded for each delivery, and must yield an absolute path. The most common settings of this option are variations on one of these examples:

  file = /var/spool/mail/$local_part
  file = /home/$local_part/inbox
  file = $home/inbox

In the first example, all deliveries are done into the same directory. If Exim is configured to use lock files (see use_lockfile below) it must be able to create a file in the directory, so the ``sticky'' bit must be turned on for deliveries to be possible, or alternatively the group option can be used to run the delivery under a group id which has write access to the directory.


file_format

Type:  string
Default:  unset

This option requests the transport to check the format of an existing file before adding to it. The check consists of matching a specific string at the start of the file. The value of the option consists of an even number of colon-separated strings. The first of each pair is the test string, and the second is the name of a transport. If the transport associated with a matched string is not the current transport, control is passed over to the other transport. For example, suppose the standard local_delivery transport has this added to it:

  file_format = "From       : local_delivery :\
                 \1\1\1\1\n : local_mmdf_delivery"

Mailboxes that begin with ``From'' are still handled by this transport, but if a mailbox begins with four binary ones followed by a newline, control is passed to a transport called local_mmdf_delivery, which presumably is configured to do the delivery in MMDF format. If a mailbox does not exist or is empty, it is assumed to match the current transport. If the start of a mailbox doesn't match any string, or if the transport named for a given string is not defined, delivery is deferred.


file_must_exist

Type:  boolean
Default:  false

If this option is true, the file specified by the file option must exist, and an error occurs if it does not. Otherwise, it is created if it does not exist.


lock_fcntl_timeout

Type:  time
Default:  0s

By default, the appendfile transport uses non-blocking calls to fcntl() when locking an open mailbox file. If the call fails, the delivery process sleeps for lock_interval and tries again, up to lock_retries times. Non-blocking calls are used so that the file is not kept open during the wait for the lock; the reason for this is to make it as safe as possible for deliveries over NFS in the case when processes might be accessing an NFS mailbox without using a lock file. This should not be done, but misunderstandings and hence misconfigurations are not unknown.

On a busy system, however, the performance of a non-blocking lock approach is not as good as using a blocking lock with a timeout. In this case, the waiting is done inside the system call, and Exim's delivery process acquires the lock and can proceed as soon as the previous lock holder releases it.

If lock_fcntl_timeout is set to a non-zero time, blocking locks, with that timeout, are used. There may still be some retrying: the maximum number of retries is

  (lock_retries * lock_interval) / lock_fcntl_timeout

rounded up to the next whole number. In other words, the total time during which appendfile is trying to get a lock is roughly the same, unless lock_fcntl_timeout is set very large.

You should consider setting this option if you are getting a lot of delayed local deliveries because of errors of the form

  failed to lock mailbox /some/file (fcntl)

lock_interval

Type:  time
Default:  3s

This specifies the time to wait between attempts to lock the file. See below for details of locking.


lock_retries

Type:  integer
Default:  10

This specifies the maximum number of attempts to lock the file. A value of zero is treated as 1. See below for details of locking.


lockfile_mode

Type:  octal integer
Default:  0600

This specifies the mode of the created lock file, when a lock file is being used (see use_lockfile).


lockfile_timeout

Type:  time
Default:  30m

When a lock file is being used (see use_lockfile), if a lock file already exists and is older than this value, it is assumed to have been left behind by accident, and Exim attempts to remove it.


maildir_format

Type:  boolean
Default:  false

If this option is set with the directory option, the delivery is into a new file in the ``maildir'' format that is used by other mail software. The option is available only if SUPPORT_MAILDIR is present in Local/Makefile. See section 25.3 below for further details.


maildir_retries

Type:  integer
Default:  10

This option specifies the number of times to retry when writing a file in ``maildir'' format. See section 25.3 below.


maildir_tag

Type:  string, expanded
Default:  unset

This option applies only to deliveries in maildir format, and is described in section 25.3 below.


mailstore_format

Type:  boolean
Default:  false

If this option is set with the directory option, the delivery is into two new files in ``mailstore'' format. The option is available only if SUPPORT_MAILSTORE is present in Local/Makefile. See section 25.3 below for further details.


mailstore_prefix

Type:  string, expanded
Default:  unset

This option applies only to deliveries in mailstore format, and is described in section 25.3 below.


mailstore_suffix

Type:  string, expanded
Default:  unset

This option applies only to deliveries in mailstore format, and is described in section 25.3 below.


mbx_format

Type:  boolean
Default:  false

This option is available only if Exim has been compiled with SUPPORT_MBX set in Local/Makefile. If mbx_format is set with the file option, the message is appended to the mailbox file in MBX format instead of traditional Unix format. This format is supported by Pine4 and its associated IMAP and POP daemons, by means of the c-client library that they all use. The message_prefix and message_suffix options are not automatically changed by the use of mbx_format; they should normally be set empty.

If none of the locking options are mentioned in the configuration, use_mbx_lock is assumed and the other locking options default to false. It is possible to specify the other kinds of locking with mbx_format, but use_fcntl_lock and use_mbx_lock are mutually exclusive. MBX locking interworks with c-client, providing for shared access to the mailbox. It should not be used if any program that does not use this form of locking is going to access the mailbox, nor should it be used if the mailbox file is NFS mounted, because it works only when the mailbox is accessed from a single host.

If you set use_fcntl_lock with an MBX-format mailbox, you cannot use the standard version of c-client, because as long as it has a mailbox open (this means for the whole of a Pine or IMAP session), Exim will not be able to append messages to it.


message_prefix

Type:  string, expanded
Default:  see below

The string specified here is expanded and output at the start of every message. The default is unset unless file is specified and use_bsmtp is not set, in which case it is:

  message_prefix = "From ${if def:return_path{$return_path}\
    {MAILER-DAEMON}} $tod_bsdinbox\n"

message_suffix

Type:  string, expanded
Default:  see below

The string specified here is expanded and output at the end of every message. The default is unset unless file is specified and use_bsmtp is not set, in which case it is a single newline character. The suffix can be suppressed by setting

  message_suffix =

mode

Type:  octal integer
Default:  0600

If the output file is created, it is given this mode. If it already exists and has wider permissions, they are reduced to this mode. If it has narrower permissions, an error occurs unless mode_fail_narrower is false. However, if the delivery is the result of a save command in a filter file specifing a particular mode, the mode of the output file is always forced to take that value, and this option is ignored.


mode_fail_narrower

Type:  boolean
Default:  true

This option applies in the case when an existing mailbox file has a narrower mode than that specified by the mode option. If mode_fail_narrower is true, the delivery is deferred (``mailbox has the wrong mode''); otherwise Exim continues with the delivery attempt, using the existing mode of the file.


notify_comsat

Type:  boolean
Default:  false

If this option is true, the comsat daemon is notified after every successful delivery to a user mailbox. This is the daemon that notifies logged on users about incoming mail.


quota

Type:  string, expanded
Default:  unset

This option imposes a limit on the size of the file to which Exim is appending, or to the total space used in the directory tree when the directory option is set. In the latter case, computation of the space used is expensive, because all the files in the directory (and any sub-directories) have to be individually inspected and their sizes summed (but see quota_size_regex below). Also, there is no interlock against two simultaneous deliveries into a multi-file mailbox. For single-file mailboxes, of course, an interlock is a necessity.

A file's size is take as its used value. Because of blocking effects, this may be a lot less than the actual amount of disc space allocated to the file. If the sizes of a number of files are being added up, the rounding effect can become quite noticeable, especially on systems that have large block sizes. Nevertheless, it seems best to stick to the used figure, because this is the obvious value which users understand most easily.

The value of the option is expanded, and must then be a numerical value (decimal point allowed), optionally followed by one of the letters K or M. A value of zero unsets the option. The expansion happens while Exim is running as root, before it changes uid for the delivery. This means that files which are inaccessible to the end user can be used to hold quota values that are looked up in the expansion. When delivery fails because this quota is exceeded, the handling of the error is as for system quota failures.

By default, Exim's quota checking mimics system quotas, and restricts the mailbox to the specified maximum size, though the value is not accurate to the last byte, owing to separator lines and additional headers that may get added during message delivery. When a mailbox is nearly full, large messages may get refused even though small ones are accepted, because the size of the current message is added to the quota when the check is made. This behaviour can be changed by setting quota_is_inclusive false. When this is done, the check for exceeding the quota does not include the current message. Thus, deliveries continue until the quota has been exceeded; thereafter, no further messages are delivered. See also quota_warn_threshold.


quota_filecount

Type:  string, expanded
Default:  0

This option applies when the directory option is set. It limits the total number of files in the directory (compare the inode limit in system quotas). It can only be used if quota is also set. The value is expanded; an expansion failure causes delivery to be deferred.


quota_is_inclusive

Type:  boolean
Default:  true

See quota above.


quota_size_regex

Type:  string
Default:  unset

This option applies when one of the delivery modes that writes a separate file for each message is being used. When Exim wants to find the size of one of these files in order to test the quota, it first checks quota_size_regex. If this is set to a regular expression that matches the file name, and it captures one string, that string is interpreted as a representation of the file's size. The value of quota_size_regex is not expanded.

This feature is useful only when users have no shell access to their mailboxes - otherwise they could defeat the quota simply by renaming the files. This facility can be used with maildir deliveries, by setting maildir_tag to add the file length to the file name. For example:

  maildir_tag = ,S=$message_size
  quota_size_regex = ,S=(\d+)

The regular expression should not assume that the length is at the end of the file name (even though maildir_tag puts it there) because maildir MUAs sometimes add other information onto the ends of message file names.


quota_warn_message

Type:  string, expanded
Default:  see below

See below for the use of this option. If it is not set when quota_warn_threshold is set, it defaults to

  quota_warn_message = "\
    To: $local_part@$domain\n\
    Subject: Your mailbox\n\n\
    This message is automatically created \
    by mail delivery software.\n\n\
    The size of your mailbox has exceeded \
    a warning threshold that is\n\
    set by the system administrator.\n"

quota_warn_threshold

Type:  string, expanded
Default:  0

This option is expanded in the same way as quota (see above). If the resulting value is greater than zero, and delivery of the message causes the size of the file or total space in the directory tree to cross the given threshold, a warning message is sent. If quota is also set, the threshold may be specified as a percentage of it by following the value with a percent sign. For example:

  quota = 10M
  quota_warn_threshold = 75%

If quota is not set, a setting of quota_warn_threshold that ends with a percent sign is ignored.

The warning message itself is specified by the quota_warn_message option, and it must start with a To: header line containing the recipient(s). A Subject: line should also normally be supplied. The quota option does not have to be set in order to use this option; they are independent of one another except when the threshold is specified as a percentage.


use_bsmtp

Type:  boolean
Default:  false

If this option is set true, appendfile writes messages in ``batch SMTP'' format, with the envelope sender and recipient(s) included as SMTP commands. If you want to include a leading HELO command with such messages, you can do so by setting the message_prefix option. See section 42.8 for details of batch SMTP.


use_crlf

Type:  boolean
Default:  false

This option causes lines to be terminated with the two-character CRLF sequence (carriage return, linefeed) instead of just a linefeed character. In the case of batched SMTP, the byte sequence written to the file is then an exact image of what would be sent down a real SMTP connection.

The contents of the message_prefix and message_suffix options are written verbatim, so must contain their own carriage return characters if these are needed. In cases where these options have non-empty defaults, the values end with a single linefeed, so they almost always need to be changed to end with \r\n if use_crlf is set.


use_fcntl_lock

Type:  boolean
Default:  see below

This option controls the use of the fcntl() function to lock a file for exclusive use when a message is being appended. It is set by default unless use_mbx_lock is set. Otherwise, it should be turned off only if you know that all your MUAs use lock file locking. When use_fcntl_lock is off, use_lockfile must be on if mbx_format is not set.


use_lockfile

Type:  boolean
Default:  see below

If this option is turned off, Exim does not attempt to create a lock file when appending to a mailbox file. In this situation, the only locking is by fcntl(). You should only turn use_lockfile off if you are absolutely sure that every MUA that is ever going to look at your users' mailboxes uses fcntl() rather than a lock file, and even then only when you are not delivering over NFS from more than one host.

In order to append to an NFS file safely from more than one host, it is necessary to take out a lock before opening the file, and the lock file achieves this. Otherwise, even with fcntl() locking, there is a risk of file corruption.

The use_lockfile option is set by default unless use_mbx_lock is set. It is not possible to turn both use_lockfile and use_fcntl_lock off, except when mbx_format is set.


use_mbx_lock

Type:  boolean
Default:  see below

This option is available only if Exim has been compiled with SUPPORT_MBX set in Local/Makefile. Setting the option specifies that special MBX locking rules be used. It is set by default if mbx_format is set and none of the locking options are mentioned in the configuration. The locking rules are the same as are used by the c-client library that underlies Pine and the IMAP4 and POP daemons that come with it (see the discussion below). The rules allow for shared access to the mailbox. However, this kind of locking does not work when the mailbox is NFS mounted.



25.2. Operational details for appending

Before appending to a file, the following preparations are made:

At the end of delivery, Exim closes the file (which releases the fcntl() lock) and then deletes the lock file if one was created.

25.3. Operational details for delivery to a new file

When the directory option is set, each message is delivered into a newly-created file or set of files. No locking is required while writing the message, so the various locking options of the transport are ignored. The ``From'' line that by default separates messages in a single file is not normally needed, nor is the escaping of message lines that start with ``From'', and there is no need to ensure a newline at the end of each message. Consequently, the default values for check_string, message_prefix, and message_suffix are all unset when directory is set.

There are three different ways in which delivery to individual files can be done, controlled by the settings of the maildir_format and mailstore_format options. Note that code to support maildir or mailstore formats is not included in the binary unless SUPPORT_MAILDIR or SUPPORT_MAILSTORE, respectively, is set in Local/Makefile.

In all three cases an attempt is made to create the directory and any necessary sub-directories if they do not exist, provided that the create_directory option is set (the default). The location of a created directory can be constrained by setting create_file. A created directory's mode is given by the directory_mode option. If creation fails, or if the create_directory option is not set when creation is required, delivery is deferred.

The three different kinds of ``single file'' delivery are as follows:


Previous  Next  Contents       (Exim 4.10 Specification)