bundle

(Maintains a "bundle" of files)

SYNOPSIS

bundle [-cdfhnopqsuyv] [-Dvariable=value] [bundle[.b]]

DESCRIPTION

bundle is designed for two major purposes. It can be used to track a bundle of files and synchronize them with a master repository whenever anything changes, and it can be used to make incremental changes in such a way as to maintain change history and be able to apply all of the changes at once.

bundle functions on bundle files, which are files of bundle commands, one per line. The most frequently used command specifies a source file and a destination file and tells bundle to update the destination file to be the same as the source file if they're different, but there are a variety of other commands for maintaining symbolic links, renaming, deleting, chowning, or chmoding files, running system commands, and other such things.

This flexibility allows bundle to be used as an installation tool. All that's needed is a bundle file that lists source files and where they should be installed, and if anything changes in the master distribution, bundle can just be run against the same file again to pull in any files that have changed.

It also lets bundle be used as an incremental change system, since the bundle file can list precisely what should be changed on a system, including references to the files that are needed as part of the change, and then all the incremental changes can be concatenated together to create a bundle file that will apply all changes to date.

If bundle isn't given a bundle file on the command line, or if the file given is "-", bundle will read and execute bundle commands from stdin.

OPTIONS

bundle takes the following command-line options. --force and --younger can also be set in bundle files themselves (see the documentation of variables below below). Settings in the bundle file itself for those two options override command-line flags.

-h, --help

Print out this documentation (which is done simply by feeding the script to perldoc -t.

-v, --version

Print out the version of bundle and exit.

-b, --backup

Save backup information. Any files that are replaced, rather than being deleted, will be renamed to .b.FILE.TIME.RANDOM where FILE is the original file name, TIME is the current time, and RANDOM is a random number between 0 and 1000. In addition, full details on how to reverse the change (in the form of bundle commands) will be printed out prefixed by " undo: " as the bundle commands are executed. Normally, each undo command is on a single line, but if the undo information line ends in a \, the next line is also part of the undo information and special care should be taken to include it in any backout bundle.

The backup bundle is generated in reverse order. To convert the output of bundle -b into a backup bundle, pipe it through the following commands:

    grep '^  undo: ' | sed 's/^  undo: //' | tac

This does not work if there are any lines ending in \; in that case, something more complex will be needed.

Note that system commands (or commands run as a result of setting the system variable) are not reversible and will result in comments in the backout bundle stating what command was run. Note also that if this option is used, backup files will be saved in the same directory as the original file, thus making it impossible to delete a file in a directory and then the directory itself (since the directory won't be empty).

-C directory, --cwd directory

Change the current working directory to the specified directory before finding and the running the bundle.

-c, --changes

Show the changes between the current files and the files that will be installed. Only shows changes for files installed with the file command where the new file will overwrite an existing file. Changes are given as unified diffs (except where the files are binary), but if the diff is longer than 100 lines, just the length of the diff will be shown.

Enabling this option automatically enables -n as well. Using this option requires that the first diff program found in your path support the -u option. There is currently no way to change the preferred diff format.

-D variable=value, --define variable=value

Defines the variable variable to be equal to value. This is exactly equivalent to a variable assignment inside the bundle file itself, with all the same special variables; see below for full details.

-d, --debug

Enable debugging. This causes bundle to print out quite a few additional messages about what it's doing and what internal functions it's calling, useful for debugging problems in either a bundle file or in bundle itself.

-f, --force

Force all target files to be updated, even if they appear to be the same as the source files. This can also be set in the bundle file itself, and the setting in the bundle file overrides the command line.

-n, --just-print, --dry-run

Do all of the checks that would normally be done, but rather than taking any actual actions, just print out what would have been done. Think make -n. This should be used to check the actions of a bundle file before it's used for the first time.

-q, --quiet

Don't print the normal report of what's being done. Only errors will be printed. This flag also suppresses the output of echo commands in the bundle file.

-s, --source

Report the source location of a file when it is installed. By default, only the destination location is printed.

-u, --up-to-date

Report those actions which were unnecessary as well as those which were done (in other words, report actions which weren't performed because the targets were already up to date).

-y, --younger

By default, files will be installed if the timestamp of the source differs from the timestamp of the target. With this option, a file will only be installed if the source is newer than the target. Combining this option with --force makes no sense. This can also be set in the bundle file itself, and the setting in the bundle file overrides the command line.

BUNDLE FILES

A bundle file is just a sequence of commands or variable settings, one per line, optionally interspersed with comments. Any line beginning with # is taken to be a comment, and lines can be continued on the next line by putting \ at the end of a line. The trailing backslash will be stripped, but it's possible to put a newline inside a string by having the line continuation occur inside a quoted string.

The individual arguments to commands are whitespace-separated; if it's necessary to give an argument that contains whitespace, the argument can be quoted. Both single and double quotes are supported; double-quoted strings undergo variable interpolation, while single-quoted strings do not. Inside double quotes, \ will escape any character, including quotes. Inside single quotes, \ will escape \ or ' but not any other character; \ followed by any other character in single quotes will be taken to be a literal backslash followed by that character. For more information on variable interpolation, see the section on variables below.

The following commands are recognized:

addline FILE LINE [ LINE ... ]

Adds each LINE to the end of FILE if and only if that line doesn't already occur somewhere in FILE. If --force is set, FILE is not checked first and addline acts identically to append. Otherwise, FILE is scanned a line at a time, and if any line in the list of lines to add to the file is found already in the file, it's removed from that list. Then the remaining lines are appended in order to the end of FILE. Because the entire file is checked, one can have multiple addline commands affecting the same file and still run the bundle file repeatedly, unlike append.

Note that since it takes a variable number of lines as arguments, addline does not support setting variables on the same line as the command. For information on how to set force or system for just an addline line, see the section on variables below.

append FILE LINE [ LINE ... ]

Appends each LINE to the end of FILE, separated by newlines. Unless --force is set, FILE is checked first and if it ends in precisely the text that would be appended to it, it is considered to be up to date and no action is taken. The practical implication of this is that one should only have one append command per affected file in a bundle file if one wishes to be able to run the bundle file repeatedly without having more text appended each time; if that isn't what you want, addline may be better. On the other hand, append is faster than addline, particularly for large files. (If lines should be appended every time the bundle file is run, the force variable should be set for just the append line.)

Note that since it takes a variable number of lines as arguments, append does not support setting variables on the same line as the command. For information on how to set force or system for just an append line, see the section on variables below.

chmod MODE FILE

Normally, given a MODE, changes the permissions on FILE to be MODE (where MODE is assumed to be in octal). If MODE begins with +, it is taken to be a mask, and the permissions on FILE are changed to be the same as they currently are but with all the bits in MODE turned on. If MODE begins with -, the permissions on FILE are changed so that all the bits in MODE are off.

For example, to turn on the setuid bit of file and turn off the group and world read bits, leaving the rest of the permissions unchanged, use the following two chmod commands:

    chmod +4000 file
    chmod -0044 file

Note that leading zeroes on MODE are allowed but not required; it's always assumed to be in octal.

chown USER GROUP FILE

Changes the ownership of FILE to user USER and group GROUP. The user and group can be either numeric or words (in which case, they're translated to numbers by using getpwnam() and getgrnam()). Note that either USER or GROUP may be -1, indicating no change should be made to that portion of the file ownership.

For example:

    chown root -1 file

will change the ownership of file to be root, but will leave the group ownership unchanged.

delete FILE

Deletes FILE. (This command won't work on directories; to remove a directory, use rmdir instead.)

dir DIRECTORY

Creates the directory DIRECTORY (and any missing directories in the full path, recursively if necessary, similar to the function of mkdir -p). The default permissions (if mode isn't set) are 0755 modified by umask. If the directory already exists, and if any of owner, group, mode, atime, or mtime are set and differ from the stats of the existing directory, it is updated to match the desired values.

Note that if any setting of the mode variable is in effect, it is used verbatim by the dir command. If you've set mode to something appropriate for regular files, this may result in a directory without execute permission, which is generally not what you want. One alternative is to always use an explicit setting of mode on the same line as the dir command; for details on how to do that, see below. Another alternative is to not use dir commands and just let the directories be created automatically when something is put in them with a file command or another similar command, since then the created directories will have execute permission granted corresponding to the read permission granted in the mode of the file being created.

Note also that the setting of mode affects the final directory in the chain, but any created directories above that use the same magic determination of mode as ones created by file do. This is arguably a bug.

echo TEXT [ TEXT ... ]

If --quiet was not specified, prints TEXT to stdout.

file SOURCE TARGET

Installs the file SOURCE as TARGET. In other words, if TARGET doesn't exist, or if TARGET exists but has a different timestamp or size than SOURCE, SOURCE is copied to TARGET (deleting TARGET if necessary). Any directories that are part of the path TARGET which don't already exist are created in the process. The ownership, permissions, atime, and mtime of TARGET will be set to be the same as SOURCE unless overridden by variable settings.

filexists FILE

Ensures that FILE exists. If it is present and is a regular file, it is considered up to date (permissions and ownership are not checked). If it doesn't exist, a zero-length file named FILE is created. The default permissions (if mode isn't set) will be 0644 modified by umask. If it exists and isn't a regular file, an error message will be printed.

filter FILE INIT FILTER

Sets up a lexical block, executes INIT within that block, and then executes FILTER on each line of FILE, overwriting FILE with the results, in the same lexical block that INIT ran in. This is mostly (but not quite completely) equivalent to:

    perl -i -pe 'BEGIN { INIT } FILTER' FILE

The mode and permissions of FILE are preserved. INIT and FILTER can be any valid Perl code; inside FILTER, $_ will contain the current line of FILE and modifying $_ will modify the current line of FILE. BEGIN and END blocks are unlikely to do what you expect. All code will be executed in the context of the running bundle process, so it's possible to severely confuse bundle or worse using this directive. Be cautious about how much you do, and declare any variables you use in the INIT section with my. Note that INIT can be empty if you have no need of persistent variables.

The following fairly simple example replaces all occurances of "example.com" with "example.org" in /etc/hosts:

    filter /etc/hosts '' 's/example\.com/example\.org/'

Here's a more complicated example that inserts the line "ssh\t\t22/tcp" after any line beginning with "ftp" and whitespace in /etc/services, if and only if the next line doesn't already begin with "ssh" and whitespace:

    filter /etc/services 'my $saw'                    '\
        if ($saw) {                                    \
            $_ = "ssh\t\t22/tcp\n$_" unless /^ssh\s/;  \
            $saw = 0;                                  \
        } elsif (/^ftp\s/) {                           \
            $saw = 1;                                  \
        } else {                                       \
            $saw = 0;                                  \
        }                                              '

Observe that a bundle command is still just a single line, so if it's continued on multiple lines each line except the last has to end in a backslash. Also notice the quoting; because the code fragment is in single quotes, bundle doesn't try to do variable interpolation (which would be a mess) and backslashes inside single quotes are only special if followed by another backslash or by a single quote. So strings like \n can be used as is in Perl code inside single quotes.

Another thing to notice about this example is that we go to some lengths to ensure that we don't make any changes unless necessary. This lets us run a bundle file containing the above filter repeatedly without ill effect.

The filtering is done in memory; this directive should not be used to modify large files.

link VALUE TARGET

A symbolic link named TARGET is created, with value VALUE. If TARGET already exists, it's deleted as necessary. Any directories that are part of the path TARGET which don't already exist are created in the process. Note that no effort is made to change the ownership or permissions on symbolic links; they get whatever defaults are assigned by the system.

linkval SOURCE TARGET

Copies the link SOURCE to TARGET. In other words, a symbolic link is created named TARGET which contains as its value the same value that the link SOURCE has. In general, this isn't as useful as the link command (see below) where the value can be specified directly. If TARGET already exists, it's deleted. Any directories that are part of the path TARGET which don't already exist are created in the process. Note that no effort is made to change the ownership or permissions on symbolic links; they get whatever defaults are assigned by the system.

rename OLD NEW

The file named OLD, if it exists, is renamed to NEW. (This is done as a copy and unlink if OLD and NEW are on different file systems.) If OLD does not exist and NEW does, bundle assumes this command is up to date and takes no action; other cases (including the case where NEW already exists) are considered errors.

rmdir DIRECTORY

Removes DIRECTORY. This only works if DIRECTORY is empty; there is no way in bundle to do the equivalent of rm -rf without using system.

system COMMAND [ ARGUMENT ... ]

Runs the command COMMAND, passing it the specified arguments. Note that this doesn't have any dependencies and therefore bundle has no way of knowing whether it has already been done and will run this command every time the bundle file is run. If COMMAND contains shell metacharacters, it must be specified as one quoted string rather than as a command with arguments; if given as a command with arguments, the command will be run directly without going through the shell, so there is no need to protect shell metacharacters in the arguments. (It's still necessary to protect bundle metacharacters, of course.)

Note that for any bundle command that creates files, directories, or symbolic links, such as file, dir, link, linkval, and the like, if the directory in which the target file resides does not exist, bundle will create it. Any needed directories above that directory are also created. Ownership and permissions are inherited from the settings in force at the time of the command, and the execute bit on each created directory is set if the read bit on the file would be set.

bundle also supports variables. A variable can be set with the syntax:

    VARIABLE=VALUE

on a line by itself, where VARIABLE consists of alphanumerics or underscores and VALUE is any value (it will need to be put in double quotes if it contains whitespace, and it follows the same quoting rules as commands). Note that there cannot be any spaces before or after the equal sign. After a variable has been set, it can be interpolated into any command. For example:

    SOURCE=/usr/pubsw/etc
    DEST=/etc/leland
    owner=root
    file $SOURCE/krb.conf $DEST/krb.conf

will copy /usr/pubsw/etc/krb.conf to /etc/leland/krb.conf. Variables will be interpolated within double-quoted strings; a literal dollar sign can be written as \$ or protected with single quotes. Both the $VARIABLE and ${VARIABLE} syntaxes are supported.

All variables consisting solely of all lowercase letters are special; most are reserved by bundle. These variables cannot be interpolated into commands; instead, they affect the operation of bundle in other ways. Special variables (and only those) can be set at the end of a normal bundle command as well as by the syntax described above. For example, the above could also be written as:

    SOURCE=/usr/pubsw/etc
    DEST=/etc/leland
    file $SOURCE/krb.conf $DEST/krb.conf owner=root

and would perform the same action. Assignments on a line by themselves are permanent and will affect all bundle commands after them in the bundle file. Assignments on the same line as a command, as in the previous example, are in effect only for that command.

To undefine a variable, just assign the empty string to it. For example:

    SOURCE=

The following special variables can be set in bundle files:

atime=TIME

Sets the atime (access timestamp) of files or directories created with the file, filexists, or dir commands to TIME (in the case of file, overriding the timestamp of the source file). TIME should be in Unix time format (seconds since epoch).

force=1, force=0

Turns --force on (or off). This overrides the command line flag and is mostly useful at the end of a specific bundle command which should always be performed regardless of whether the target appears to be up to date.

group=GROUP

Sets the group ownership of created files to GROUP. By default, created files are owned by the same group as the source file. GROUP may be either by GID or by name. It's recommended that, for files that should be owned by group 0, the group be specified by number rather than by name since the name of that group differs across platforms. GROUP may be set to -1 to indicate that no changes in group ownership should be made.

mode=MODE

Sets the permissions of created files to MODE (assumed to be in octal). By default, created files are given the same mode as the source file.

mtime=TIME

Sets the mtime (modification timestamp) of files or directories created with the file, filexists, or dir commands to TIME (in the case of file, overriding the timestamp of the source file). TIME should be in Unix time format (seconds since epoch). Note that if this variable is set, the target file of a file command will be considered to be out of date and require an update if its mtime isn't equal to TIME (rather than comparing it to the timestamp on the source file).

owner=OWNER

Sets the owner of created files to OWNER. By default, created files are owned by the same group as the source file. OWNER may be either by UID or by name. OWNER may be set to -1 to indicate that no change in ownership should be made.

system=COMMAND

Runs COMMAND for every successful bundle command. (For obvious reasons, this is most useful at the end of a specific bundle command rather than as a standalone variable setting that would affect all subsequent commands, although the standalone setting is necessary to run a system command based on the results of an append command.) If COMMAND contains arguments, or otherwise contains whitespace, it will need to be in double quotes to prevent bundle from interpreting the whitespace as separators. %S will be replaced with the source file %D with the target file of the associated command, if any. The command specified in the system variable is run only if the bundle command the variable is associated with was performed, so this is a way to get system commands with dependencies.

younger=1, younger=0

Turns --younger on (or off). This overrides the command line flag and is mostly useful at the end of a specific bundle command which should take effect only if the target is older than the source file.

The echo and system commands don't allow any variable settings on the same line and ignore all variable settings (except for interpolation). The addline and append commands similarly don't allow variable settings on the same line but do honor the force and system variables. In order to set one of those variables for just an addline or append command, it's necessary to use a sequence of lines like:

    force=1
    addline /path/to/file "First line" "Second line"
    force=0

to force the addline or append but not the other commands in the bundle file.

EXAMPLES

The following sample bundle file installs a full set of Kerberos executables and configuration files. Notice that in no case is the ownership or mode of the original files trusted; instead, an explicit default ownership of root.0 is given and a default mode of 644 for configuration files and 755 for executable files is used. Note also the rule for installing ksu overrides the default mode for just that one bundle command.

  owner=root
  group=0

  mode=644

  file /usr/pubsw/etc/krb.conf      /etc/leland/krb.conf
  link leland/krb.conf              /etc/krb.conf

  file /usr/pubsw/etc/krb5.conf     /etc/leland/krb5.conf
  link leland/krb5.conf             /etc/krb5.conf

  file /usr/pubsw/etc/krb.realms    /etc/leland/krb.realms
  link leland/krb.realms            /etc/krb.realms

  file /usr/pubsw/etc/hesiod.conf   /etc/leland/hesiod.conf
  link leland/hesiod.conf           /etc/hesiod.conf

  file /usr/pubsw/lib/zephyr/zephyr.vars /etc/leland/zephyr.vars

  mode=755

  file /usr/pubsw/sbin/login.krb    /etc/leland/login.krb
  file /usr/pubsw/sbin/klogind      /etc/leland/klogind
  file /usr/pubsw/sbin/kshd         /etc/leland/kshd
  file /usr/pubsw/sbin/kftgtd       /etc/leland/kftgtd

  file /usr/pubsw/bin/aklog         /etc/leland/aklog

  # comment this out if you don't want ksu installed locally
  file /usr/pubsw/sbin/ksu.nosetuid /usr/bin/ksu mode=4111

  file /usr/pubsw/sbin/telnetd      /etc/leland/telnetd
  file /usr/pubsw/sbin/zhm          /etc/leland/zhm
  file /usr/pubsw/sbin/sidentd      /etc/leland/sidentd
  file /usr/pubsw/sbin/tcpd         /etc/leland/tcpd

ENVIRONMENT

BUNDLE_LIB

A colon-separated list of directories to search through for bundle files. Note that . (the current directory) is not searched by default unless it is included in the search path. The default search path is set at the top of the bundle script.

NOTES

While bundle is very free-form about the syntax allowed in its files, in practice there are a few style guidelines that I've found useful for keeping the files clear, readable, and maintainable. Here's a short list.

If you're using bundle to install a set of related files, consider writing your bundle file to use relative paths for the source location of those files. That way, you can take your bundle file and all of its associated files and move them around or distribute them as a package and no changes to the bundle file are needed; anyone running it simply has to change to that directory before doing so. If this doesn't fit your application, at least put the common source directory as a variable set at the top of the bundle file and have all of the commands in the file use that variable so that you only have to change one line.

It's worth going to some effort to prevent things from going over 80 columns to keep them more readable. Remember that bundle allows continuation lines just by ending a line with a backslash, and remember that you can define variables and thereby abbreviate long paths.

There are a lot of bundle commands, particularly the frequently used file command, that take two arguments. I generally line up the second arguments at column 34 (provided the first argument isn't too long); the whitespace makes things look more readable to me.

Putting one-line system variable settings on the next line from the command, indented four spaces (and with a backslash on the previous line, of course), seems to make them stand out better. For example:

    file $FILES/aliases               /etc/mail/aliases \
        system="/usr/lib/sendmail -bi"

Remember that bundle files allow blank lines and comments. Comments are good.

BUGS

The bundle generated by the --backup flag in the case of a file directive that replaces an existing file is not idempotent, and in fact running the backout bundle twice will cause the file to be deleted completely. This is obviously bad and should be fixed; the only clear way of fixing it would appear to be to add a new directive like rename that is willing to overwrite a destination file.

The dir directive honors mode verbatim for the last directory created, but any directories higher on the chain use the magic mode formed by ensuring that there is execute permission for every user who has read permission. This is arguably a bug.

There is no way to set a variable to the empty string.

WARNINGS

The behavior of chown on symlinks is undefined; this is not the fault of bundle or Perl but the fault of the underlying system call. On some systems, chown will follow the link and change the ownership of the file it points to; on other systems, chown will change the ownership of the link. Having a symbolic link be the target of a chown command is therefore not recommended. bundle does not implement lchown.

Those sensitive about security should note that while some effort has been taken to avoid possible security problems, there are a number of race conditions involving file creation and overwriting still remaining, particularly with the filter and file commands. Running those commands with target files in world-writeable directories while bundle is running as root is not recommended.

There is a possibility (roughly one in a thousand) that, when running bundle with the --backup flag and making multiple changes to the same file in the course of a single bundle file, that bundle will fail to make a backup of intermediate versions of the file. --just-print may also return errors or incorrect information for a bundle file that makes multiple changes to the same file.

Setting owner or group to -1 indicates that no changes should be made to the file ownership or group ownership, not to chown the file to the UID or GID one less than the maximum value (often nobody or nogroup). To specify that UID or GID, use its name or its positive UID value.

AUTHORS

The original bundle program (based on synctree from the University of Michigan) was written by Roland Schemers <schemers@stanford.edu>. It has since been rewritten by Russ Allbery <rra@stanford.edu> who currently maintains it.

COPYRIGHT AND LICENSE

Copyright 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2002, 2003, 2004, 2006, 2008 Board of Trustees, Leland Stanford Jr. University

This program is free software; you may redistribute it and/or modify it under the same terms as Perl itself.

Last spun 2013-07-01 from POD modified 2008-10-06