NAME
    helm - Easy server and cluster automation

SYNOPSIS
        helm TASK [OPTIONS]
        helm help tasks
        helm help [TASK]

        # patch the same file on all of the machines in your cluster 
        helm patch --file my_fix.patch --target /opt/cool_system/do_stuff.pl

        # run a command on only the servers which run memcache
        helm run "pidof memcached" --roles memcache 

        # copy a file to every server in your cluster
        helm put --local foo.tar.gz --remote /tmp/bar.tar.gz

        # rsync a local folder to specific servers
        helm rsync_put --local lib/important --remote /tmp/important_libs --servers web1,web2,web4

DESCRIPTION
    "helm" is command-line utility to make it easy to automate system tasks
    for individual machines, a cluster of machines or a subset of machines
    in a cluster. It has the following features:

    *   Combine multiple commands into a single tasks and to have groups of
        related tasks gathered together.

    *   Uses SSH as the transport layer for tasks and uses SSH keys for
        automatic authorization.

    *   Simple optional configuration file describing your cluster, allowing
        tasks to target a single machine, multiple machines which share the
        same role, or all machines in your cluster. Can also be extended to
        pull configuration from more complicated sources like LDAP, etc.

    *   Logging of each action performed to multiple channels: console, log
        file, irc, email, etc.

    *   Interact with the remote processes via "STDIN", "STDOUT" and
        "STDERR".

    *   Convenient .helmrc file to reduce the number of options you need to
        pass on every invocation of "helm".

    *   Locking on the client and/or server so that multiple invocations of
        "helm" aren't running at the same time.

OPTIONS
    "helm" has several options that can be specified for all commands. Any
    remaining arguments are passed to the task that was invoked. These
    global options are:

    --user
        By default, helm assumes you're running the comands as the current
        local. This works really well with SSH keys set up so that you don't
        have to try and type in your password on lots of login attempts.
        However, if you want to change the user, this option allows for
        that. You will however be prompted for a password if you don't have
        SSH keys properly set up.

    --servers
        Specifically list out the hostnames of the servers to send the task
        to. These can either be full hostnames (or an unambiguous
        abbreviation of your servers as defined in your CONFIGURATION).
        Multiple hostnames must me specified in a comma separated list.

        If no servers (or roles) are specified, then the task will be
        performed on all servers (as defined in your CONFIGURATION).

            # using full hostnames 
            --servers 192.168.0.23,192.168.0.24,web1.company.com

            # using abbreviations from config file 
            --servers web1,db2

            # using abbreviations from config file and ranges
            --servers web[1-3],db[1-2]

    --roles
        Instead of specifying the servers explicitly, you can instead
        specify which server roles you want to target (as defined in your
        CONFIGURATION). Multiple roles must be specified in a comma
        separated list.

        If no roles (or servers) are specified, then the task will be
        performed on all servers (as defined in your CONFIGURATION).

            # single role 
            --roles web

            # multiple roles
            --roles web,cache

    --exclude
        Specifically exclude servers from the list to be used. This can be
        combined with both "--servers" and "--roles" and even by itself to
        exclude certain servers from the list of all servers as defined in
        your CONFIGURATION).

            # combined with --servers
            --servers web[1-10] --exclude web3

            # combined with --roles, using ranges
            --roles web --exclude web[4-5]

    --exclude-roles
        Specifically exclude servers with certain roles from the list to be
        used. This can be combined with both "--servers" and "--roles" and
        even by itself to exclude certain servers from the list of all
        servers as defined in your CONFIGURATION).

            # combined with --servers: any web[1-10] servers that aren't also caches
            --servers web[1-10] --exclude-roles cache

            # combined with --roles: any web servers that aren't also proxies
            --roles web --exclude-roles proxy

    --parallel
        Execute tasks in parallel on the remote servers. The default is to
        execute in serial instead.

    --parallel-max
        The maximum number of parallel processes to be running at the same
        time if "--parallel" is used. The default is 100.

    --config
        Which resource to use for pulling cluster configuration information
        (see CONFIGURATION).

            --config helm:///etc/helm.conf

    --log-level
        The log level used. Can be one of "debug", "info", "warn", and
        "error". Defaults to "info".

    --log
        Log messages to a specific channel. Multiple channels can be
        specified by specifying this option multiple times. The value of
        this option is the URI of the channel to be used. See "LOG CHANNELS"
        for more details.

            # send log messages to a comany tech IRC channel 
            --log irc://helm@irc.company.com/tech

            # log messages to a file and to email address
            --log file:///var/log/helm.log --log mailto:helm@company.com?from=helm@company.com

        Log messages are also sent to "STDERR" unless the --quiet option is
        also passed.

    --load
        Allows you to load other 3rd party plugin modules to extend "helm"
        functionality. This could be to add more log channels, different
        configuration loading, etc. The value of this option is the full
        Perl module name of the plugin. This can be specified multiple
        times.

            # load a hypothetical Yammer log plugin 
            --load Helm::Log::Channel::Yammer

            # load hypothetical custom LDAP configuration and
            --load Helm::Conf::Loader::CompanyLDAP

        See "WRITING HELM PLUGINS" for more information.

    --sudo
        The user that should be used to perform the commands on the remote
        server. The actual SSH connection will be made using the current
        user's SSH keys, but then once the connection is made to the remote
        server, it's sometimes useful for the commands to be run as a
        different user. We also try to make sure that things like file
        permissions (on tasks like "put" and "patch") are also handled so
        that the resulting files are owned by this sudo user.

    --lock
        This options allows you to have control over whether concurrent
        "helm" processes can be running on either the local or remote
        servers. The value can be one of: "none", "local", "remote", "both".

    --no-rc-file
        Suppress the default loading of the ".helmrc file" file.

    --port
        The port to use for SSH on all of the remote servers. Defaults to
        the standard (22).

    --timeout
        The timeout in seconds to give the ssh connections. Default is 30
        seconds.

    --continue-with-errors
        If a command fails executing against this will normally cause
        execution to halt and not run against any other remote servers.
        Using this option will tell helm to continue on to the server(s)
        even if a previous one has errors.

    --quiet
        Suppress the default logging to "STDERR"

    --man
        Display this documentation

    --version
        Display the version of Helm installed

    --dump-config
        Display a dump of the configuration data as understood by Helm

    --debug
        Tell helm to dump out verbose information about what it is doing
        internally to a log file named helm.debug in the current working
        directory. This is different than the "--log-level" since that is
        mainly for end user logging of individual task actions. This flag is
        meant to debug the internals of helm, including the logging
        subsystem, which is why it's a separate flag.

    --sleep
        Sleep a given number of seconds in between each server.

TASKS
    Helm comes with the following built-in tasks

    To get more information on how to run each task, simply use "helm help":

        helm help run

  run
    This is the simplest of helm commands and will run some arbitrary
    command on the remote server.

  get
    Get a file from a remote server and put it on the local machine.

  put
    Put a local file onto a remote server.

  rsync_put
    Put a local file onto a remote server using "rsync" rather than a full
    copy.

  patch
    Patch a file on a remote server with a patch on the local machine.

  unlock
    Clear out any stuck Helm locks on a remote server.

CONFIGURATION
    By default, "helm" doesn't use a configuration file, but certain
    features require it (using roles, server abbreviations, etc) so it's
    best to have one. You can tell "helm" which configuration resource to
    use by using the --config option. Currently, only the "helm://" URI
    scheme is supported.

        --config helm:///etc/helm.conf

    A configuration file will look something like this:

        <Server web[1-5].company.com>
            Role web
        </Server>

        <Server db1.company.com>
            Role db Role db_master
        </Server>

        <Server db2.company.com>
            Role db Role db_slave
        </Server>

    This configuration would define 7 servers (web1.company.com,
    web2.company.com, web3.company.com, web4.company.com, web5.company.com,
    db1.company.com and db2.company.com). It defines 4 different roles (web,
    db, db_master, db_slave).

    "helm" currently just supports a single configuration resource format
    ("helm://"), but the internals are flexible enough that more formats
    could be supported in the future, including other configuration methods
    like LDAP, etc.

    If, for instance you wanted to support a URI like:

        --config ldap://ldap.company.com

    See "WRITING HELM PLUGINS" for more information.

    If you are having problems getting your configuration right, you can
    pass the --dump-config option to tell "helm" to display what it thinks
    things are configured to be.

LOG CHANNELS
    "helm" can be told to send various levels of log data to different
    channels. By default we support a log file, IRC and email logs. We also
    support various log levels (see the --log-level option).

    You can specify which channel is used by giving a URI which indicates
    what type of channel and where to send the log. The following URI
    schemes are supported:

    file://
        This is basically a log file where messages are immediately
        appended.

            --log file:///var/log/helm.log

    irc://
        NOTE: IRC logging is still experimental and quite tempermental.
        Improvements are welcome.

        This is an IRC channel where messages are immediately sent. For
        example to send messages to the "sysadmin" IRC channel on
        "irc.company.com" using the user "michael" and the password "s3cr3t"
        you would have:

            --log irc://michael@irc.company.com/sysadmin?pass=s3cr3t

    mailto:
        Similar to Mail-To links in HTML documents, this just specifies an
        email address to log. Log messages aren't sent immediately, but are
        instead queued up to be sent once the command has been completed.

            --log mailto:michael@company.com?from=helm@company.com

    Plugins can be written to load allow other log channels. See "WRITING
    HELM PLUGINS" for more information on how this is done.

THE .helmrc FILE
    The .helmrc file contains command-line options that are prepended to the
    command line before processing. Multiple options may live on multiple
    lines. Lines beginning with a # are ignored. A .helmrc might look like
    this:

        # always log to a file 
        --log file:///var/log/helm.log

        # always load our custom plugins 
        --load Helm::Conf::Loader::CompanyLDAP 
        --load Helm::Log::CompanyYammer
        --load Company::CustomHelmTasks

    "helm" looks for the rc file first in the current directory and then in
    your home directory. You can specify another location with the "HELMRC"
    environment variable.

    If --no-rc-file is specified on the command line, the .helmrc file is
    ignored.

WRITING HELM PLUGINS
    Helm can be extended in many ways to make it more convenient for your
    projects. Helm has 3 customization points where plugins can be written
    and registered to interact: "task", "log" and "configuration". Using the
    "--load" option, you can tell Helm about your custom or 3rd party Perl
    modules that you would like to load. Each plugin module must register
    itself with Helm along with the type of events it will handle using the
    "Helm->register_module" method. For example, if I were creating a new
    plugin module for a custom task named "spiffy" I might invoke Helm like:

        helm spiffy --load MyCompany::Helm::Task::spiffy

    And my Perl module might look something like this:

        package MyCompany::Helm::Task::spiffy;
        use strict;
        use warnings;
        use Helm;
        use Moose;
        extends 'Helm::Task';
        Helm->register_module('task', spiffy => 'MyCompany::Helm::Task::spiffy')

        sub validate {
            my $self = shift;
            # custom validation
        }

        sub execute {
            my ($self, %args) = @_;
            # do something spiffy
        }

        1;

    Most likely you'd put the "--load" statment in your .helmrc file so you
    wouldn't have to worry about it again. Now we'll get into the details of
    what is expected of each type of plugin.

  Task Plugins
    A task plugin should inherit from Helm::Task and implement the following
    methods:

    validate
        This method would perform any custom validation needed before the
        task is executed against any servers. Normally this involves
        validating the custom options this task might use. This method only
        receives a single object, the task itself. As an example, lets say
        you want a "--nifty" option to your "spiffy" plugin above:

            helm spiffy --nifty foo

        Then your validation method might look something like this:

            sub validate {
                my $self          = shift;
                my $helm          = $self->helm;
                my $extra_options = $helm->extra_options;
                my $nifty         = $extra_options->{nifty};
                $helm->die("You need to provide a --nifty option!") unless $nifty;
                $helm->die("--nifty option ($nifty) is not a valid value") unless $nifty =~ /^fo+/;
            }

    execute
        This is the meat of your task plugin and is where the work happens.
        This method is called once for every server the task is being
        executed against. It receives the following named arguments:

        ssh A Net::OpenSSH object with an already open SSH connection to the
            server in question.

        server
            A Helm::Server object for the currently executing task.

        As long as your method doesn't die (or preferrably calls
        "$helm->die()", then we will assume that all was fine and dandy in
        the execution of the task.

  Log Plugins
    Log plugins can offer new channels for logging Helm output based on the
    URI given to Helm. For instance, if you wanted to send SMS logging of
    critical messages only, you might invoke helm with a logger like:

        helm foo --log sms:+15105550101 --load MyCompany::Helm::Log::SMS

    And then your SMS module might look something like

        package MyCompany::Helm::Log::SMS;
        use strict;
        use warnings;
        use Helm;
        use Moose;
        extends 'Helm::Log::Channel';
        Helm->register_module('log', sms => 'MyCompany::Helm::Log::SMS');

        sub initialize   {}
        sub finalize     {}
        sub start_server {}
        sub end_server   {}
        sub debug        {}
        sub info         {}
        sub warn         {}
        sub error        { 
            my ($self, $msg) = @_;
            # send SMS message
        }
        1;

    In this example, we don't care about anything except errors (since SMS
    messages cost money and would get really annoying for anything with
    frequency). Log plugins need to inherit from Helm::Log::Channel and
    implement the following methods:

    intialize
    finalize
    start_server
    end_server
    debug
    info
    warn
    error

   Configuration Plugins
    Configuration plugins can implement new ways to load configuration data
    about your servers based on the URI given to helm. For instance, if you
    wanted to load the list of your servers and their roles from a company
    LDAP server, you might invoke Helm like:

        helm foo --config ldap://ldap.company.com --load MyCompany::Helm::Conf::ldap

    Helm will look for the last module registered to handle the "ldap"
    scheme of that url. That module might look like:

        package MyCompany::Helm::Conf::ldap;
        use strict;
        use warnings;
        use Helm;
        use Moose;
        extends 'Helm::Conf::Loader';
        Helm->register_module('configuration', ldap => 'MyCompany::Helm::Conf::ldap')

        sub load {
            my ($self, %args) = @_;
            my $uri = $args{uri};
            # poke around in our LDAP server and create a list of Helm::Server objects
            my @servers = ...;
            # then return a Helm::Conf object
            return Helm::Conf->new(servers => \@servers)
        }

        1;

    A configuration loading plugin should inherit from Helm::Conf::Loader
    and implement the following methods:

    load
        This method is the backbone of a configuration plugin. It receives
        the following named arguments:

        uri A URI object representing the URI passed on the command line to
            be loaded by this configuration loader.

        helm
            The "Helm" object doing the loading.

        This method must create a list of Helm::Server objects and use them
        to return a Helm::Conf object.

CAVEATS
    Multi-Platform
        This has been developed and tested on Linux (with bash as the shell
        on the remote hosts) only. Dealing with multiple platforms and
        writing multi-platform tasks has been punted to the future.

    Alpha Software
        This software is very ALPHA, which means it's interface is likely to
        change in the future. It's used currently and happily in production
        but please be aware if you start using it that you'll probably want
        to follow future versions carefully to make sure you aren't bitten
        by API changes as thing are flushed out.

TODO
    In the not too distant future, we'd like to add the following features
    to Helm:

    * Add a capture option which allows stdout/stderr to be handled
    differently
    * Add a compare option which allows the output (stdout/stderr) to be
    compared between servers in an intelligent manner
    * A real exception system to avoid parsing error messages