Running LinchPin

This guide will walk you through the basics of using LinchPin. LinchPin is a command-line utility, a Python API, and Ansible playbooks. As this guide is intentionally brief to get you started, a more complete version can be found in the documentation links found to the left in the index.

Running the linchpin command

The linchpin CLI is used to perform tasks related to managing resources. For detail about a specific command, see Commands (CLI).

Getting Help

Getting help from the command line is very simple. Running either linchpin or linchpin --help will yield the command line help page.

$ linchpin --help
Usage: linchpin [OPTIONS] COMMAND [ARGS]...

  linchpin: hybrid cloud orchestration

Options:
  -c, --config PATH               Path to config file
  -p, --pinfile PINFILE           Use a name for the PinFile different from
                                  the configuration.
  -d, --template-data TEMPLATE_DATA
                                  Template data passed to PinFile template
  -o, --output-pinfile OUTPUT_PINFILE
                                  Write out PinFile to provided location
  -w, --workspace PATH            Use the specified workspace. Also works if
                                  the familiar Jenkins WORKSPACE environment
                                  variable is set
  -v, --verbose                   Enable verbose output
  --version                       Prints the version and exits
  --creds-path PATH               Use the specified credentials path. Also
                                  works if CREDS_PATH environment variable is
                                  set
  -h, --help                      Show this message and exit.

Commands:
  init     Initializes a linchpin project.
  up       Provisions nodes from the given target(s) in...
  destroy  Destroys nodes from the given target(s) in...
  fetch    Fetches a specified linchpin workspace or...
  journal  Display information stored in Run Database...

For subcommands, like linchpin up, passing the --help or -h option produces help related to the provided subcommand.

$ linchpin up -h
Usage: linchpin up [OPTIONS] TARGETS

  Provisions nodes from the given target(s) in the given PinFile.

  targets:    Provision ONLY the listed target(s). If omitted, ALL targets
  in the appropriate PinFile will be provisioned.

  run-id:     Use the data from the provided run_id value

Options:
  -r, --run-id run_id  Idempotently provision using `run-id` data
  -h, --help           Show this message and exit.

As can easily be seen, linchpin up has additional arguments and options.

Basic Usage

The most basic usage of linchpin might be to perform an up action. This simple command assumes a PinFile in the workspace (current directory by default), with one target dummy.

$ linchpin up
Action 'up' on Target 'dummy' is complete

Target              Run ID      uHash   Exit Code
-------------------------------------------------
dummy                   75       79b9           0

Upon completion, the systems defined in the dummy target will be provisioned. An equally basic usage of linchpin is the destroy action. This command is peformed using the same PinFile and target.

$ linchpin destroy
Action 'destroy' on Target 'dummy' is complete

Target              Run ID      uHash   Exit Code
-------------------------------------------------
dummy                   76       79b9           0

Upon completion, the systems which were provisioned, are destroyed (or torn down).

Preview Feature:

linchpin up and destroy includes –use-shell parameter which makes linchpin run as a subprocess rather than ansible api call usefull when we would like to overwrite environment varibles

$ linchpin -vvvv up --use-shell --env-vars TESTENV testenv value

Options and Arguments

The most common argument available in linchpin is the TARGET. Generally, the PinFile will have many targets available, but only one or two will be requested.

$ linchpin up dummy-new libvirt-new
Action 'up' on Target 'dummy' is complete
Action 'up' on Target 'libvirt' is complete

Target              Run ID     uHash    Exit Code
-------------------------------------------------
dummy                   77      73b1            0
libvirt                 39      dc2c            0

In some cases, you may wish to use a different PinFile.

$ linchpin -p PinFile.json up
Action 'up' on Target 'dummy-new' is complete

Target              Run ID      uHash   Exit Code
-------------------------------------------------
dummy-new           29          c70a            0

As you can see, this PinFile had a target called dummy-new, and it was the only target listed.

Other common options include:

  • --verbose (-v) to get more output

  • --config (-c) to specify an alternate configuration file

  • --workspace (-w) to specify an alternate workspace

Combining Options

The linchpin command also allows combinining of general options with subcommand options. A good example of these might be to use the verbose (-v) option. This is very helpful in both the up and destroy subcommands.

$ linchpin -v up dummy-new -r 72
using data from run_id: 72
rundb_id: 73
uhash: a48d
calling: preup
hook preup initiated

PLAY [schema check and Pre Provisioning Activities on topology_file] ********

TASK [Gathering Facts] ******************************************************
ok: [localhost]

TASK [common : use linchpin_config if provided] *****************************

What can be immediately observed, is that the -v option provides more verbose output of a particular task. This can be useful for troubleshooiting or giving more detail about a specitic task. The -v option is placed before the subcommand. The -r option, since it applies directly to the up subcommand, it is placed afterward. Investigating the linchpin -help and linchpin up --help can help differentiate if there’s confusion.

Common Usage

Verbose Output

$ linchpin -v up dummy-new

Specify an Alternate PinFile

$ linchpin -vp Pinfile.alt up

Specify an Alternate Workspace

$ export WORKSPACE=/tmp/my_workspace
$ linchpin up libvirt

or

$ linchpin -vw /path/to/workspace destroy openshift

Provide Credentials

$ export CREDS_PATH=/tmp/my_workspace
$ linchpin -v up libvirt

or

$ linchpin -v --creds-path /credentials/path up openstack

Note

The value provided to the --creds-path option is a directory, NOT a file. This is generally due to the topology containing the filename where the credentials are stored.

Workspaces

Initialization (init)

Running linchpin init will generate the workspace directory structure, along with an example PinFile, topology, and layout files. Performing the following tasks will generate a simple dummy folder with All in one PinFile which includes topology, and layout structure.

$ pwd
/tmp/workspace
$ linchpin init
Created destination workspace <path>
$ tree

├── dummy
│   ├── PinFile
│   ├── PinFile.json
│   └── README.rst
└── linchpin.log

Resources

With LinchPin, resources are king. Defining, managing, and generating outputs are all done using a declarative syntax. Resources are managed via the PinFile. The PinFile can hold two additional files, the topology, and layout. Linchpin also supports Linchpin Hooks.

Topology

The topology is declarative, written in YAML or JSON (v1.5+), and defines how the provisioned systems should look after executing the linchpin up command. A simple dummy topology is shown here.

---
topology_name: "dummy_cluster" # topology name
resource_groups:
  - resource_group_name: "dummy"
    resource_group_type: "dummy"
    resource_definitions:
      - name: "web"
        role: "dummy_node"
        count: 1

This topology describes a single dummy system that will be provisioned when linchpin up is executed. Once provisioned, the resources outputs are stored for reference and later lookup. Additional topology examples can be found in :dirs1.5:`the source code <workspace/topologies>`.

Inventory Layout

An inventory_layout (or layout) is written in YAML or JSON (v1.5+), and defines how the provisioned resources should look in an Ansible static inventory file. The inventory is generated from the resources provisioned by the topology and the layout data. A layout is shown here.

---
inventory_layout:
  vars:
    hostname: __IP__
  hosts:
    example-node:
      count: 1
      host_groups:
        - example

The above YAML allows for interpolation of the ip address, or hostname as a component of a generated inventory. A host group called example will be added to the Ansible static inventory. The all group always exists, and includes all provisioned hosts.

$ cat inventories/dummy_cluster-0446.inventory
[example]
web-0446-0.example.net hostname=web-0446-0.example.net

[all]
web-0446-0.example.net hostname=web-0446-0.example.net

Note

A keen observer might notice the filename and hostname are appended with -0446. This value is called the uhash or unique-ish hash. Most providers allow for unique identifiers to be assigned automatically to each hostname as well as the inventory name. This provides a flexible way to repeat the process, but manage multiple resource sets at the same time.

Advanced layout examples can be found by reading ra_inventory_layouts.

Note

Additional layout examples can be found in :dirs1.5:`the source code <workspace/layouts>`.

PinFile

A PinFile takes a topology and an optional layout, among other options, as a combined set of configurations as a resource for provisioning. An example Pinfile is shown.

# Example 1
dummy_cluster:
  topology: dummy-topology.yml
  layout: dummy-layout.yml

# Example 2
dummy-topo:
  topology:
    topology_name: "dummy_cluster" # topology name
    resource_groups:
    - resource_group_name: "dummy"
      resource_group_type: "dummy"
      resource_definitions:
      - name: "{{ distro | default('') }}web"
        role: "dummy_node"
        count: 3
      - name: "{{ distro | default('') }}test"
        role: "dummy_node"
        count: 1
  layout:
    inventory_layout:
      vars:
        hostname: __IP__
      hosts:
        example-node:
          count: 3
          host_groups:
          - example
        test-node:
          count: 1
          host_groups:
          - test

The PinFile collects the given topology and layout into one place. Many targets can be referenced in a single PinFile.

To use a PinFile with an Ansible Galaxy role, simply provide the role name as the resource_group_type. An example is shown below.

dummy-new:
  topology:
    topology_name: "dummy_cluster" # topology name
    resource_groups:
      - resource_group_name: "dummy"
        resource_group_type: "14rcole.ansible_role_lp_dummy"
        resource_definitions:
          - name: "{{ distro | default('') }}web"
            role: "dummy_node"
            count: 3
          - name: "{{ distro | default('') }}test"
            role: "dummy_node"
            count: 1

More detail about the PinFile can be found in the PinFiles document.

Additional PinFile examples can be found in :dirs1.5:`the source code <workspace>`

Provisioning (up)

Once a PinFile, topology, and optional layout are in place, provisioning can happen. Performing the command linchpin up should provision the resources and inventory files based upon the topology_name value. In this case, is dummy_cluster.

$ linchpin up
target: dummy_cluster, action: up
Action 'up' on Target 'dummy_cluster' is complete

Target              Run ID  uHash       Exit Code
-------------------------------------------------
dummy_cluster       70      0446                0

As you can see, the generated inventory file has the right data. This can be used in many ways, which will be covered elsewhere in the documentation.

$ cat inventories/dummy_cluster-0446.inventory
[example]
web-0446-0.example.net hostname=web-0446-0.example.net

[all]
web-0446-0.example.net hostname=web-0446-0.example.net

To verify resources with the dummy cluster, check /tmp/dummy.hosts

$ cat /tmp/dummy.hosts
web-0446-0.example.net
test-0446-0.example.net

Teardown (destroy)

As expected, LinchPin can also perform teardown of resources. A teardown action generally expects that resources have been provisioned. However, because Ansible is idempotent, linchpin destroy will only check to make sure the resources are up. Only if the resources are already up will the teardown happen.

The command linchpin destroy will look up the resources and/or topology files (depending on the provider) to determine the proper teardown procedure. The dummy Ansible role does not use the resources, only the topology during teardown.

$ linchpin destroy
target: dummy_cluster, action: destroy
Action 'destroy' on Target 'dummy_cluster' is complete

Target              Run ID  uHash       Exit Code
-------------------------------------------------
dummy_cluster       71      0446                0

Verify the /tmp/dummy.hosts file to ensure the records have been removed.

$ cat /tmp/dummy.hosts
-- EMPTY FILE --

Note

The teardown functionality is slightly more limited around ephemeral resources, like networking, storage, etc. It is possible that a network resource could be used with multiple cloud instances. In this way, performing a linchpin destroy does not teardown certain resources. This is dependent on each providers implementation.

Authentication

Some Examples for all Providers require authentication to acquire managing_resources. LinchPin provides tools for these providers to authenticate. The tools are called credentials.

Credentials

Credentials come in many forms. LinchPin wants to let the user control how the credentials are formatted. In this way, LinchPin supports the standard formatting and options for a provider. The only constraints that exist are how to tell LinchPin which credentials to use, and where they credentials data resides. In every case, LinchPin tries to use the data similarly to the way the provider might.

One method to provide AWS credentials that can be loaded by LinchPin is to use the INI format that the AWS CLI tool uses.

Credentials File

An example credentials file may look like this for aws.

$ cat aws.key
[default]
aws_access_key_id=ARYA4IS3THE3NO7FACEB
aws_secret_access_key=0Hy3x899u93G3xXRkeZK444MITtfl668Bobbygls

[herlo_aws1_herlo]
aws_access_key_id=JON6SNOW8HAS7A3WOLF8
aws_secret_access_key=Te4cUl24FtBELL4blowSx9odd0eFp2Aq30+7tHx9

See also

Examples for all Providers for provider-specific credentials examples.

To use these credentials, the user must tell LinchPin two things. The first is which credentials to use. The second is where to find the credentials data.

Using Credentials

In the topology, a user can specific credentials. The credentials are described by specifying the file, then the profile. As shown above, the filename is ‘aws.key’. The user could pick either profile in that file.

---
topology_name: ec2-new
resource_groups:
  - resource_group_name: "aws"
    resource_group_type: "aws"
    resource_definitions:
      - name: demo-day
        flavor: m1.small
        role: aws_ec2
        region: us-east-1
        image: ami-984189e2
        count: 1
    credentials:
      filename: aws.key
      profile: default

The important part in the above topology is the credentials section. Adding credentials like this will look up, and use the credentials provided.

Credentials Location

By default, credential files are stored in the default_credentials_path, which is ~/.config/linchpin.

Hint

The default_credentials_path value uses the interpolated :dirs1.5:`default_config_path <workspace/linchpin.conf#L22>` value, and can be overridden in the :docs1.5:`linchpin.conf`.

The credentials path (or creds_path) can be overridden in two ways.

It can be passed in when running the linchpin command.

$ linchpin -vvv --creds-path /dir/to/creds up aws-ec2-new

Note

The aws.key file could be placed in the default_credentials_path. In that case passing --creds-path would be redundant.

Or it can be set as an environment variable.

$ export CREDS_PATH=/dir/to/creds
$ linchpin -v up aws-ec2-new

See also

Commands (CLI)

Linchpin Command-Line Interface

Common Workflows

Common LinchPin Workflows

Managing Resources

Managing Resources

Examples for all Providers

Providers in Detail