Installing Chef Delivery, AWS
This topic guides you through setting up Chef Delivery for AWS provisioning with the
delivery-cluster cookbook and Chef provisioning, and validating the installation once Chef
Delivery is set up. With the delivery-cluster cookbook, you install the Chef Delivery cluster
on your own infrastructure using Chef provisioning and your license key.
Process Overview
The process for installing Chef Delivery using the delivery-cluster cookbook involves the
following tasks:







Create and configure a dedicated provisioning node
Configure the delivery-cluster cookbook to use Amazon Web Services (AWS)
provisioning
Run automation for delivery-cluster, which sets up a full Chef Delivery cluster and
gives you login credentials
Verify the Chef Delivery provisioning
Set up your workstation to develop projects in Chef Delivery
Configure Chef Delivery
Validate the installation by creating a cookbook and kicking off a pipeline
Prerequisites
Chef Delivery requires a license from Chef to install.
Supported Platforms
Chef Delivery may be run on the following platforms. Do not mix platform versions within the
Chef Delivery cluster.
OS
Versions
Ubuntu 12.04, 14.04
Centos 6.5, 6.6, 7
Redhat 6.5, 6.6, 7
Infrastructure
Chef Delivery has the following infrastructure requirements:
Function
vCPU
Free disk
space (in /var)
RAM
Provisioning Node
1 (2.5GHz)
1GB
8GB
Delivery Server
4
8GB
80GB
Chef Server (must be v12). See additional
information in note, below.
4
8GB
80GB
Supermarket Server
4
8GB
80GB
Build Nodes (at least 3, one for each Verify
phase: Lint, Syntax, and Functional)
2
4GB
60GB
Infrastructure Nodes (at least 4, one for each
environment stage: Acceptance, Union,
Rehearsal, and Delivered)
2 (for test
purposes)
4GB (for test
purposes)
60GB (for test
purposes)
NOTE: If you want to use your existing Chef server in the Chef Delivery cluster, the version of
the Chef server must be 12.2, it cannot have a high availability configuration.
Network and Ports
Chef Delivery has the following network and port requirements:
Ports
Description
Server
State
10000-10003 TCP Push Jobs
Chef server
LISTEN
8989
TCP Delivery Git (SCM)
Delivery server
LISTEN
443
TCP HTTP Secure
Chef server, Delivery server LISTEN
22
TCP SSH
All
80
TCP HTTP
Chef server, Delivery server LISTEN
5672
TCP Analytics MQ
n/a
n/a
10012-10013 TCP Analytics Messages/Notifier n/a
n/a
LISTEN
Install Chef Delivery
Begin installing Chef Delivery by creating a provisioning node to provision and maintain the
cluster.
Provisioning Node
Chef Delivery uses Chef provisioning to create, upgrade, and manage the cluster. A dedicated
provisioning node ensures a central place from which the Chef Delivery cluster is managed,
including managing those allowed to control it. Use your existing means to provision a
dedicated provisioning node with a supported platform. This machine does not need to be
powerful because it runs only provisioning code. Within Amazon Web Services (AWS), Chef
generally uses a t2.micro instance, which is a single CPU 2.5GHz, 1 GB of memory, and 8 GB of
disk space.
On the provisioning node:
1. Install a development environment:
For Debian-based (apt):
$ sudo apt-get install build-essential
For Red Hat Enterprise Linux-based (yum):
$ sudo yum groupinstall "Development Tools"
For Mac OS X:
$ sudo xcode-select --install
2. Install Git and configure your GitHub username and email, for help see: Installing Git and
First-Time Git Setup.
3. Install the Chef Development Kit. Be sure to set the system Ruby; for details, see Add
Ruby to $PATH.
4. Install the Chef Delivery knife push utility:
$ chef gem install knife-push
5. Move your Chef Delivery license key to ~/delivery.license.
6. Clone the Git repo for the Chef Delivery cluster:
$ git clone https://github.com/opscode-cookbooks/delivery-cluster.git
~/delivery-cluster
AWS Settings
The following section details settings for Chef Delivery with AWS provisioning.
Option
Description
Environment
Name
This name becomes the default name for your Chef Delivery cluster
environment, enterprise, and organization. The naming convention prohibits
the use of capital letters, but should otherwise be alphanumeric.
Key Name
This is the name of your Amazon Web Services (AWS) .pem credential file.
Option
Description
Enter only the name, without the extension. This file is typically located in the
~.ssh directory, and is associated with the Key name pair value in Amazon
Web Services (AWS) console: https://console.aws.amazon.com/iam/home.
Image ID,
Subnet ID, and
Security Group
ID
Use existing
chef-server
This is the information that Amazon Web Services (AWS) needs to provision
the nodes. It is wise to verify that the given defaults are actually the ones that
you want by opening your Amazon Web Services (AWS) console and checking.
Accept the default, “no”, if you want delivery-cluster to configure a Chef
server for you (you will be asked for the IP address of the server). Enter “yes”
if you have a configured Chef server that you want to use and do not need
delivery-cluster to configure one for you. If you do specify “yes”, please
note that the version of the Chef server must be 12.2, it cannot have a high
availability configuration. In addition, a special entry must be made to the
environments configuration file (environments/NAME_OF_ENV.json) similar
to:
"chef-server": {
"organization": "test",
"existing": true,
"host": "chef-server.example.com"
},
License File
The path to your Chef Delivery license file; the default path is your home
directory.
Enable
Supermarket
Enter “yes” if you intend to run cookbooks through your Chef Delivery
pipeline.
AWS Provisioning
This section describes setting up a Chef Delivery cluster to use AWS for provisioning. This
process creates at least five machines, depending on the selected configuration options. The
typical (and recommended) starting setup is five nodes in addition to the provisioning node:





One to run the Chef server
One to run Chef Delivery
Three to act as build nodes
One to run the Chef Supermarket server; for cookbook dependency resolution, using
Chef Supermarket server is required
For a test environment, use stand-in infrastructure nodes; we recommend four, one for
each environment stage: Acceptance, Union, Rehearsal, and Delivered
On the provisioning node:
1. Configure your Amazon Web Services (AWS) key credentials in two files, ~/.aws/config
and ~/.aws/credentials as shown, with the text editor of your choice:
For ~/.aws/config:
[default]
region = us-west-2
output = text
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_KEY
For ~/.aws/credentials:
[default]
aws_access_key_id = YOUR_ACCESS_KEY_ID
aws_secret_access_key = YOUR_SECRET_KEY
2. Transfer the private key of the key pair (private-key) that you will use to create the
servers in Amazon Web Services (AWS), to ~/.ssh on the provisioning node. If you do
not have one already, use this link and create one: Amazon EC2 Key Pairs.
3. Ensure that an Amazon Web Services (AWS) security group is available, with the
appropriate ports open. See Network and Ports for details.
4. Move to the delivery-cluster directory: cd ~/delivery-cluster.
5. Create a configuration file to drive Chef provisioning:
$ rake setup:generate_env
This opens a wizard, the following code block shows the interactions. Variables you
must enter are shown in ALL CAPs. Default values are shown in square brackets;
pressing ENT sets the default value. This command creates an
environments/ENV_NAME.json file with the specified settings that you can modify if
need be.
$ rake setup:generate_env
Gathering Cluster Information
Provide the following information to generate your environment.
Global Attributes
Environment Name [test]:
Cluster ID [test]:
Available Drivers: [ aws | ssh | vagrant ]
Driver Name [vagrant]: aws
Driver Information [aws]
Key Name: MY-KEY-NAME
SSH Username [ubuntu]:
Image ID [ami-3d50120d]:
Subnet ID [subnet-19ac017c]:
Security Group ID [sg-cbacf8ae]:
Use private ip for ssh? [yes]:
Would you like to configure Proxy Settings? [no]:
Chef Server
Organization Name [test]:
Use existing chef-server? [no]:
Flavor [c3.xlarge]:
Delivery Server
Package Version [latest]:
Enterprise Name [test]:
License File [/Users/YOUR-NAME/delivery.license]:
Flavor [c3.xlarge]:
Analytics Server
Enable Analytics? [no]:
Supermarket Server
Enable Supermarket? [no]: yes
Flavor [c3.xlarge]:
Build Nodes
Number of Build Nodes [1]: 3
Flavor [c3.large]:
Specify a delivery-cli artifact? [no]:
Rendering Environment => environments/test.json
6. Important! Export your new environment by executing the following: (example uses
test as the environment name):
$ export CHEF_ENV=test
NOTE: You can name your environment file anything, but note the name for future reference.
Run Provisioning
To run provisioning issue this command from inside the delivery-cluster directory:
$ rake setup:cluster
NOTES:

If the first converge fails on the build nodes, try running the above step again. The Chef
Delivery cluster is complicated and sometimes there are timeouts before the setup
completes.

Running provisioning creates the requested instances on Amazon Web Services (AWS). If
there are any failures, check the Amazon Web Services (AWS) console for nodes without
names. Nodes without names can be removed.
Verify Provisioning
To verify provisioning do the following, from inside the delivery-cluster directory:
1. Run this command:
$ rake info:list_core_services
2. Navigate to your CHEF_SERVER_URL, and login with delivery:delivery.
3. Click on Nodes. You’ll see your Chef Delivery server, your build nodes, and nodes for any
other optional features that you installed.
4. Get the credentials and URL with this command:
$ rake info:delivery_creds
5. Navigate to the Web Login and use the admin credentials to log in.
6. Run this command:
$ knife node status
All build nodes should report available.
Configure Delivery
In Chef Delivery there are multiple levels of organization: enterprises, organizations, and
projects. The provisioning step created the initial enterprise you specified in your environment
file as the first name option. Enterprises are designed to provide units of multi-tenancy with
separate sets of organizations and users. Begin configuring Chef Delivery by adding users and
organizations.
NOTE: Chef Delivery, by default, hosts a Git server that you interact with through the delivery
CLI commands. Additionally, you can integrate GitHub Enterprise or GitHub.com.
Add Users
Chef recommends that the default admin account not be used after Chef Delivery is installed.
Instead, use the following web UI procedure to create a new user for yourself. Then log out as
admin and log back in as the user you created.
1. Log into the Chef Delivery web UI as an administrator.
2. Select Users from the drop-down menu on the upper right.
The Users list page opens. You can use the search filter in the upper right corner to
make sure that the user is not already added.
3. Click the plus sign (+) next to Add a New User.
4. In the Add a New User text area, select Internal, which means you are manually adding
the user to the Chef Delivery database. The selection box is grey for the active selection.
Options for Name and Email, first name, last name, email address, and Security
Information, a login name and password, appear.
5. Enter the appropriate information.
Select Admin in the Roles Within the Enterprise option list.
Click Save and Close.
The User list page opens and a status message appears.
To check, click Edit and verify that the user details are present.
Create Organizations
Chef suggests creating a sandbox organization in which a test project can be run for verification
and testing purposes. Additionally, it is recommended that each organization created
corresponds to a working directory within which projects for that organization are managed.
To create an organization, do the following:
1. Log into the Chef Delivery web UI as the Admin user you created; use this command
(run in your delivery-cluster directory) to find the credentials:
$ rake info:delivery_creds
2. Select Organizations from the drop-down menu on the upper right. The organizations
list page opens.
3. Click the plus sign (+) next to Add a New Organization.
4. Enter an organization name in the text area, and then click Save & Close.
Setup a Workstation
After the Chef Delivery cluster has been created, team members need to install the Chef
Delivery CLI on their workstations.
NOTE: If you reuse the provisioning node in the previous section of this installation guide as a
workstation for using Chef Delivery, you can skip steps 1 through 4 below.
Setting up a workstation requires that you do the following:
1. Install a development environment:
Debian based (apt): apt-get install build-essential
RHEL based (yum): yum groupinstall "Development Tools"
OS X: xcode-select --install
2. Install Git: http://git-scm.com/book/en/v2/Getting-Started-Installing-Git. Also, make
sure that your SSH public key is added to your user profile in Chef Delivery.
3. Install the Chef Development Kit: https://downloads.chef.io/chef-dk/. Be sure to follow
all instructions; for example, to run delivery commands you must set the following:
o
o
o
echo 'eval "$(chef shell-init bash)"' >> ~/.bash_profile
4. Install the Chef knife push utility: chef gem install knife-push.
5. Install and configure the Chef Delivery CLI, described next.
Install the Chef Delivery CLI
The delivery command line interface (CLI) is required for the workstation and for many Chef
Delivery functions.
Mac OS X
For Mac OS X, run the following commands:
1. Download the package https://s3.amazonaws.com/delivery-packages/cli/deliverycli0.0.0%2B20151118205039-1.pkg.
2. Click the package and install.
3. You may need to allow the package on the machine. To do this, open System
Preferences -> Security & Privacy, and click Allow. You can also go to your Downloads
folder and right-click the package to open the installer; simply accept the agreement
and the command line interface is installed.
Ubuntu
For Ubuntu 14.04, run the following commands:
$ curl
https://packagecloud.io/install/repositories/chef/current/script.deb.sh |
sudo bash
$ sudo apt-get install delivery-cli
RHEL, SUSE
For Red Hat Enterprise Linux 6.5 and SUSE Enterprise Linux Server 6.5, run the following
commands:
$ curl
https://packagecloud.io/install/repositories/chef/current/script.rpm.sh |
sudo bash
$ sudo yum install delivery-cli.rpm
Microsoft Windows
For Microsoft Windows:
1. Download the binary from https://s3.amazonaws.com/delivery-packages/cli/deliverycli-0.0.0%2B20151020165859-1-x64.msi.
2. Install Git (2.5) on Microsoft Windows: https://github.com/git-forwindows/git/releases/download/v2.5.3.windows.1/Git-2.5.3-64-bit.exe.
WARNING: Git 2.6 won’t work.
3. Modify the PATH environment variable to the path to the location of the delivery
command line interface. For example: C:\\chef\\delivery-cli\\bin
Configure the Chef Delivery CLI
Before you use the Chef Delivery CLI from a workstation, you need to provide it with details
such as the URL of the Delivery server, and the names of the relevant enterprise, organization,
and user. The delivery setup subcommand creates a configuration file named
.delivery/cli.toml with the required information.
The placement of the .delivery directory in your file hierarchy is significant. Like Git, Chef
Delivery CLI commands search the current directory and parent directories to locate server
settings. Because server settings are unique to an organization, we recommend that you create
a directory for each organization as required and run the delivery setup command from each
of those directories.
$ delivery setup --server=DELIVERY_SERVER_IP_ADDR --ent=ENTERPRISE -org=ORGANIZATION --user=USERNAME
Authenticate
You must authenticate access to the Chef Delivery Git server, which runs on port 8989 by
default.
NOTE: This authentication step is for the Git server that comes with Chef Delivery. If you
actually integrate with GitHub, you do not need to do this.
Run the following SSH command; enter “Yes” when prompted.
$ ssh -l USER@ENTERPRISE -p 8989 DELIVERY_SERVER_IP_ADDRESS
You should get output similar to this:
$ ssh -l sally@chef -p 8989 10.194.11.99
The authenticity of host '[10.194.11.99]:8989 ([10.194.11.99]:8989)' can't
be established.
RSA key fingerprint is ba:db:0c:97:f8:d4:6d:0f:0b:57:0d:0f:0e:a4:15:01.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '[10.194.11.99]:8989' (RSA) to the list of known
hosts.
channel 0: protocol error: close rcvd twice
Hi sally@chef! You've successfully authenticated, but Chef Delivery does
not provide shell access.
Connection to 10.194.11.99 closed.
Your workstation is now set up for use with Chef Delivery.
Validate the Installation
The surest way to validate your Chef Delivery installation is by creating a cookbook and
submitting it to GitHub, which kicks off a pipeline allowing you to see how the process works.
If a project is a cookbook, we recommend starting with delivery-truck, an open source build
cookbook created for driving cookbook pipelines in Chef Delivery. You can customize some
aspects of delivery-truck through your project’s .delivery/config.json. To have more
control, or to opt-out of some of the behavior of delivery-truck, create a wrapper build
cookbook. To learn more about wrapper cookbooks, see Chef Supermarket -> Wrapper
Cookbooks and the blog post Doing Wrapper Cookbooks Right.
NOTE: These instructions assume that you will use Chef Delivery as your source code source of
truth and that Chef Delivery is not integrated with GitHub Enterprise or GitHub.com.
This topic describes the recommended setup for a Chef cookbook project using Chef Delivery.
The following example shows how to create a cookbook, with project and pipeline, configure it
to be built with delivery-truck, and then import the new cookbook into Chef Delivery. From
your workstation, do the following:
1. Make a working directory (workspace in the example):
$ mkdir ~/workspace && cd ~/workspace
2. Setup the delivery-cli to, by default, contact the appropriate delivery server at
SERVER, with a default ENTERPRISE and ORGANIZATION:
$ delivery setup --server=SERVER --ent=ENTERPRISE --org=ORGANIZATION -user=USERNAME
NOTE: The server, enterprise, organization, and user must already exist.
3. Create a cookbook:
$ chef generate cookbook NEW-COOKBOOK-NAME
$ cd NEW-COOKBOOK-NAME
This uses the Chef Development Kit to generate a new cookbook, including a default
recipe and default ChefSpec tests.
4. Create an initial commit on the master branch (use git status to verify the change):
$ git add .
$ git commit -m 'Initial Commit'
Running chef generate initialized a Git repository automatically for this cookbook. If
you created the build cookbook manually, initialize the Git repository with the git init
command.
5. Get an API token for Chef Delivery:
$ delivery token
6. Initialize the cookbook for Chef Delivery:
$ delivery init
This creates a new project in Chef Delivery, pushes the master branch, creates a feature
branch, generates a default Chef Delivery project configuration file, pushes the first
change for review, and then opens a browser window that shows the change.