Prepare the Source Instance and Volume

Start an instance in the source region

Here I’ll start an instance in us-west-1a where I have the EBS image I want to copy. In this case I’ll use the image I want to copy, but it could be any image as long as its in the same region as the EBS AMI image that is to be copied. Though we are going to use the instance info to figure out some parameters for creating the new AMI. So if you don’t make the source instance the same AMI as the one you are copying you will need to supply some of the parameters yourself.

You can use a tool like ElasticFox to do the following creating of instances. Here we’ll do it with command line tools.

Set some Shell source variables on host machine

Just to make using these instructions as a cookbook, we’ll have some shell variables that you can set once and then all the instructions will use the variables so you can just cut and paste the instructions into your shell.

src_keypair=id_runa-staging-us-west
src_fullpath_keypair=~/.ssh/runa/id_runa-staging-us-west
src_availability_zone=us-west-1a
src_instance_type=m1.large
src_region=us-west-1
src_origin_ami=ami-1f4e1f5a
src_device=/dev/sdh
src_dir=/src
src_user=ubuntu

Start up the source instance and capture the instanceid

src_instanceid=$(ec2-run-instances \
  --key $src_keypair \
  --availability-zone $src_availability_zone \
  --instance-type $src_instance_type \
  $src_origin_ami \
  --region $src_region  | \
  egrep ^INSTANCE | cut -f2)
echo "src_instanceid=$src_instanceid"

# Wait for the instance to move to the “running” state
while src_public_fqdn=$(ec2-describe-instances --region $src_region "$src_instanceid" | \
  egrep ^INSTANCE | cut -f4) && test -z $src_public_fqdn; do echo -n .; sleep 1; done
echo src_public_fqdn=$src_public_fqdn

This should loop till you see something like:

$ echo src_public_fqdn=$src_public_fqdn
src_public_fqdn=ec2-184-72-2-93.us-west-1.compute.amazonaws.com

Create a volume from the EBS AMI snapshot

Normally when starting an EBS AMI instance, it automatically created a volume from the snapshot associated with the AMI. Here we create the volume from the snapshot ourselves

# Get the volume id
ec2-describe-instances --region $src_region "$src_instanceid" > /tmp/src_instance_info
src_volumeid=$(egrep ^BLOCKDEVICE /tmp/src_instance_info | cut -f3); echo $src_volumeid
# Now get the snapshot id from the volume id
ec2-describe-volumes --region $src_region $src_volumeid | egrep ^VOLUME > /tmp/volume_info
src_snapshotid=$(cut /tmp/volume_info | cut -f2)
echo $src_snapshotid
src_size=$(cut /tmp/volume_info | cut -f2)
echo $src_size
# Create a new volume from the snapshot
src_volumeid=$(ec2-create-volume --region $src_region --snapshot $src_snapshotid -z $src_availability_zone | egrep ^VOLUME | cut -f2)
echo $src_volumeid

Mount the EBS Image of the AMI you want to copy

Now we’ll mount the EBS AMI image as a plain mount on the running source instance. In this case we’re going to use the same image as we launched, but it doesn’t have to be the same image or even the same architecture.

ec2-attach-volume --region $src_region $src_volumeid -i $src_instanceid -d $src_device

You should see something like:

ATTACHMENT	vol-6e7fee06	i-fb0804be	/dev/sdh	attaching	2010-03-14T09:02:58+0000

Prepare the Destination Instance and Volume

Set some Shell destination variables on host machine

You’ll want to tune these to your needs. This example makes the destination size the same as the source. You could make the destination an arbitrary size as long as it fits the source data.

dst_keypair=runa-production-us-east
dst_fullpath_keypair=~/.ssh/runa/id_runa-production-us-east
dst_availability_zone=us-east-1b
dst_instance_type=m1.large
dst_region=us-east-1
dst_origin_ami=ami-7d43ae14
dst_size=$src_size
dst_device=/dev/sdh
dst_dir=/dst
dst_user=ubuntu

Start up the destination instance and capture the dst_instanceid

dst_instanceid=$(ec2-run-instances \
  --key $dst_keypair \
  --availability-zone $dst_availability_zone \
  --instance-type $dst_instance_type \
  $dst_origin_ami \
  --region $dst_region  | \
  egrep ^INSTANCE | cut -f2)
echo "dst_instanceid=$dst_instanceid"

# Wait for the instance to move to the “running” state
while dst_public_fqdn=$(ec2-describe-instances --region $dst_region "$dst_instanceid" | \
  egrep ^INSTANCE | cut -f4) && test -z $dst_public_fqdn; do echo -n .; sleep 1; done
echo dst_public_fqdn=$dst_public_fqdn

This should loop till you see something like:

$ echo dst_public_fqdn=$dst_public_fqdn
dst_public_fqdn=ec2-184-73-71-160.compute-1.amazonaws.com

Create an empty destination volume

dst_volumeid=$(ec2-create-volume --region $dst_region --size $dst_size -z $dst_availability_zone | egrep ^VOLUME | cut -f2)
echo $dst_volumeid

Mount the EBS Image of the AMI you want to copy

Now we’ll mount the EBS AMI image as a plain mount on the running source instance. In this case we’re going to use the same image as we launched, but it doesn’t have to be the same image or even the same architecture.

ec2-attach-volume --region $dst_region $dst_volumeid -i $dst_instanceid -d $dst_device

You should see something like:

ATTACHMENT	vol-450ed02c	i-65be1f0e	/dev/sdh	attaching	2010-03-14T09:39:20+0000

Copy the data from the Source Volume to the Destination Volume

Copy your credentials to the source machine

We’re going to use rsync to copy from the source to the destination tunneled thru ssh. This eliminates any issues with EC2 security groups. But it does mean you have to copy an ssh private key to the source machine that will then be able to access the destination machine via ssh.

scp -i $src_fullpath_keypair $dst_fullpath_keypair ${src_user}@${src_public_fqdn}:.ssh

Mount the source and destination volumes on their instances

ssh -i $src_fullpath_keypair ${src_user}@${src_public_fqdn} sudo mkdir -p $src_dir
ssh -i $src_fullpath_keypair ${src_user}@${src_public_fqdn} sudo mount $src_device $src_dir
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo mkfs.ext3 -F $dst_device
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo mkdir -p $dst_dir
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo mount $dst_device $dst_dir

Get the FQDN of the Amazon internal address of the destination machine

We’re assuming that the dst instance is the us-east equivalent base AMI of the us-west source base AMI so we can use these kernel and ramdisk to build the new AMI later.

ec2-describe-instances --region $dst_region "$dst_instanceid" > /tmp/dst_instance_info
dst_internal_fqdn=$(egrep ^INSTANCE /tmp/dst_instance_info | cut -f5); echo $dst_internal_fqdn
dst_kernel=$(egrep ^INSTANCE /tmp/dst_instance_info | cut -f13); echo $dst_kernel
dst_ramdisk=$(egrep ^INSTANCE /tmp/dst_instance_info | cut -f14) ;echo $dst_ramdisk

Commands to run on the source machine

You could do the rsync by logging into the source machine and do the following. I tried to do this by using ssh commands, but the fact that the first ssh from source to destination has to be authenticated was a blocker for me. You could log into the source machine and then sudo ssh to the destination machine (you have to do sudo ssh since the rsync has to be run with sudo and the keys are stored separately for the sudo user and the regular user).
I’ll show both ways.
Here’s how you can ssh to the source machine:

ssh -i $src_fullpath_keypair ${src_user}@${src_public_fqdn}

Set up some shell variables on the source machine shell environment

# This is the key you just copied over
dst_fullpath_keypair=~/.ssh/id_runa-production-us-east
# You need to use the Public FQDN of the destination since its cross region
dst_keypair=runa-production-us-east
src_public_fqdn=ec2-184-72-2-93.us-west-1.compute.amazonaws.com
dst_public_fqdn=ec2-184-73-71-160.compute-1.amazonaws.com
dst_user=ubuntu
src_user=ubuntu
src_dir=/src
dst_dir=/dst

Do the rsync

We are using the rsync options

• P Keep partial transferred files and Show Progress
• H Preserve Hard Links
• A Preserve ACLs
• X Preserve extended attributes
• a Archive mode
• z Compress files for transfer

rsync -PHAXaz --rsh "ssh -i /home/${src_user}/.ssh/id_${dst_keypair}" --rsync-path "sudo rsync" ${src_dir}/ ${dst_user}@${dst_public_fqdn}:${dst_dir}/

If you want to do the rsync from your local host

I found that I still had to log into the source instance

ssh -i $src_fullpath_keypair ${src_user}@${src_public_fqdn}

and then on the source instance do:

sudo ssh -i /home/${src_user}/.ssh/id_${dst_keypair} ${dst_user}@${dst_public_fqdn}

and accept “The authenticity of host” for the first time so the destination host is in the known keys of the sudo user
Then back on your local host you can issue the remote command that will run on the source instance and rsync to the destination host:

ssh -i $src_fullpath_keypair ${src_user}@${src_public_fqdn} sudo "rsync -PHAXaz --rsh \"ssh -i /home/${src_user}/.ssh/id_${dst_keypair}\" --rsync-path \"sudo rsync\" ${src_dir}/ ${dst_user}@${dst_public_fqdn}:${dst_dir}/"

Complete the new AMI from your Local Host

The remaining steps will be done back on your local host. This assumes that the shell variables we set up earlier are still there.

Some Cleanup for new Region

Ubuntu has their apt sources tied to the region you are in. So we have to update the apt sources for the new region.
We’ll do this by chrooting to the mount /dst directory and running some commands as if they were being run on an ami with the /dst image. We might as well update things at the same time to the latest packages.

# Allow network access from chroot environment
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo cp /etc/resolv.conf $dst_dir/etc/

# Upgrade the system and install packages
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo -E chroot $dst_dir mount -t proc none /proc
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo -E chroot $dst_dir mount -t devpts none /dev/pts

cat <<EOF > /tmp/policy-rc.d
#!/bin/sh
exit 101
EOF
scp -i $dst_fullpath_keypair /tmp/policy-rc.d ${dst_user}@${dst_public_fqdn}:/tmp
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo mv /tmp/policy-rc.d $dst_dir/usr/sbin/policy-rc.d

ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} chmod 755 $dst_dir/usr/sbin/policy-rc.d

# This has to be done to set up the Locale & apt sources
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} DEBIAN_FRONTEND=noninteractive sudo -E chroot $dst_dir /usr/bin/ec2-set-defaults

# Update the apt sources
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} DEBIAN_FRONTEND=noninteractive sudo -E chroot $dst_dir apt-get update

# Optionally update the packages
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} DEBIAN_FRONTEND=noninteractive sudo -E chroot $dst_dir apt-get dist-upgrade -y

# Optionally update your gems
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo -E chroot $dst_dir gem update --system
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo -E chroot $dst_dir gem update

Clean up from the building of the image

ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo chroot $dst_dir umount /proc
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo -E chroot $dst_dir umount /dev/pts
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo -E rm -f $dst_dir/usr/sbin/policy-rc.d

There are a few more shell variables we’ll need

I got the kernel and ramdisk from the destination instance since it has the alestic.com us-east-1 equivalent base AMI to the us-west-1 one that we are copying from.

# Some info for creating the name and description
codename=karmic
release=9.10
tag=server

# Make sure you set this as appropriate
# 64bit
arch=x86_64

# You will need to set the aki and ari values base on the actual base AMI you used
# It will be different for different regions.  These are set for x86_64 and us-east-1
ebsopts="--kernel=${dst_kernel} --ramdisk=${dst_ramdisk}"
ebsopts="$ebsopts --block-device-mapping /dev/sdb=ephemeral0"

now=$(date +%Y%m%d-%H%M)
# Make this specific to what you are making
chef_version="0.8.6"
prefix=runa-chef-${chef_version}-ubuntu-${release}-${codename}-${tag}-${arch}-${now}
description="Runa Chef ${chef_version} Ubuntu $release $codename $tag $arch $now"

Snapshot the Destination Volume and register the new AMI in the destination region

# Unmount the destination filesystem
ssh -i $dst_fullpath_keypair ${dst_user}@${dst_public_fqdn} sudo umount $dst_dir

# Detach the Destination Volume (it may speed up the snapshot)
ec2-detach-volume --region $dst_region "$dst_volumeid"

# Make the snapshot
dst_snapshotid=$(ec2-create-snapshot -region $dst_region -d "$description" $dst_volumeid | cut -f2)

# Wait for snapshot to complete. This can take a while
while ec2-describe-snapshots --region $dst_region "$dst_snapshotid" | grep -q pending
  do echo -n .; sleep 1; done

# Register the Destination Snapshot as a new AMI in the Destination Region
new_ami=$(ec2-register \
  --region $dst_region \
  --architecture $arch \
  --name "$prefix" \
  --description "$description" \
  $ebsopts \
  --snapshot "$dst_snapshotid")
echo $new_ami

Conclusion

You should now have a shiny new ami in your destination region. Use the value of $new_ami to start a new instance in your destination region using your favorite tool or technique.

• Mar 3, 2010 Added call to script ec2-set-defaults that is normally called on ec2 init that sets the locale and apt sources for EC availability Zone

Introduction

Opscode has officially released 0.8.x of Chef. It is now even more fabulous. I’ve been using the pre-release version for the last couple of months and it is rock steady and very powerful. I’ll be having a post soon on how I used it to deploy a pretty complicated cloud stack with multiple Rails/Mysql/Nginx/Unicorn/Postfix apps for front-ends, and a back end made up of a mix of a Clojure/Swarmiji distributed processing swarm, HBase/Hadoop, Redis, RabbitMQ.

But first, I needed to upgrade my Amazon EC2 AMIs for the officially released Chef 0.8.x. I also wanted to try the EBS Boot image as a basis for the AMI.

This is an update to my earlier post, Creating an Amazon EC2 AMI for Opscode Chef 0.8, but now using the official Opscode 0.8.x Gems instead of building your own Gems. A lot of the content is the same, but you can consider this mostly superceding the older one except where mentioned otherwise. This version will use the EBS Boot AMIs as per Eric Hammond’s Tutorial Building EBS Boot AMIs Using Canonical’s Downloadable EC2 Images. Much of this is blog post is taken from Eric’s blog post but in the context of creating a Chef Client base AMI and a Chef Server. Note that Opscode now has their own AMIs, including ones for Chef 0.8.4, but as of this writing, they do not have AMIs for Amazon us-west.

Setup

Prerequisites

On your host development machine (ie your laptop or whatever machine you are developing from) you should have already installed:

• ec2-api-tools and ec2-ami-tools (these assume you have a modern Java run time setup)
• chef-0.8.4 or later chef client gem (which implies the entire ruby 1.8.x and rubygems toolchain)

Set some Shell variables on host machine

Just to make using these instructions as a cookbook, we’ll have some shell variables that you can set once and then all the instructions will use the variables so you can just cut and paste the instructions into your shell.

keypair=id_runa-staging-us-west
fullpath_keypair=~/.ssh/runa/id_runa-staging-us-west
availability_zone=us-west-1a
instance_type=m1.large
region=us-west-1

# Pick one of these two AMIs (Note that it will be different for different Amazon Regions)
# 32bit AMI
origin_ami=ami-fd5100b8
#64bit AMI
origin_ami=ami-ff5100ba

Start up an instance and capture the instanceid

instanceid=$(ec2-run-instances \
  --key $keypair \
  --availability-zone $availability_zone \
  --instance-type $instance_type \
  $origin_ami \
  --region $region  |
  egrep ^INSTANCE | cut -f2)
echo "instanceid=$instanceid"

Wait for the instance to move to the “running” state

while host=$(ec2-describe-instances --region $region "$instanceid" |
  egrep ^INSTANCE | cut -f4) && test -z $host; do echo -n .; sleep 1; done
echo host=$host

This should loop till you see something like:

$ echo host=$host
host=ec2-184-72-2-93.us-west-1.compute.amazonaws.com

Upload your certs

This assumes that your Amazon certs are in ~/.ec2

rsync                            \
 --rsh="ssh -i $fullpath_keypair" \
 --rsync-path="sudo rsync"      \
 ~/.ec2/{cert,pk}-*.pem         \
 ubuntu@$host:/mnt/

Connect to the instance

ssh -i $fullpath_keypair ubuntu@$host

Update the Amazon ec2 tools on the instance

export DEBIAN_FRONTEND=noninteractive
echo "deb http://ppa.launchpad.net/ubuntu-on-ec2/ec2-tools/ubuntu karmic main" |
  sudo tee /etc/apt/sources.list.d/ubuntu-on-ec2-ec2-tools.list &&
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9EE6D873 &&
sudo apt-get update &&
sudo -E apt-get dist-upgrade -y &&
sudo -E apt-get install -y ec2-api-tools

Set some parameters on instance shell environment

Again this makes it easier to cut and paste the instructions.

codename=karmic
release=9.10
tag=server
region=us-west-1
availability_zone=us-west-1a
if [ $(uname -m) = 'x86_64' ]; then
  arch=x86_64
  arch2=amd64
  # You will need to set the aki and ari values base on the actual base AMI you used
  # It will be different for different regions.  These are set for us-west-1
  ebsopts="--kernel=aki-7f3c6d3a --ramdisk=ari-cf2e7f8a"
  ebsopts="$ebsopts --block-device-mapping /dev/sdb=ephemeral0"
else
  arch=i386
  arch2=i386
  # You will need to set the aki and ari values base on the actual base AMI you used
  # It will be different for different regions. These are set for us-west-1
  ebsopts="--kernel=aki-773c6d32 --ramdisk=ari-c12e7f84"
  ebsopts="$ebsopts --block-device-mapping /dev/sda2=ephemeral0"
fi

Download and unpack the latest released Ubuntu server image file

This contains the output of vmbuilder as run by Canonical.

imagesource=http://uec-images.ubuntu.com/releases/$codename/release/unpacked/ubuntu-$release-$tag-uec-$arch2.img.tar.gz
image=/mnt/$codename-$tag-uec-$arch2.img
imagedir=/mnt/$codename-$tag-uec-$arch2
wget -O- $imagesource |
  sudo tar xzf - -C /mnt
sudo mkdir -p $imagedir
sudo mount -o loop $image $imagedir

Bring the packages on the instance up to date

# Allow network access from chroot environment
sudo cp /etc/resolv.conf $imagedir/etc/

# Fix what I consider to be a bug in vmbuilder
sudo rm -f $imagedir/etc/hostname

# Add multiverse
sudo perl -pi -e 's%(universe)$%$1 multiverse%' \
$imagedir/etc/ec2-init/templates/sources.list.tmpl

# Add Alestic PPA for runurl package (handy in user-data scripts)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu karmic main" |
sudo tee $imagedir/etc/apt/sources.list.d/alestic-ppa.list
sudo chroot $imagedir \
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571

# Add ubuntu-on-ec2/ec2-tools PPA for updated ec2-ami-tools
echo "deb http://ppa.launchpad.net/ubuntu-on-ec2/ec2-tools/ubuntu karmic main" |
sudo tee $imagedir/etc/apt/sources.list.d/ubuntu-on-ec2-ec2-tools.list
sudo chroot $imagedir \
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9EE6D873

# Upgrade the system and install packages
sudo chroot $imagedir mount -t proc none /proc
sudo chroot $imagedir mount -t devpts none /dev/pts

cat <<EOF > /tmp/policy-rc.d
#!/bin/sh
exit 101
EOF
sudo mv /tmp/policy-rc.d $imagedir/usr/sbin/policy-rc.d

chmod 755 $imagedir/usr/sbin/policy-rc.d
DEBIAN_FRONTEND=noninteractive

# It seems this has to be done to set up the Locale & apt sources
sudo -E chroot $imagedir /usr/bin/ec2-set-defaults

# Update the apt sources and packages
sudo chroot $imagedir apt-get update &&
sudo -E chroot $imagedir apt-get dist-upgrade -y &&
sudo -E chroot $imagedir apt-get install -y runurl ec2-ami-tools

Install Chef Client and other customizations

Install Ruby and needed packages

sudo -E chroot $imagedir apt-get -y install ruby ruby1.8-dev libopenssl-ruby1.8 rdoc ri irb \
build-essential wget ssl-cert git-core rake librspec-ruby libxml-ruby \
thin couchdb zlib1g-dev libxml2-dev emacs23-nox

Install Rubygems

Rubygems will be installed from source since debian/ubuntu try to control rubygems upgrades. If you don’t care you can install it via apt-get install rubygems

cd $imagedir/tmp
wget http://rubyforge.org/frs/download.php/69365/rubygems-1.3.6.tgz
tar zxf rubygems-1.3.6.tgz
cd rubygems-1.3.6
sudo -E chroot $imagedir ruby /tmp/rubygems-1.3.6/setup.rb
cd ..
sudo rm -rf rubygems-1.3.6
sudo -E chroot $imagedir ln -sfv /usr/bin/gem1.8 /usr/bin/gem
sudo -E chroot $imagedir gem sources -a http://gems.opscode.com
sudo -E chroot $imagedir gem sources -a http://gemcutter.org
sudo -E chroot $imagedir gem install chef

Use Opscode Chef Solo Bootstrap to configure the Chef Client

The following will set up all the default paths and directories as well as install and configure runit to start and monitor the chef-client. Originally I shied away from runit, but this time I’m going as Opscode Vanilla as possible and they like runit.

Create the solo.rb file

All of the following files should be done in $imagedir as we are going to have to run this as chroot to $imagedir

Create $imagedir/solo.rb with an editor and put in the following:

file_cache_path "/tmp/chef-solo"
cookbook_path "/tmp/chef-solo/cookbooks"
recipe_url "http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz"

Create the chef.json file

Create $imagedir/chef.json with the following. (set the server_fqdn to the chef server you are using):

{
  "bootstrap": {
    "chef": {
      "url_type": "http",
      "init_style": "runit",
      "path": "/srv/chef",
      "serve_path": "/srv/chef",
      "server_fqdn": "chef-server-staging.runa.com"
    }
  },
  "run_list": [ "recipe[bootstrap::client]" ]
}

Run the chef-solo command

sudo -E chroot $imagedir chef-solo -c solo.rb -j chef.json \
  -r http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz

I had to run it 3 times before it completed with no errors.
After it does work, clean up the chef-solo stuff:

sudo rm $imagedir/{solo.rb,chef.json}

Update the client config file

The Chef Solo Client bootstrap process creates an /etc/chef/client.rb that is not ideal for Amazon EC2. The following will replace that:

mkdir -p /etc/chef
chown root:root /etc/chef
chmod 755 /etc/chef

Put the following in /etc/chef/client.rb:

# Chef Client Config File
# Automatically grabs configuration from ohai ec2 metadata.

require 'ohai'
require 'json'

o = Ohai::System.new
o.all_plugins
chef_config = JSON.parse(o[:ec2][:userdata])
if chef_config.kind_of?(Array)
  chef_config = chef_config[o[:ec2][:ami_launch_index]]
end

log_level        :info
log_location     STDOUT
node_name        o[:ec2][:instance_id]
chef_server_url  chef_config["chef_server"]

unless File.exists?("/etc/chef/client.pem")
  File.open("/etc/chef/validation.pem", "w", 0600) do |f|
    f.print(chef_config["validation_key"])
  end
end

if chef_config.has_key?("attributes")
  File.open("/etc/chef/client-config.json", "w") do |f|
    f.print(JSON.pretty_generate(chef_config["attributes"]))
  end
  json_attribs "/etc/chef/client-config.json"
end

validation_key "/etc/chef/validation.pem"
validation_client_name chef_config["validation_client_name"]

Mixlib::Log::Formatter.show_time = true

Finish creating the new image

Clean up from the building of the image

sudo chroot $imagedir umount /proc
sudo chroot $imagedir umount /dev/pts
sudo rm -f $imagedir/usr/sbin/policy-rc.d

Copy the image files to a new EBS volume, snapshot and register the snapshot

size=15 # root disk in GB
now=$(date +%Y%m%d-%H%M)
prefix=runa-chef-0.8.4-ubuntu-$release-$codename-$tag-$arch-$now
description="Runa Chef 0.8.4 Ubuntu $release $codename $tag $arch $now"
export EC2_CERT=$(echo /mnt/cert-*.pem)
export EC2_PRIVATE_KEY=$(echo /mnt/pk-*.pem)

volumeid=$(ec2-create-volume --region $region --size $size \
  --availability-zone $availability_zone | cut -f2)

instanceid=$(wget -qO- http://instance-data/latest/meta-data/instance-id)

ec2-attach-volume --region $region --device /dev/sdi --instance "$instanceid" "$volumeid"

while [ ! -e /dev/sdi ]; do echo -n .; sleep 1; done

sudo mkfs.ext3 -F /dev/sdi
ebsimage=$imagedir-ebs
sudo mkdir $ebsimage
sudo mount /dev/sdi $ebsimage

sudo tar -cSf - -C $imagedir . | sudo tar xvf - -C $ebsimage
sudo umount $ebsimage

ec2-detach-volume --region $region "$volumeid"
snapshotid=$(ec2-create-snapshot --region $region "$volumeid" | cut -f2)

ec2-delete-volume --region $region "$volumeid"

# This takes a while
while ec2-describe-snapshots --region $region "$snapshotid" | grep -q pending
  do echo -n .; sleep 1; done

ec2-register \
  --region $region \
  --architecture $arch \
  --name "$prefix" \
  --description "$description" \
  $ebsopts \
  --snapshot "$snapshotid"

Afterward

That will get you an AMI that you can now use as a chef-client. You can use the directions from the section Creating a Chef Server from your new Image in the previous article: Creating an Amazon EC2 AMI for Opscode Chef 0.8.

• 1/13/10: Fix various minor inaccuracies and improved description on how to set up the chef-server. Also removed nanite as a requirement (its no longer used)
• 1/17/10: Add the requirement to build and install mixlib-authentication for the chef-client
• 1/21/10: Added a mkdir for /var/log/chef
• 1/22/10: Added step to insure that /tmp permissions are set

Introduction

Here’s my experience setting up an Amazon EC2 AMI and Instance for a Chef Server and Client. It is based mostly on Bryan Mclellan (btm)‘s post of Nov 24, 2009 Installing Chef 0.8 alpha on Ubuntu Karmic and his more up to date GIST: chef 0.8 alpha installation. It has a slightly different focus and is a bit stale if you are building your own 0.8 gems from the source.

Instantiate an Amazon EC2 Instance

We’ll start with the Canonical Ubuntu 9.10 Karmic AMI. I always go to Eric Hammond’s site  alestic.com to get the pointers to the right AMIs. In this case we’re using a 32bit image for the US-West Region: ami-7d3c6d38 US-East 32bit: ami-1515f67c. You can use the US-West 64bit image ami-7b3c6d3e, US-East 64bit: ami-ab15f6c2

Start the instance from your local dev machine using the command line ec2-api-tools (available as a package or directly from Amazon) or using something like the Firefox Elasticfox and then ssh into the instance so that you can do the following steps on the instance. For the sake of this example, lets say that the Public DNS name for the instance you started is ec2-204-222-170-10.us-west-1.compute.amazonaws.com and the ssh keypair you associated with this new instance is now on your local dec machine in ~/.ssh/gsg-keypair

Prerequisite preparation

The first set of steps need to be done on the instance you just created so login via ssh:

ssh -i ~/.ssh/gsg-keypair ec2-204-222-170-10.us-west-1.compute.amazonaws.com

If on Amazon us-west

There is a bug in the current us-west Canonical AMI where it does not use the us-west apt server. So you have to correct the apt soruces.list:

sed -i.bak '1,$s/us.ec2.archive.ubuntu.com/us-west-1.ec2.archive.ubuntu.com/' \
/etc/apt/sources.list

For all cases

sudo sed -i.bak2 '1,$s/universe/universe multiverse/' /etc/apt/sources.list
sudo apt-get -y update
sudo apt-get -y upgrade
sudo apt-get -y install emacs23 # Of course this is the first package to install!

# Will need these to manipulate ec2 images
sudo apt-get -y install ec2-api-tools ec2-ami-tools 

Set up the ruby environment and install rubygems

Install Ruby and needed packages

sudo apt-get -y install -y ruby ruby1.8-dev libopenssl-ruby1.8 rdoc ri irb \
build-essential wget ssl-cert git-core rake librspec-ruby libxml-ruby \
thin couchdb zlib1g-dev libxml2-dev

Install Rubygems

Rubygems will be installed from source since debian/ubuntu try to control rubygems upgrades. If you don’t care you can install it via apt-get install rubygems

cd /tmp
wget http://rubyforge.org/frs/download.php/60718/rubygems-1.3.5.tgz
tar zxf rubygems-1.3.5.tgz
cd rubygems-1.3.5
sudo ruby setup.rb
sudo ln -sfv /usr/bin/gem1.8 /usr/bin/gem
sudo gem sources -a http://gems.opscode.com
sudo gem sources -a http://gemcutter.org

Install Pre-requisit Gems

sudo gem install cucumber merb-core jeweler uuidtools \
json libxml-ruby --no-ri --no-rdoc

Building and Installing Chef Related Gems

Until there are final 0.8.x Chef gems, you will have had to build them on your local machine and upload them to this instance. On your dev machine (this example builds things in ~/src, but it could be anywhere appropriate) follow these instructions to build all the gems and install gems you might need to use your local machine. You will use your local dev machine to develop and manage cookbooks and to manage a remote chef-server:

mkdir ~/src
cd ~/src
git clone git://github.com/opscode/chef.git
git clone git://github.com/opscode/ohai.git
git clone git://github.com/opscode/mixlib-log
git clone git://github.com/opscode/mixlib-authentication.git
# Need to get mixlib-log for client & server and
# mixlib-authentication for the client from git till the 1.1.0 update hits
# See http://tickets.opscode.com/browse/CHEF-823
cd mixlib-log
sudo rake install
cd mixlib-authentication
sudo rake install
cd ../ohai
sudo rake install
cd ../chef
rake gem
# Now cd into ~/src/chef/chef to install the chef client/dev gem on your local machine
cd chef
rake install 

Upload the gems needed for the client to your instance. From ~/src on your local dev machine do:

scp -i ~/.ssh/gsg-keypair chef/chef/pkg/chef-0.8.0.gem  ohai/pkg/ohai-0.3.7.gem \
mixlib-authentication/pkg/mixlib-authentication-1.1.0.gem \
mixlib-log/pkg/mixlib-log-1.1.0.gem  ec2-204-222-170-10.us-west-1.compute.amazonaws.com:

Set up the Chef Client on the new Instance

Now back in your home directory on the instance ec2-204-222-170-10.us-west-1.compute.amazonaws.com install the gems you just copied over:

sudo gem install mixlib-log-1.1.0.gem ohai-0.3.7.gem
sudo gem install chef-0.8.0.gem 

Create the client config file

mkdir /var/log/chef
mkdir /etc/chef
chown root:root /etc/chef
chmod 755 /etc/chef

Put the following in /etc/chef/client.rb:

# Chef Client Config File

require 'ohai'
require 'json'

o = Ohai::System.new
o.all_plugins
chef_config = JSON.parse(o[:ec2][:userdata])
if chef_config.kind_of?(Array)
  chef_config = chef_config[o[:ec2][:ami_launch_index]]
end

log_level        :info
log_location     "/var/log/chef/client.log"
chef_server_url  chef_config["chef_server"]
registration_url chef_config["chef_server"]
openid_url       chef_config["chef_server"]
template_url     chef_config["chef_server"]
remotefile_url   chef_config["chef_server"]
search_url       chef_config["chef_server"]
role_url         chef_config["chef_server"]
client_url       chef_config["chef_server"]

node_name        o[:ec2][:instance_id]

unless File.exists?("/etc/chef/client.pem")
  File.open("/etc/chef/validation.pem", "w") do |f|
    f.print(chef_config["validation_key"])
  end
end

if chef_config.has_key?("attributes")
  File.open("/etc/chef/client-config.json", "w") do |f|
    f.print(JSON.pretty_generate(chef_config["attributes"]))
  end
  json_attribs "/etc/chef/client-config.json"
end

validation_key "/etc/chef/validation.pem"
validation_client_name chef_config["validation_client_name"]

Mixlib::Log::Formatter.show_time = true

Set up the /etc/init.d/chef-client

Copy the example init.d script (You can also use runit instead, but we’re not going to describe that here)

cp /usr/lib/ruby/gems/1.8/gems/chef-0.8.0/distro/debian/etc/init.d/chef-client /etc/init.d
cd /etc/init.d
update-rc.d chef-client defaults

Create an Init script to set /tmp to proper permmissions

It looks like the Canonical Images will not have /tmp with proper permissions if you exclude /tmp from your bundle process. Eric Hammond recommends doing the following.

Create a file /etc/init.d/ec2-mkdir-tmp with the following contents:

#!/bin/sh
#
# ec2-mkdir-tmp Create /tmp if missing (as it's nice to bundle without it).
#
mkdir -p    /tmp
chmod 01777 /tmp

Then set up the /etc/rc dirs to launch this on boot:

chmod a+x /etc/init.d/ec2-mkdir-tmp
ln -s /etc/init.d/ec2-mkdir-tmp /etc/rcS.d/S36ec2-mkdir-tmp

Build the EC2 Image

The always amazingly helpful Eric Hammond has a post, Creating a New Image for EC2 by Rebundling a Running Instance, that describes the basics of how to do this. The following is pretty much a direct synopsis with minimal explanation. See his blog post for more details.

Clean up potential security holes

Remove stuff you don’t want to freeze into your image.

sudo rm -f /root/.*hist* $HOME/.*hist*
sudo rm -f /var/log/*.gz

Copy AWS Certs to Instance

Back on your local development system, copy your Amazon certificates to the instance.

remotehost=<ec2-instance-hostname>
remoteuser=ubuntu
scp -i <private-ssh-key> \
  <path-to-certs>/{cert,pk}-*.pem \
  $remoteuser@$remotehost:/tmp

Create the new Image on the Instance

Back on the ec2 instance, you’ll do the following to create the image.

Define where to store the image on S3

This assumes you have an S3 account setup on AWS. You don’t have to have already created the bucket. Set some bash variables that will be used by the commands that follow. You should set the prefix to something that is meaningful. Below is what I used as an example. You’ll want to make it unique to your environment. The Bucket name must be Globally unique across all of Amazon S3.

bucket=runa-west-amis
prefix=runa-ubuntu-9.10-i386-20100101-base

Define your AWS credentials and target processor

export AWS_USER_ID=<your-value>
export AWS_ACCESS_KEY_ID=<your-value>
export AWS_SECRET_ACCESS_KEY=<your-value>

if [ $(uname -m) = 'x86_64' ]; then
  arch=x86_64
else
  arch=i386
fi

Bundle the files
This also runs on the current instance and will bundle the everything on the instance file system except for dirs specified with the -e flag into a copy of the image under /mnt:

sudo -E ec2-bundle-vol           \
  -r $arch                       \
  -d /mnt                        \
  -p $prefix                     \
  -u $AWS_USER_ID                \
  -k /tmp/pk-*.pem               \
  -c /tmp/cert-*.pem             \
  -s 10240                       \
  -e /mnt,/tmp,/root/.ssh,/home/ubuntu/.ssh

If you are deploying to US-West-1 AWS Region

Looks like the Amazon ec2 ami tools are not super aware about us-west yet. So you have to do this extra step right now. You’ll have to change the –kernel and –ramdisk to the ones appropriate for your kernel. You can inspect the values used for the AMI you used to boot the original instance. You can do this with ElasticFox or with the command (specify the AMI and region its in thatyou want to check):

ec2-describe-images ami-7d3c6d38   -C /tmp/cert-*.pem -K /tmp/pk-*.pem --region us-west-1

Then execute the following command and specify the right kernel and ramdisk

sudo -E ec2-migrate-manifest        \
  -c /tmp/cert-*.pem             \
  -k /tmp/pk-*.pem               \
  -m /mnt/$prefix.manifest.xml   \
  --access-key $AWS_ACCESS_KEY_ID  \
  --secret-key $AWS_SECRET_ACCESS_KEY \
  --kernel aki-773c6d32          \
  --ramdisk ari-713c6d34         \
  --region us-west-1

Upload the bundle to a bucket on S3:

sudo -E ec2-upload-bundle        \
    -b $bucket                   \
    -m /mnt/$prefix.manifest.xml \
    -a $AWS_ACCESS_KEY_ID        \
    -s $AWS_SECRET_ACCESS_KEY    \
    --location us-west-1

You may be prompted with something like:

You are bundling in one region, but uploading to another. If the kernel or ramdisk associated with this AMI are not in the target region, AMI registration will fail.
You can use the ec2-migrate-manifest tool to update your manifest file with a kernel and ramdisk that exist in the target region.
Are you sure you want to continue? [y/N]

You should enter y return to accept.

Register the AMI

Back on your local development machine:

ec2-register $bucket/$prefix.manifest.xml --region us-west-1

The output of this will be the ami-id of your new instance. You can use this to instantiate your new ami.

You now have a private ami image you can start just like any other image. If you want to make it public

ec2-modify-image-attribute -l -a all 

Using the new AMI Image

You can now use this instance as the basis for chef clients and also the basis to create a Chef Server. Use the Amazon EC2 tool, ElasticFox or whatever you favorite tool for managing EC2 instances to make a new instance first to create a Chef Server. Then after that you can create clients and have them load their roles and recipes from the chef server. Once you have a Chef Server, you can use knife ec2 instance command to create user data that includes a run list, credentials and other json that can be passed to the general ec2 tools to build specific instances.

Creating a Chef Server from your new Image

Using an EC2 tool like ec2-tools or elasticfox, create a new instance based on the AMI created earlier. You should use at least a c1.medium as the m1.small is just too painfully wimpy to use. Assume the new instance has the Public DNS name: ec2-204-203-51-20.us-west-1.compute.amazonaws.com
Copy the chef server gems to the new instance from the ~/src directory in your local dev environment to the new instance:

scp -i ~/.ssh/gsg-keypair chef/*/pkg/*.gem \
ec2-204-203-51-20.us-west-1.compute.amazonaws.com:

ssh to the new instance and do the following:

sudo gem install chef-server-0.8.0.gem chef-server-api-0.8.0.gem \
chef-server-webui-0.8.0.gem chef-solr-0.8.0.gem

Set things up to use bootstrap client using chef-solo

We’ll be using the last part of BTM’s GIST, and danielsdeleo (Dan DeLeo)’s bootstrap cookbook and chef-solo to set up this initial server.

mkdir -p /tmp/chef-solo
cd /tmp/chef-solo
git clone git://github.com/danielsdeleo/cookbooks.git
cd cookbooks
git checkout 08boot

Create ~/chef.json:

{
  "bootstrap": {
    "chef": {
      "url_type": "http",
      "init_style": "runit",
      "path": "/srv/chef",
      "serve_path": "/srv/chef",
      "server_fqdn": "localhost"
    }
  },
  "recipes": "bootstrap::server"
}
# End of file

Create ~/solo.rb with the following content:

file_cache_path "/tmp/chef-solo"
cookbook_path "/tmp/chef-solo/cookbooks"
# End of ~/solo.rb file

Run chef-solo which will execute the chef bootstrap recipes using the bootstrap params in ~/chef.json to actually setup and configure this chef server

If you had installed rubygems with the ubuntu apt package you may have to specify the path:

/var/lib/gems/1.8/bin/

instead of:

/usr/bin

for the knife and various chef commands in the following code.

/usr/bin/chef-solo -j ~/chef.json -c ~/solo.rb -l debug

You will see a lot of Debug statements go by and it will take several minutes to complete. It should complete with something like:

[Thu, 14 Jan 2010 00:19:38 +0000] INFO: Chef Run complete in 38.59808 seconds
[Thu, 14 Jan 2010 00:19:38 +0000] DEBUG: Exiting

Setup basic cookbooks

The following will install the standard cookbooks on the chef server

cd
git clone git://github.com/opscode/chef-repo.git
cd chef-repo
rm cookbooks/README
git clone git://github.com/opscode/cookbooks.git

Now upload the standard cookbooks using the credentials set up by the bootstrap process (user chef-webui)

knife cookbook upload --all -u chef-webui \
-k /etc/chef/webui.pem -o cookbooks

Startup the Chef Server web ui

Do to a bug (http://tickets.opscode.com/browse/CHEF-839) you have to run this twice, the first time will create the admin user:

sudo /usr/bin/chef-server-webui -p 4002

But the first time will abort with an error message like:

Loading init file from /usr/lib/ruby/gems/1.8/gems/chef-server-0.8.0/config/init-webui.rb
Loading /usr/lib/ruby/gems/1.8/gems/chef-server-0.8.0/config/environments/development.rb
~ Loaded slice 'ChefServerWebui' ...
WARN: HTTP Request Returned 404 Not Found: Cannot load user admin
~ Compiling routes...
~ Could not find resource model Node
~ Could not find resource model Client
~ Could not find resource model Role
~ Could not find resource model Search
~ Could not find resource model Cookbook
~ Could not find resource model Client
~ Could not find resource model Databag
~ Could not find resource model DatabagItem
/usr/lib/ruby/gems/1.8/gems/chef-server-0.8.0/config/init-webui.rb:32: uninitialized constant OpenID (NameError)
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/bootloader.rb:1258:in `call'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/bootloader.rb:1258:in `run'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/bootloader.rb:1258:in `each'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/bootloader.rb:1258:in `run'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/bootloader.rb:99:in `run'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/server.rb:172:in `bootup'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core/server.rb:42:in `start'
from /usr/lib/ruby/gems/1.8/gems/merb-core-1.0.15/lib/merb-core.rb:173:in `start'
from /usr/lib/ruby/gems/1.8/gems/chef-server-0.8.0/bin/chef-server-webui:76
from /usr/bin/chef-server-webui:19:in `load'
from /usr/bin/chef-server-webui:19

Then again to actually start the WebUI and have it run in the background. You might want to start it in screen for now or possibly redirect its output to a log file The following example shows sending the output of the command to a log file. You’ll want to check that log file after starting to make sure there were no errors.

sudo sh -c '/usr/bin/chef-server-webui -p 4002 > /var/log/chef-server-webui.log' &

If you look at the output of a ps, you’ll see the shell command above, but the real work is being done by a merb instance with the port you specified (4002):

#ps ax | grep webui
5533 pts/0    S      0:00 sh -c /usr/bin/chef-server-webui -p 4002 > /var/log/chef-server-webui.log
#ps ax | grep merb
3694 ?        Sl     0:55 merb : worker (port 4000)
5534 pts/0    Sl     0:07 merb : worker (port 4002)

The first merb worker is the chef-server itself, the second is the WebUI server.

Accessing the Chef Web UI

You can access the Chef Web UI web server using a web browser at the IP address / Public DNS name of this server that was just set up. Assuming the Public DNS is

ec2-204-203-51-20.us-west-1.compute.amazonaws.com

Assuming that you set up this instance to allow you to access port 4002 from the IP adddress of your local dev machine, you should be able to access the Web UI at

http://ec2-204-203-51-20.us-west-1.compute.amazonaws.com:4002

You can allow access to port 4002 from specific ip address ranges by updating your security group. You can do that with ElasticFox (easy) or via the command line tools (a pain for a one off). Eventually you (or hopefully Opscode) will set up an apache or nginx reverse proxy, Passenger or equiv to allow normal port 80 / 443 http/https access.

Conclusion

You should now be able to use knife your local dev environment to develop cookbooks and upload roles and cookbooks to your new Chef Server and spin up new chef cookbook driven instances. You should use the knife documentation from the Opscode main wiki Knife Page NOT the docs in the Alpha Forums / Getting Started With Opscode / Knife – Commandline API as the later is actually more obsolete in terms of the version that you built from the opscode git repository. There is also a man page and knife –help gives you pretty much the same correct info as the wiki.

I hope to have a follow up post on how to do this in more details.

Feel free to leave comments if you find problems or have questions.

I am having problems using the chef dev tools/client from the HEAD of the chef git repository with the Opscode Alpha Server service. I’m not sure if its me or if the latest versions of the chef client from HEAD is compatible with the Alpha Server Service. So the following is still useful for understanding how to build from HEAD, but it will not work with the Opscode Alpha SaaS server. It will work with the server you build from HEAD. See the next article Creating an Amazon EC2 AMI for Opscode Chef 0.8 for info on creating a Chef client and server on EC2.

Introduction

Opscode is introducing a pretty major set of changes in Chef in the 0.8 release. Its a major step forward and has some major changes as to how one interacts with Chef. (as well as some major bug fixes that alone make it worth the move). The Opscode Alpha Program introduces a new service where Opscode runs the actual Chef Server as a service.

This post will describe setting up your User/Dev environment by building your own Chef Client / Dev Gems from the latest HEAD of the Chef repo from Github. It assumes that you did sign up for the Alpha program and have access to the Opscode Alpha Server. Though much of it would be the same if you were running your own chef server also built from the latest source from github. This post does not show how to actually use Chef and the chef-client on a target node. Hope to have a post on that in the next few days.

The documentation on how to move to and use Chef 0.8 is still very sparse, so I figured I would jot down some of the things we are learning as we apply this to our infrastructure at Runa. If any of you OpsChefs out there see something wrong or something I left out, let me know in the comments or via email.

The Opscode Chef Alpha Environment

If you are in the Opscode Alpha program, you would have been given login[s] and some pem keys. I won’t go into the details of this since they do have pretty good docs on setting this up (if you have an alpha login you can get them at http://opscode.zendesk.com/forums/58858/entries/49336). Its probably a good idea to follow these and start with their 0.8.0 gem to make sure you are talking with the Alpha Server before trying to use the Chef Git Repository to build your own gems.

The Alpha instructions use a Chef gem that is frozen at 0.8.0. But the Chef folks have already progressed much further than the Oct 29h release of 0.8.0 in the Chef Git Repository.

The HEAD of the Git Repository has many changes since 0.8.0. Some big ones include:

• The Knife sub commands are completely different
• There is now a Chef Shell (A REPL like irb but for the chef client)
• Lots of Bug Fixes

And if we’re going to be on the bleeding edge, we might as well go all the way! So the rest of this blog will be about using the Chef HEAD branch from the Chef git repository. We’ll still use the Alpha Chef Server at least to start with.

Configuring your Dev Environment

Prerequisites

I’m using Mac OS X 10.6 (snow leopard). Our target environments are Ubuntu Linux on Amazon EC2. But assuming you have *nix, Ruby and Ruby Gems set up on your environment it should generally be the same (don’t know about people stuck in the Legacy Windows environment though).

So you will need to have installed and know how to use:

• Ruby
• RubyGems
• Git

And the following Ruby Gems should be installed (I think this is the minimum you need, these will include their own dependencies:

• rake
• rspec
• cucumber
• uuidtools
• nanite
• gemcutter
• jeweler

You will need http://gems.opscode.com as a gem source for the following. You can use the command:

sudo gem sources -a http://gems.opscode.com

• mixlib-authentication

Getting and building the code/GEMs for the Dev Environment

The instructions that are in the README.doc of the Chef Git Repository are out of date as of now (Dec 20, 2009). The instructions on the wiki, Installing Chef from HEAD are more accurate. Even though it seems like one can use the mixlib gems as the repository and the gems have the same version number, I found that I needed to install the mixlib libraries from source.

Getting and building Ohai & Mixlib Gems from Github

We won’t be making any changes in these, so we’ll just git clone and build it:

cd to where you want to keep your local repositories
git clone git://github.com/opscode/ohai.git
cd ohai
sudo rake install
cd ..
git clone git://github.com/opscode/mixlib-config.git
sudo rake install
cd ..
git clone git://github.com/opscode/mixlib-log.git
sudo rake install
cd ..
git clone git://github.com/opscode/mixlib-cli.git
sudo rake install
cd ..

Getting the Chef code from github

You can get the Chef repository from github. The readme there has most of the info you need for

If you plan to submit any patches or other changes back to Opscode, or you would like to have your own repository of this, you can fork the Opscode repository into your own Github account. This is what I did and will demonstrate below. If you don’t want any hardcore forking action, you can just git clone the opscode repository as shown here (assuming your current working directory is where you want the local directory repository placed. It will be named using the default “chef”):

git clone git://github.com/opscode/chef.git

If you have forked into your own github account (mine is rberger), you would git clone using the “Your Clone URL”:

git clone git@github.com:rberger/chef.git rberger-chef

This assumes you want your local directory name for the repository to be rberger-chef, just so you can distinguish it from the official opscode one. (I will refer to the top of the local repository as rberger-chef from now on).

What’s in the Chef Git Repository

Change directory into the local repository and do an ls. You’ll see that there are several components here.

$ cd rberger-chef
$ ls
CHANGELOG         README.rdoc       chef-server       chef-solr         scripts
LICENSE           Rakefile          chef-server-api   cucumber.yml
NOTICE            chef              chef-server-webui features

There are 2 main trees:

• chef: chef-client and dev gem
• chef-server: Chef Server gem. Used only if you build your own server
  • chef-server-api: Implements the REST interface sub-system as part of the full Chef Server
  • chef-server-webui: Implements the WebUI as part of the full Chef Server
  • chef-solar: Implements the Solar Search sub-system as part of the full Chef Server
  • features: Not 100% sure all its used for, definately for the cucumber tests. But is part of the Server as far as I can tell

For now we are only interested in the chef tree. That will be used to set up the local dev environment. We’re not going to follow the outdated instructions that are in the README.doc in the root of the chef repository which assumes you are setting up the whole stack on the Dev machine. We’re going to just install the chef client and tools from the chef sub-tree on the dev machine.

This post will not describe how to build /use the chef-server, though you can pretty much build everything by running

sudo rake install

from the top of the distro. There are more gem dependencies that need to be installed before you can build the chef-server trees.

Building and Installing the Chef Client / Dev tools

Change directory to the chef subdirectory so you should be in rberger-chef/chef (or if you have a direct clone of the opscode chef repository: chef/chef)

cd chef

Some minor tweaks to the Source

(shef is now included in the executables in the latest repository and setting my own sub-version number was lame)

I have done a few mods to the source. Mainly to set the version number to something that will not conflict with the official numbering now or when new releases come out and to have shef be installed by the gem.

1. Changed line 30 in the Rakefile to s.executables = %w( chef-client chef-solo knife shef ) so the install puts shef in /usr/bin
2. Changed line 7 in the Rakefile to CHEF_VERSION = "0.8.0.1"
3. Change line 30 in lib/chef.rb to VERSION = '0.8.0.1'

Build and install

rake install

Its going to eventually ask for your sudo password as it needs to use sudo to do the gem install. The run should look something like:

(in /Users/rberger/work/Chef/rberger-chef/chef)
mkdir -p pkg
WARNING:  no rubyforge_project specified
WARNING:  description and summary are identical
  Successfully built RubyGem
  Name: chef
  Version: 0.8.0.1
  File: chef-0.8.0.1.gem
mv chef-0.8.0.1.gem pkg/chef-0.8.0.1.gem
sudo gem install pkg/chef-0.8.0.1 --no-rdoc --no-ri
Password:
Building native extensions.  This could take a while...
Successfully installed eventmachine-0.12.10
Successfully installed amqp-0.6.5
Successfully installed thor-0.12.0
Successfully installed deep_merge-0.1.0
Successfully installed moneta-0.6.0
Successfully installed chef-0.8.0.1
6 gems installed

Using Chef with the Opscode Alpha SaaS Server

This just touches on some of the things that are described in The Official Guide to Getting Started With Opscode

Setting up your Dev Environment

Its not clear if you really have to do everything as described in the document if you are building the latest release from the chef repository and using the ~/.chef/knife.rb config described below. For instance I didn’t have to set the environment variables for OPSCODE_USER and OPSCODE_KEY since they are now set in the knife.rb nor did I have to create /etc/chef/client.rb. And even without the global Chef config, I was able to use most of the knife commands. But not some like the ec2 instances data seemed to need the organization validation key to be in /etc/chef/validation.pem

Copy your assigned validation key to /etc/chef

When you got your Opscode Alpha welcome stuff, you should have gotten your user keys and a key for your organization. Copy your organization (in our case runa.pem) to /etc/chef/validation.pem. You will probably have to create /etc/chef directory first.

The User Chef/Knife config

You must configure a knife config file in your home directory under ~/.chef/knife.rb and have your key that you got from Opscode somewhere pointed to by a line in ~/.chef/knife.rb. The configuration parameters are described on the Knife Wiki Page. For instance my config file:

log_level        :info
log_location     STDOUT
node_name        'rberger'
client_key       '/Users/rberger/.chef/rberger.pem'
chef_server_url  "https://api.opscode.com/organizations/runa"
cache_type       'BasicFile'
cache_options( :path => '/Users/rberger/.chef/checksums' )

Once you have this set up you can now use knife and the chef rake commands. You can test things out by saying something like:

knife client list

Which should return and empty list assuming you haven’t set up any clients on this server yet.

The first real useful command you want to do is to upload your cookbooks to the Opscode Server:

cd to where your chef cookbook repository is
rake upload_cookbooks

You can also do it with just knife:

knife cookbook upload -a

This may take a while as it will upload all the cookbooks in cookbooks and site-cookbooks in your current repository.

After that you can upload single cookbooks

knife cookbook upload

Just remember the knife documentation on the Alpha site no longer applies to the knife that you get from building from the HEAD of the chef git repository. Strangely enough, the knife documentation on the wiki is accurate.

Conclusion

Once you’ve been thru it, its all quite simple. I hope to post some more on using 0.8.0+ soon. See a more recent blog post for building your own Chef Server Creating an Amazon EC2 AMI for Opscode Chef 0.8