Updates
- Mar 3, 2010 Added call to script ec2-set-defaults that is normally called on ec2 init that sets the locale and apt sources for EC availability Zone
Introduction
Opscode has officially released 0.8.x of Chef. It is now even more fabulous. I’ve been using the pre-release version for the last couple of months and it is rock steady and very powerful. I’ll be having a post soon on how I used it to deploy a pretty complicated cloud stack with multiple Rails/Mysql/Nginx/Unicorn/Postfix apps for front-ends, and a back end made up of a mix of a Clojure/Swarmiji distributed processing swarm, HBase/Hadoop, Redis, RabbitMQ.
But first, I needed to upgrade my Amazon EC2 AMIs for the officially released Chef 0.8.x. I also wanted to try the EBS Boot image as a basis for the AMI.
This is an update to my earlier post, Creating an Amazon EC2 AMI for Opscode Chef 0.8, but now using the official Opscode 0.8.x Gems instead of building your own Gems. A lot of the content is the same, but you can consider this mostly superceding the older one except where mentioned otherwise. This version will use the EBS Boot AMIs as per Eric Hammond’s Tutorial Building EBS Boot AMIs Using Canonical’s Downloadable EC2 Images. Much of this is blog post is taken from Eric’s blog post but in the context of creating a Chef Client base AMI and a Chef Server. Note that Opscode now has their own AMIs, including ones for Chef 0.8.4, but as of this writing, they do not have AMIs for Amazon us-west.
Setup
Prerequisites
On your host development machine (ie your laptop or whatever machine you are developing from) you should have already installed:
- ec2-api-tools and ec2-ami-tools (these assume you have a modern Java run time setup)
- chef-0.8.4 or later chef client gem (which implies the entire ruby 1.8.x and rubygems toolchain)
Set some Shell variables on host machine
Just to make using these instructions as a cookbook, we’ll have some shell variables that you can set once and then all the instructions will use the variables so you can just cut and paste the instructions into your shell.
keypair=id_runa-staging-us-west
fullpath_keypair=~/.ssh/runa/id_runa-staging-us-west
availability_zone=us-west-1a
instance_type=m1.large
region=us-west-1
# Pick one of these two AMIs (Note that it will be different for different Amazon Regions)
# 32bit AMI
origin_ami=ami-fd5100b8
#64bit AMI
origin_ami=ami-ff5100ba
Start up an instance and capture the instanceid
instanceid=$(ec2-run-instances \
--key $keypair \
--availability-zone $availability_zone \
--instance-type $instance_type \
$origin_ami \
--region $region |
egrep ^INSTANCE | cut -f2)
echo "instanceid=$instanceid"
Wait for the instance to move to the “running” state
while host=$(ec2-describe-instances --region $region "$instanceid" |
egrep ^INSTANCE | cut -f4) && test -z $host; do echo -n .; sleep 1; done
echo host=$host
This should loop till you see something like:
$ echo host=$host
host=ec2-184-72-2-93.us-west-1.compute.amazonaws.com
Upload your certs
This assumes that your Amazon certs are in ~/.ec2
rsync \
--rsh="ssh -i $fullpath_keypair" \
--rsync-path="sudo rsync" \
~/.ec2/{cert,pk}-*.pem \
ubuntu@$host:/mnt/
Connect to the instance
ssh -i $fullpath_keypair ubuntu@$host
Update the Amazon ec2 tools on the instance
export DEBIAN_FRONTEND=noninteractive
echo "deb http://ppa.launchpad.net/ubuntu-on-ec2/ec2-tools/ubuntu karmic main" |
sudo tee /etc/apt/sources.list.d/ubuntu-on-ec2-ec2-tools.list &&
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9EE6D873 &&
sudo apt-get update &&
sudo -E apt-get dist-upgrade -y &&
sudo -E apt-get install -y ec2-api-tools
Set some parameters on instance shell environment
Again this makes it easier to cut and paste the instructions.
codename=karmic
release=9.10
tag=server
region=us-west-1
availability_zone=us-west-1a
if [ $(uname -m) = 'x86_64' ]; then
arch=x86_64
arch2=amd64
# You will need to set the aki and ari values base on the actual base AMI you used
# It will be different for different regions. These are set for us-west-1
ebsopts="--kernel=aki-7f3c6d3a --ramdisk=ari-cf2e7f8a"
ebsopts="$ebsopts --block-device-mapping /dev/sdb=ephemeral0"
else
arch=i386
arch2=i386
# You will need to set the aki and ari values base on the actual base AMI you used
# It will be different for different regions. These are set for us-west-1
ebsopts="--kernel=aki-773c6d32 --ramdisk=ari-c12e7f84"
ebsopts="$ebsopts --block-device-mapping /dev/sda2=ephemeral0"
fi
Download and unpack the latest released Ubuntu server image file
This contains the output of vmbuilder as run by Canonical.
imagesource=http://uec-images.ubuntu.com/releases/$codename/release/unpacked/ubuntu-$release-$tag-uec-$arch2.img.tar.gz
image=/mnt/$codename-$tag-uec-$arch2.img
imagedir=/mnt/$codename-$tag-uec-$arch2
wget -O- $imagesource |
sudo tar xzf - -C /mnt
sudo mkdir -p $imagedir
sudo mount -o loop $image $imagedir
Bring the packages on the instance up to date
# Allow network access from chroot environment
sudo cp /etc/resolv.conf $imagedir/etc/
# Fix what I consider to be a bug in vmbuilder
sudo rm -f $imagedir/etc/hostname
# Add multiverse
sudo perl -pi -e 's%(universe)$%$1 multiverse%' \
$imagedir/etc/ec2-init/templates/sources.list.tmpl
# Add Alestic PPA for runurl package (handy in user-data scripts)
echo "deb http://ppa.launchpad.net/alestic/ppa/ubuntu karmic main" |
sudo tee $imagedir/etc/apt/sources.list.d/alestic-ppa.list
sudo chroot $imagedir \
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys BE09C571
# Add ubuntu-on-ec2/ec2-tools PPA for updated ec2-ami-tools
echo "deb http://ppa.launchpad.net/ubuntu-on-ec2/ec2-tools/ubuntu karmic main" |
sudo tee $imagedir/etc/apt/sources.list.d/ubuntu-on-ec2-ec2-tools.list
sudo chroot $imagedir \
apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 9EE6D873
# Upgrade the system and install packages
sudo chroot $imagedir mount -t proc none /proc
sudo chroot $imagedir mount -t devpts none /dev/pts
cat <<EOF > /tmp/policy-rc.d
#!/bin/sh
exit 101
EOF
sudo mv /tmp/policy-rc.d $imagedir/usr/sbin/policy-rc.d
chmod 755 $imagedir/usr/sbin/policy-rc.d
DEBIAN_FRONTEND=noninteractive
# It seems this has to be done to set up the Locale & apt sources
sudo -E chroot $imagedir /usr/bin/ec2-set-defaults
# Update the apt sources and packages
sudo chroot $imagedir apt-get update &&
sudo -E chroot $imagedir apt-get dist-upgrade -y &&
sudo -E chroot $imagedir apt-get install -y runurl ec2-ami-tools
Install Chef Client and other customizations
Install Ruby and needed packages
sudo -E chroot $imagedir apt-get -y install ruby ruby1.8-dev libopenssl-ruby1.8 rdoc ri irb \
build-essential wget ssl-cert git-core rake librspec-ruby libxml-ruby \
thin couchdb zlib1g-dev libxml2-dev emacs23-nox
Install Rubygems
Rubygems will be installed from source since debian/ubuntu try to control rubygems upgrades. If you don’t care you can install it via apt-get install rubygems
cd $imagedir/tmp
wget http://rubyforge.org/frs/download.php/69365/rubygems-1.3.6.tgz
tar zxf rubygems-1.3.6.tgz
cd rubygems-1.3.6
sudo -E chroot $imagedir ruby /tmp/rubygems-1.3.6/setup.rb
cd ..
sudo rm -rf rubygems-1.3.6
sudo -E chroot $imagedir ln -sfv /usr/bin/gem1.8 /usr/bin/gem
sudo -E chroot $imagedir gem sources -a http://gems.opscode.com
sudo -E chroot $imagedir gem sources -a http://gemcutter.org
sudo -E chroot $imagedir gem install chef
Use Opscode Chef Solo Bootstrap to configure the Chef Client
The following will set up all the default paths and directories as well as install and configure runit to start and monitor the chef-client. Originally I shied away from runit, but this time I’m going as Opscode Vanilla as possible and they like runit.
Create the solo.rb file
All of the following files should be done in $imagedir as we are going to have to run this as chroot to $imagedir
Create $imagedir/solo.rb with an editor and put in the following:
file_cache_path "/tmp/chef-solo"
cookbook_path "/tmp/chef-solo/cookbooks"
recipe_url "http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz"
Create the chef.json file
Create $imagedir/chef.json with the following. (set the server_fqdn to the chef server you are using):
{
"bootstrap": {
"chef": {
"url_type": "http",
"init_style": "runit",
"path": "/srv/chef",
"serve_path": "/srv/chef",
"server_fqdn": "chef-server-staging.runa.com"
}
},
"run_list": [ "recipe[bootstrap::client]" ]
}
Run the chef-solo command
sudo -E chroot $imagedir chef-solo -c solo.rb -j chef.json \
-r http://s3.amazonaws.com/chef-solo/bootstrap-latest.tar.gz
I had to run it 3 times before it completed with no errors.
After it does work, clean up the chef-solo stuff:
sudo rm $imagedir/{solo.rb,chef.json}
Update the client config file
The Chef Solo Client bootstrap process creates an /etc/chef/client.rb that is not ideal for Amazon EC2. The following will replace that:
mkdir -p /etc/chef
chown root:root /etc/chef
chmod 755 /etc/chef
Put the following in /etc/chef/client.rb:
# Chef Client Config File
# Automatically grabs configuration from ohai ec2 metadata.
require 'ohai'
require 'json'
o = Ohai::System.new
o.all_plugins
chef_config = JSON.parse(o[:ec2][:userdata])
if chef_config.kind_of?(Array)
chef_config = chef_config[o[:ec2][:ami_launch_index]]
end
log_level :info
log_location STDOUT
node_name o[:ec2][:instance_id]
chef_server_url chef_config["chef_server"]
unless File.exists?("/etc/chef/client.pem")
File.open("/etc/chef/validation.pem", "w", 0600) do |f|
f.print(chef_config["validation_key"])
end
end
if chef_config.has_key?("attributes")
File.open("/etc/chef/client-config.json", "w") do |f|
f.print(JSON.pretty_generate(chef_config["attributes"]))
end
json_attribs "/etc/chef/client-config.json"
end
validation_key "/etc/chef/validation.pem"
validation_client_name chef_config["validation_client_name"]
Mixlib::Log::Formatter.show_time = true
Finish creating the new image
Clean up from the building of the image
sudo chroot $imagedir umount /proc
sudo chroot $imagedir umount /dev/pts
sudo rm -f $imagedir/usr/sbin/policy-rc.d
Copy the image files to a new EBS volume, snapshot and register the snapshot
size=15 # root disk in GB
now=$(date +%Y%m%d-%H%M)
prefix=runa-chef-0.8.4-ubuntu-$release-$codename-$tag-$arch-$now
description="Runa Chef 0.8.4 Ubuntu $release $codename $tag $arch $now"
export EC2_CERT=$(echo /mnt/cert-*.pem)
export EC2_PRIVATE_KEY=$(echo /mnt/pk-*.pem)
volumeid=$(ec2-create-volume --region $region --size $size \
--availability-zone $availability_zone | cut -f2)
instanceid=$(wget -qO- http://instance-data/latest/meta-data/instance-id)
ec2-attach-volume --region $region --device /dev/sdi --instance "$instanceid" "$volumeid"
while [ ! -e /dev/sdi ]; do echo -n .; sleep 1; done
sudo mkfs.ext3 -F /dev/sdi
ebsimage=$imagedir-ebs
sudo mkdir $ebsimage
sudo mount /dev/sdi $ebsimage
sudo tar -cSf - -C $imagedir . | sudo tar xvf - -C $ebsimage
sudo umount $ebsimage
ec2-detach-volume --region $region "$volumeid"
snapshotid=$(ec2-create-snapshot --region $region "$volumeid" | cut -f2)
ec2-delete-volume --region $region "$volumeid"
# This takes a while
while ec2-describe-snapshots --region $region "$snapshotid" | grep -q pending
do echo -n .; sleep 1; done
ec2-register \
--region $region \
--architecture $arch \
--name "$prefix" \
--description "$description" \
$ebsopts \
--snapshot "$snapshotid"
Afterward
That will get you an AMI that you can now use as a chef-client. You can use the directions from the section Creating a Chef Server from your new Image in the previous article: Creating an Amazon EC2 AMI for Opscode Chef 0.8.
