Tutorial: Hello Ansible on iMinds Virtual Wall

In this tutorial, you will learn how you can use the open source configuration managment tool “Ansible” to ease the development, scaling and reproducability of your next experiment.

1. Design the experiment

In this exercise we will convert the install script below into an Ansible playbook.

This install script does the following:

• Install apache2 and iperf

On the server:

• Configure the default site on Apache

• Copy some files to /var/www

• Start the apache service

On the client:

• Start a wget client script

• Start an iperf client script

#!/bin/bash
#----------------------------------------------------------------------
# Copyright (c) 2012 Raytheon BBN Technologies
#
# Permission is hereby granted, free of charge, to any person obtaining
# a copy of this software and/or hardware specification (the "Work") to
# deal in the Work without restriction, including without limitation the
# rights to use, copy, modify, merge, publish, distribute, sublicense,
# and/or sell copies of the Work, and to permit persons to whom the Work
# is furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be
# included in all copies or substantial portions of the Work.
#
# THE WORK IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS
# OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
# MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
# NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
# HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
# WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE WORK OR THE USE OR OTHER DEALINGS
# IN THE WORK.
#----------------------------------------------------------------------

# Usual directory for downloading software in ProtoGENI hosts is `/local`
cd /local
#Get the hostname to see if we are at the server or the client
hn=`echo $HOSTNAME | cut -d'.' -f 1`

##### Check if file is there #####
if [ ! -f "./installed.txt" ]
then
       #### Create the file ####
        sh -i -c `sudo touch "./installed.txt"`

       #### Run  one-time commands ####

       #Install necessary packages
        sudo apt-get update
        sudo apt-get -y install apache2 iperf & EPID=$!
        wait $EPID

       # Install custom software
       ## Customize apache installation


       ## Reboot the host if needed
fi
##### Run Boot-time commands
# Start services
# If this is the server then start the iperf and configure and start the http server
if [ $hn == "server" ]
then

   # Enable web server stats
   sudo /usr/sbin/a2enmod status
   sudo rm /etc/apache2/mods-enabled/status.conf

   echo "<Location /server-status>" | sudo tee -a /etc/apache2/sites-available/default > /dev/null
   echo "   SetHandler server-status" |sudo tee -a /etc/apache2/sites-available/default > /dev/null
   echo "   Allow from all" | sudo tee -a /etc/apache2/sites-available/default > /dev/null
   echo "</Location>" | sudo tee -a /etc/apache2/sites-available/default > /dev/null
   echo "ExtendedStatus On" | sudo tee -a /etc/apache2/conf.d/extendedstatus > /dev/null

   # Copy the website under /var/www/
   sudo cp -R ./website/* /var/www/
   sudo rm -rf /var/www/html
   sudo ln -s /var/www/ /var/www/html

   # Start the webserver
   sudo /usr/sbin/apache2ctl restart
   sudo service apache2 restart

   # Start the iperf server
   sudo mkdir -p /var/www/iperflogs
   iperf_server_log="/var/www/iperflogs/iperf-server.log"
   sudo bash -c "iperf -s -i 10 &> $iperf_server_log"
else
  # If this is the client start the script for transfers
  # Wait 60 seconds just to give some time to the server to come up
  sleep 60
  ./scripts/client-wget.sh&
  ./scripts/client-iperf.sh&
fi
# Start my service -- assume it was installed at /usr/local/bin

2. Establish the Environment

1. To run this exercise, you will need an account and two pieces of software.

1. Have a Fed4FIRE or GENI account

2. jFed Experimenter GUI BETA: use the Quickstart Experimenter GUI BETA-button on <http://jfed.iminds.be/>

3. Have Ansible installed:

   1. On Linux and Mac: find the instructions for your package manager here

   2. On Windows: Ansible has no support for Windows on the control machine. You can swap in a Linux VM to overcome this issue (Use ansible_controller.rspec for a readily configured machine).

2. Download the webpages and scripts needed for the exercise.

Use wget to download the tarball of files onto your local machine and use tar to uncompress it:

wget http://doc.ilabt.iminds.be/jfed-documentation/_downloads/helloansible-install.tar.gz
tar zxvf helloansible-install.tar.gz

Note

Windows users should download and unpack this on their Ansible controller testbed-node.

3. Obtain resources

Reserve helloansible.rspec in jFed in an experiment named ansblXX where XX is your initials.

4. Configure and Initialize

jFed can export the login information of your nodes into several formats for configuration management systems. An Ansible inventory is one of the supported formats. This is a flat file format which tells Ansible the name and login information for your nodes.

To export these files:

• Go to the “RSpec Viewer” tab on the top of your screen

• Choose “Save Experiment” -> “Export Configuration Management Settings”

• Save the ZIP-file in a convenient place

• Unpack the ZIP-file in your working directory

Note

Windows users should copy and unpack their zip-file onto their ansible controller. You can use WinSCP to copy the resources to your testbed node.

The exported ZIP-file contains several files:

• id_rsa

• id_rsa.pub

• ssh-config

• ansible-hosts

• ansible.cfg

• fabfile.py

• README.txt

The README.txt-file contains an comprehensive introduction for each of these files. In this tutorial, we will be using the files mentioned in bold.

An example of the content of these files can be found below:

ansible-hosts:

# Sample commands:
# ansible nodes -m ping
# ansible nodes -m shell -a "uptime"

[nodes]
client        ansible_ssh_host=n091-vm26-10.wall2.ilabt.iminds.be     ansible_ssh_port=22     ansible_ssh_user=twalcari
server        ansible_ssh_host=n091-vm26-11.wall2.ilabt.iminds.be     ansible_ssh_port=22     ansible_ssh_user=twalcari

ansible.cfg:

[defaults]
private_key_file = ./id_rsa
host_key_checking = false
inventory = ansible-hosts

Warning

Before continuing, you need to properly secure the private key id_rsa. Otherwise, all commands will fail. To do this, execute:

$ chmod 600 id_rsa

Check to see if your nodes are up and ready. This command uses the ping module to ping all the servers from the nodes-group (in which jFed stores all servers by default) listed in the inventory file:

$ ansible nodes -m ping

Example output showing all of the nodes responding to ping:

$ ansible -i ansible-hosts nodes -m ping
client | success >> {
    "changed": false,
    "ping": "pong"
}

server | success >> {
    "changed": false,
    "ping": "pong"
}

Try using the ping module in Ansible to only ping server or client by replacing all in the above with server or client.

5. Using Ansible Ad Hoc

The following are some example Ansible Ad Hoc commands. You can run these commands one at a time from the machine where you have Ansible installed. -s tells Ansible to use sudo when executing the command. All modules are described in the Ansible Documentation.

apt module is used to install packages using the apt package manager:

ansible [nodes/server/client] -s -m apt -a "name=apache2 update_cache=yes"

command module is used to execute an arbitrary command on the remote node:

ansible [nodes/server/client]  -s -m command -a "/usr/sbin/a2enmod status"

file module is used to set attributes of files:

ansible [nodes/server/client] -s -m file -a "path=/var/www/html state=absent"

synchronize module is an implementation of rsync and is used to efficiently synchronize files between your local machine and a remote node:

ansible [nodes/server/client] -s  -m synchronize \
    -a "src=website/index.html dest=/var/www"

lineinfile module is used to see if an arbitrary line exists in a file:

ansible [nodes/server/client] -s  -m lineinfile \
   -a "line='ExtendedStatus On' dest=/etc/apache2/conf.d/extendedstatus create=yes state=present"

service module is used to start/stop/restart/etc services:

ansible [nodes/server/client] -s  -m service -a "name=apache2 state=restarted"

Use it!

Using the above Ad Hoc Ansible commands configure the client node by doing the following:

• on both client and server, install apache2 and iperf

• copy the scripts directory into /local on the client node with permissions 755.

Verify that the above items have been done, by logging into the client node and running:

$ apache2 -v
$ iperf -v
$ ls /local/scripts

6. Using Ansible Playbooks

Ansible commands can be collected into files called Playbooks. Playbooks are in a configuration file format called YAML which is very straightforward. In particular, Ansible Ad Hoc commands easily map to commands used in an Ansible Playbook.

Use the name construct to leave a comment describing the Ansible command you are running.

The Playbook to configure the client node as in the install script is as follows:

---
- name: Configure client
  hosts: client
  sudo: True
  tasks:
   - name: install apache2
     apt: name=apache2 update_cache=yes
   - name: install iperf
     apt: name=iperf update_cache=yes
   - name: copy scripts into /local
     synchronize: src=scripts dest=/local
   - name: set permission of /local to 755
     file: 
       path: /local 
       mode: 0755
   - name: start client wget
     command: /local/scripts/client-wget.sh 
     async: 86400
     poll: 0
   - name: start client iperf
     command: /local/scripts/client-iperf.sh
     async: 86400
     poll: 0

Do these commands look like the Ad Hoc commands you came up with in the previous step?

Put the above content in a file called hello-client.yml.

Run the playbook with the following command on the local machine:

ansible-playbook -i ansible-hosts hello-client.yml

Use it!

Using the above Ansible modules, configure the server node by reproducing the steps in the HelloGENI install script for the server node. As you find a command that works, use it to construct a hello-server.yml playbook. (See the solution or hello-server.yml)

Here is a template you can fill in to create the hello-server.yml-file:

- name: Configure server
  hosts: server
  sudo: True
  vars: 
    iperf_server_log: /var/www/iperflogs/iperf-server.log
  tasks:
   - name: install apache2
     apt: name=apache2 update_cache=yes
   - name: install iperf
     apt: name=iperf update_cache=yes
   - name: /usr/sbin/a2enmod status
     # INSERT COMMAND HERE
   - name: check /etc/apache2/mods-enabled/status.conf file is absent
     # INSERT COMMAND HERE
   - name: copy website/index.html into /var/www
     # INSERT COMMAND HERE
   - name: copy the website/graphics directory into /var/www
     # INSERT COMMAND HERE
   - name: rm -rf /var/www/html
     # INSERT COMMAND HERE
   - name: Make simlink to webfiles
     # INSERT COMMAND HERE
   - name: Make sure /etc/apache2/conf.d/extendedstatus file contains "ExtendedStatus On"
     # INSERT COMMAND HERE
   - name: Make sure /etc/apache2/sites-available/000-default.conf contains Location information
     # INSERT COMMAND HERE
   - name: restart apache2 service
     # INSERT COMMAND HERE
   - name: create directory for iperf logs in /var/www/iperflogs with permissions of 755
     # INSERT COMMAND HERE
   - name: start server
     command: /usr/bin/sudo /bin/bash -c "iperf -s -i 10 &> {{ iperf_server_log }}" 
     async: 86400
     poll: 0

• Run both the client and server playbooks on your nodes.

• Browse to the server node

7. Advanced Ansible

• You may have noticed that the server playbook and the client playbook contained duplicate commands. This is because both nodes need to have apache installed so that they can act as webservers. As you might have guessed, there is a way to modularize this configuration. In Ansible this is done with “roles”.

• Explore some of the intermediate features of Ansible such as roles, variables, and handlers.

8. Teardown Experiment

• Reload the OS of your client and server, and test your playbooks from scratch in the new experiment. Do you get the same behavior?

• When you are done, delete the resources in your experiment.

9. Archive Experiment

If this was a real experiment, you would commit your Ansible playbook to a version control system like git as you go.

Acknowledgements

This tutorial is an adaptation of the GENI-tutorial “Converting the Hello GENI Install Script to Ansible”