Infrastructure API and Provisioning Resources

Work Completed By: Tim West

DESCRIPTION

Three areas discussed within cloud computing are provisioning of resources, the ability to scale those resources in line with demand and the availability of the resource. Previously the infrastructure API research focused on how the cloud can provide elastic resources, that is scaling both up and down, in line with demand.

This project continues investigation and aims to demonstrate how infrastructure resources can be provisioned when an event, or trigger, occurs. Focusing again on the use of and application programming interface (API) to manage infrastructure resources. Events can be manual or electronic and the resources provisioned can include computing, storage and networking.

DETAILS/LOCATION OF PRACTICAL DEMONSTRABLE OUTPUT

To demonstrate the use of the API based on events or triggers, I wrote two simple shell scripts to create and destroy instances with Amazons EC2 Cloud environments.

Note: These scripts are very basic, with little or no error checking. They are provided to demonstrate the use of an API and should not be used in any form of production environment.

Time Based Trigger
The scenario for a time based trigger was:

• If the minute ends in either a 0 or a 5 create a new instance of an ami,
• If the minute ends in a 3, destroy the oldest running instance,
• If the minute is either 0 or 30, destroy all the running instances.

The shell script follows:

while true
do
        MINUTE=`date +%M`
        REMAINDER=`expr ${MINUTE} % 10`
        echo ${REMAINDER}

        case ${REMAINDER} in
                0|5)
                        # its the start of a five minute window start a new instance
                        echo Creating an instance ...
                        ec2-run-instances ami-2e5fba47 -k wcc-test-key-3 > start-instance.log

                        # find the instance ID and log it
                        tail -1 start-instance.log | awk ' { print $2 } ' >> running-instances.log
                        ;;
                3)
                        # stop the longest running instance
                        INSTANCE=`head -1 running-instances.log`
                        echo Terminating an instance ...
                        ec2-terminate-instances ${INSTANCE}
                        grep -v ${INSTANCE} running-instances.log > running-instance.log.new
                        mv running-instance.log.new running-instances.log
                        ;;
                *)
                        # Nothing happening this minute
                        echo Nothing to do. Running instances ...
                        cat running-instances.log
                        ;;
        esac

        if [ ${MINUTE} -eq 0 -o ${MINUTE} -eq 30 ]
        then
                cat running-instances.log | while read INSTANCE
                do
                        echo Shutting down ...
                        ec2-terminate-instances ${INSTANCE}
                        grep -v ${INSTANCE} running-instances.log >> running-instance.log.new
                        mv running-instance.log.new running-instances.log
                done
                exit
        fi
        sleep 60
done

Manual Trigger
The scenario for a manual trigger was a simple menu with the follow options:

• List all the running instances,
• Create a new instance from a list of available AMIs,
• Stop one of the running instances

The shell script follows:

while true
do
        clear
        /bin/echo
        /bin/echo "  Amazon EC2 Instance Control Menu"
        /bin/echo "  ================================"
        /bin/echo
        /bin/echo "  1. Display all running Instances"
        /bin/echo
        /bin/echo "  2. Create a new Instance"
        /bin/echo
        /bin/echo "  3. Destroy an Instance"
        /bin/echo
        /bin/echo "  4. Exit"
        /bin/echo
        /bin/echo -n "OPTION: "
        read OPTION

        case ${OPTION} in
                1)
                        # Display all running Instances
                        clear
                        /bin/echo "The following Instances are running:"
                        /bin/echo -e "Instance\tStatus/Public DNS Entry\t\t\t\tDescription"
                        ec2-describe-instances | grep INSTANCE | grep -v terminated | awk '{ print $2, $3, $4 }' > menutest-display.log
                        cat menutest-display.log | while read INSTANCE AMI STATUS
                        do
                                DESCRIPTION=`ec2-describe-images ${AMI} | awk ' { print $3 }' `
                                /bin/echo -e "${INSTANCE}\t${STATUS}\t${DESCRIPTION}"
                        done
                        rm -f menutest-display.log
                        /bin/echo
                        /bin/echo Press Enter to return to the menu.
                        read OPTION
                        ;;
                2)
                        # Create a new Instance
                        clear
                        /bin/echo Select which AMI to use:
                        /bin/echo -e "AMI\t\tDescription"
                        for AMI in ami-2e5fba47 ami-de4daab7
                        do
                                DESCRIPTION=`ec2-describe-images ${AMI} | awk ' { print $3 }' `
                                /bin/echo -e "${AMI}\t${DESCRIPTION}"
                        done
                        /bin/echo
                        /bin/echo -n "Enter name of AMI to create or ENTER for none: "
                        read AMI
                        if [ "x${AMI}" != "x" ]
                        then
                                /bin/echo -n "Creating a new Instance ... "
                                ec2-run-instances ${AMI} -k wcc-test-key-3 > menutest-start.log
                                INSTANCE=`tail -1 menutest-start.log | awk '{ print $2 }'`
                                /bin/echo ${INSTANCE} created.
                                /bin/echo
                                /bin/echo Press Enter to return to the menu.
                                read OPTION
                        fi
                        ;;
                3)
                        # Destroy an Instance
                        clear
                        /bin/echo The following Instances are running:
                        /bin/echo -e "Instance\tStatus/Public DNS Entry\t\t\t\tDescription"
                        ec2-describe-instances | grep INSTANCE | grep -v terminated | awk '{ print $2, $3, $4 }' > menutest-display.log
                        cat menutest-display.log | while read INSTANCE AMI STATUS
                        do
                                DESCRIPTION=`ec2-describe-images ${AMI} | awk ' { print $3 }' `
                                /bin/echo -e "${INSTANCE}\t${STATUS}\t${DESCRIPTION}"
                        done
                        rm -f menutest-display.log
                        /bin/echo
                        /bin/echo -n "Please enter the Instance to destroy or ENTER for none: "
                        read INSTANCE
                        if [ "x${INSTANCE}" != "x" ]
                        then
                                /bin/echo
                                /bin/echo -n "Destroying instance - ${INSTANCE} ... "
                                ec2-terminate-instances ${INSTANCE} > /dev/null
                                /bin/echo destroyed.
                                /bin/echo
                                /bin/echo Press Enter to return to the menu.
                                read OPTION
                        fi
                        ;;
                4)
                        # Exit
                        exit
                        ;;
                *)
                        # User has entered an invalid option.
                        ;;
        esac
done

PROJECT OUTCOME

Time Based Trigger
Running the above script, starting at 10 minutes past the hour generated the following output:

$ ./timetest
0
Creating an instance ...
1
Nothing to do. Running instances ...
i-e123e989
2
Nothing to do. Running instances ...
i-e123e989
3
Terminating an instance ...
INSTANCE        i-e123e989      running shutting-down
4
Nothing to do. Running instances ...
5
Creating an instance ...
7
Nothing to do. Running instances ...
i-c720eaaf
8
Nothing to do. Running instances ...
i-c720eaaf
9
Nothing to do. Running instances ...
i-c720eaaf
0
Creating an instance ...
1
Nothing to do. Running instances ...
i-c720eaaf
i-2121eb49
2
Nothing to do. Running instances ...
i-c720eaaf
i-2121eb49
3
Terminating an instance ...
INSTANCE        i-c720eaaf      running shutting-down
4
Nothing to do. Running instances ...
i-2121eb49
5
Creating an instance ...
6
Nothing to do. Running instances ...
i-2121eb49
i-5126ec39
7
Nothing to do. Running instances ...
i-2121eb49
i-5126ec39
8
Nothing to do. Running instances ...
i-2121eb49
i-5126ec39
9
Nothing to do. Running instances ...
i-2121eb49
i-5126ec39
0
Creating an instance ...
Shutting down ...
INSTANCE        i-2121eb49      running shutting-down
Shutting down ...
INSTANCE        i-5126ec39      running shutting-down
Shutting down ...
INSTANCE        i-4324ee2b      pending shutting-down

This is also confirmed by the output from the AWS Console. Ignore the first two Instances listed, they are for a different project.

Manual Trigger

$ ./menutest

  Amazon EC2 Instance Control Menu
  ================================

  1. Display all running Instances

  2. Create a new Instance

  3. Destroy an Instance

  4. Exit

OPTION: 1
The following Instances are running:
Instance        Status/Public DNS Entry                         Description
i-e6a45e8e      ec2-72-44-60-248.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml
i-e2f3098a      ec2-75-101-239-3.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml

Press Enter to return to the menu.

  Amazon EC2 Instance Control Menu
  ================================

  1. Display all running Instances

  2. Create a new Instance

  3. Destroy an Instance

  4. Exit

OPTION: 2
Select which AMI to use:
AMI             Description
ami-2e5fba47    ec2-public-images/fedora-core4-apache-v1.07.manifest.xml
ami-de4daab7    ec2-public-windows-images/Server2003r2-i386-Win-v1.06.manifest.xml

Enter name of AMI to create or ENTER for none: ami-2e5fba47
Creating a new Instance ... i-f937fd91 created.

Press Enter to return to the menu.

  Amazon EC2 Instance Control Menu
  ================================

  1. Display all running Instances

  2. Create a new Instance

  3. Destroy an Instance

  4. Exit

OPTION: 1
The following Instances are running:
Instance        Status/Public DNS Entry                         Description
i-e6a45e8e      ec2-72-44-60-248.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml
i-e2f3098a      ec2-75-101-239-3.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml
i-f937fd91      ec2-75-101-216-16.compute-1.amazonaws.com       ec2-public-images/fedora-core4-apache-v1.07.manifest.xml

Press Enter to return to the menu.

  Amazon EC2 Instance Control Menu
  ================================

  1. Display all running Instances

  2. Create a new Instance

  3. Destroy an Instance

  4. Exit

OPTION: 3
The following Instances are running:
Instance        Status/Public DNS Entry                         Description
i-e6a45e8e      ec2-72-44-60-248.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml
i-e2f3098a      ec2-75-101-239-3.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml
i-f937fd91      ec2-75-101-216-16.compute-1.amazonaws.com       ec2-public-images/fedora-core4-apache-v1.07.manifest.xml

Please enter the Instance to destroy or ENTER for none: i-f937fd91

Destroying instance - i-f937fd91 ... destroyed.

Press Enter to return to the menu.

  Amazon EC2 Instance Control Menu
  ================================

  1. Display all running Instances

  2. Create a new Instance

  3. Destroy an Instance

  4. Exit

OPTION: 1
The following Instances are running:
Instance        Status/Public DNS Entry                         Description
i-e6a45e8e      ec2-72-44-60-248.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml
i-e2f3098a      ec2-75-101-239-3.compute-1.amazonaws.com        /aws-console-quickstart-amis/ruby/1.1/rubyquickstart.manifest.xml

Press Enter to return to the menu.

  Amazon EC2 Instance Control Menu
  ================================

  1. Display all running Instances

  2. Create a new Instance

  3. Destroy an Instance

  4. Exit

OPTION: 4

SHORT TERM BENEFITS

As we currently don't use any Cloud services, we currently have no use of these APIs.

STRATEGIC IMPLICATIONS

It was relatively easy to write the two scripts above to provide the basic functionally required by the scenarios. The choice of using shell scripts to write the applications provided a quick prototyping environment. Although the applications worked, they were slow when trying to lookup information about each of the running Instances. If anything more advanced needed to be written then another language more suited to the web should be used.

OTHER NOTES ETC