Usage of AWS CLI with Eucalyptus

Follow

Using AWS CLI with Eucalyptus

Versions of Eucalyptus:  4.0.0 and Greater

Background

AWS CLI is the all-services-in-one CLI services API tool developed and maintained by Amazon Web Services team and community.  Since Eucalyptus implements services that you can interact through the AWS service API, the AWS CLI can be used with Eucalyptus as well.  This knowledge base article will provide information on how this can be accomplished.

Before we begin, understand that the services accessible through the AWS CLI when used against Eucalyptus can only be used against the services that Eucalyptus has implemented.  At the time of this KB update, the following AWS service APIs have been implemented on Eucalyptus:

  • Eucalyptus 4.0.x
    • Compute (EC2)
    • Object Storage Gateway (S3)
    • Cloudwatch
    • Token Service (STS)
    • Load Balancing (ELB)
    • Autoscaling
    • Euare (IAM)
    • Cloudformation (tech preview in Eucalyptus 4.0.0)
  • Eucalyptus 4.1.x
    • Compute (EC2)
    • Object Storage Gateway (S3)
    • Cloudwatch
    • Token Service (STS)
    • Load Balancing (ELB)
    • Autoscaling
    • Euare (IAM)
    • Cloudformation
    • VPC (tech preview)
    • SWF (tech preview)

The services marked as tech preview may not work with all the API calls performed by the AWS CLI tools.  All the service APIs that are not tagged with tech preview should work. If there are any issues, please let us know.

Setup 

Install AWS CLI

To get started, use the instructions to install the AWS CLI on the desired platform.  For example, on a CentOS 6.6 machine, the following commands were executed:

# sudo yum install -y python-pip
# sudo pip install awscli

After the installation has been successful, confirm that everything has been installed correctly by running the following command:

# aws help

The help manual for AWS CLI should present itself in the terminal window.  Once the test is successful, we are ready to set up the configuration file.

Configure AWS CLI

When Eucalyptus users receive their credentials from the cloud administrator, typical they receive a zip file, that contains a eucarc file.  This file will look similar to this:

EUCA_KEY_DIR=$(cd $(dirname ${BASH_SOURCE:-$0}); pwd -P)
export EC2_URL=http://compute.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export S3_URL=http://objectstorage.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export AWS_IAM_URL=http://euare.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export EUARE_URL=http://euare.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export TOKEN_URL=http://tokens.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export AWS_AUTO_SCALING_URL=http://autoscaling.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export AWS_CLOUDFORMATION_URL=http://cloudformation.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export AWS_CLOUDWATCH_URL=http://cloudwatch.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export AWS_ELB_URL=http://loadbalancing.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export AWS_SIMPLEWORKFLOW_URL=http://simpleworkflow.h-12.autoqa.qa1.eucalyptus-systems.com:8773/
export EUSTORE_URL=http://emis.eucalyptus.com/
export EC2_PRIVATE_KEY=${EUCA_KEY_DIR}/euca2-admin-4dcfc665-pk.pem
export EC2_CERT=${EUCA_KEY_DIR}/euca2-admin-4dcfc665-cert.pem
export EC2_JVM_ARGS=-Djavax.net.ssl.trustStore=${EUCA_KEY_DIR}/jssecacerts
export EUCALYPTUS_CERT=${EUCA_KEY_DIR}/cloud-cert.pem
export EC2_ACCOUNT_NUMBER='439629811643'
export EC2_ACCESS_KEY='AKIE8HHW2D7D2HVJPQKQ'
export EC2_SECRET_KEY='Sw5b8Lx0UsGqgjqAkltXcdIXtdKdiFEOC8uSRA5w'
export AWS_ACCESS_KEY='AKIE8HHW2D7D2HVJPQKQ'
export AWS_SECRET_KEY='Sw5b8Lx0UsGqgjqAkltXcdIXtdKdiFEOC8uSRA5w'
export AWS_CREDENTIAL_FILE=${EUCA_KEY_DIR}/iamrc
export EC2_USER_ID='439629811643'
alias ec2-bundle-image="ec2-bundle-image --cert ${EC2_CERT} --privatekey ${EC2_PRIVATE_KEY} --user ${EC2_ACCOUNT_NUMBER} --ec2cert ${EUCALYPTUS_CERT}"
alias ec2-upload-bundle="ec2-upload-bundle -a ${EC2_ACCESS_KEY} -s ${EC2_SECRET_KEY} --url ${S3_URL}"

The key variables needed for the AWS CLI configuration file are as follows:

  • AWS_ACCESS_KEY
  • AWS_SECRET_KEY

As mentioned in the AWS CLI documentation regarding setting up credentials file, create the directory .aws, then create the config file.  For example:

# mkdir .aws
# touch config

Once the config file is created, use your favorite editor (e.g. vi/vim) to add the following contents to the file - using the values for AWS_ACCESS_KEY and AWS_SECRET_KEY where appropriate.  In this example, we will use the variables displayed previous in the contents of the eucarc file:

[profile default]
aws_access_key_id = AKIE8HHW2D7D2HVJPQKQ
aws_secret_access_key = Sw5b8Lx0UsGqgjqAkltXcdIXtdKdiFEOC8uSRA5w
region = us-east-1
output = text

Thats it, we have completed the configuring AWS CLI.  Now, a few examples to show how to use the AWS CLI with Eucalyptus.

Examples

The key thing to remember when using the AWS CLI tools against Eucalyptus is the --endpoint-url option.  This option is needed to tell the given service which Eucalyptus Service API endpoint to use.  For these examples, we will use the URL service variables mentioned in the eucarc file discussed earlier.  

To describe EC2 security groups using AWS CLI, run the following command:

# aws --endpoint-url http://compute.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ ec2 describe-security-groups
SECURITYGROUPS default group sg-1dab229a default 439629811643
IPPERMISSIONS -1 icmp -1
IPRANGES 0.0.0.0/0
IPPERMISSIONS 22 tcp 22
IPRANGES 0.0.0.0/0

To list OSG (S3) buckets using the AWS CLI, do the following:

# aws --endpoint-url http://objectstorage.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ s3 ls
2015-01-23 07:03:07 elbautoscalenginx-bucketstack-mcbjidl0sa-s3bucket-kzk0iss16c3x6
2015-01-20 15:08:13 ubuntu-trusty-20150120

To list Load Balancers (ELBs):

# aws --endpoint-url http://loadbalancing.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ elb describe-load-balancers
LOADBALANCERDESCRIPTIONS 2015-01-24T03:57:55.413Z ELBAutoS-ElasticLo-EAQYCU62XLKFQ-439629811643.lb.h-12.autoqa.qa1.eucalyptus-systems.com ELBAutoS-ElasticLo-EAQYCU62XLKFQ internet-facing
AVAILABILITYZONES two
HEALTHCHECK 3 30 HTTP:80/ 5 5
LISTENER 80 80 HTTP
SOURCESECURITYGROUP euca-internal-439629811643-ELBAutoS-ElasticLo-EAQYCU62XLKFQ 453485424274

To list AutoScaling groups:

# aws --endpoint-url http://autoscaling.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ autoscaling describe-auto-scaling-groups
AUTOSCALINGGROUPS arn:aws:autoscaling::439629811643:autoScalingGroup:bf93d60c-266c-4ac6-8833-63c783b59488:autoScalingGroupName/ELBAutoScaleNginx-WebServerGroup-EK3ERPM194NEC ELBAutoScaleNginx-WebServerGroup-EK3ERPM194NEC 2015-01-24T03:58:13.674Z 300 2 3600 EC2 ELBAutoScaleNginx-LaunchConfig-UWRRUNWIHFY9K 4 2
AVAILABILITYZONES two
INSTANCES two Healthy i-5be345b0 ELBAutoScaleNginx-LaunchConfig-UWRRUNWIHFY9K InService
INSTANCES two Healthy i-024a2f55 ELBAutoScaleNginx-LaunchConfig-UWRRUNWIHFY9K InService
LOADBALANCERNAMES ELBAutoS-ElasticLo-EAQYCU62XLKFQ
TAGS Name True ELBAutoScaleNginx-WebServerGroup-EK3ERPM194NEC auto-scaling-group ELBAutoScaleNginx
TAGS aws:cloudformation:logical-id True ELBAutoScaleNginx-WebServerGroup-EK3ERPM194NEC auto-scaling-group WebServerGroup
TAGS aws:cloudformation:stack-id True ELBAutoScaleNginx-WebServerGroup-EK3ERPM194NEC auto-scaling-group arn:aws:cloudformation::439629811643:stack/ELBAutoScaleNginx/4e092dc7-1bcc-41e7-9c87-07ce0226db41
TAGS aws:cloudformation:stack-name True ELBAutoScaleNginx-WebServerGroup-EK3ERPM194NEC auto-scaling-group ELBAutoScaleNginx
TERMINATIONPOLICIES Default

To list Cloudformation stacks:

# aws --endpoint-url http://cloudformation.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ cloudformation describe-stacks
STACKS 2015-01-24T03:57:51.779Z AWS CloudFormation Sample Template AutoScalingSingleAZ: Create a single-az, load balanced and Auto Scaled sample web site running on an Nginx Web Server. The application is configured to span in one availability zone (one cluster) and is Auto-Scaled based on the CPU utilization of the web servers. False 2015-01-24T03:58:24.001Z arn:aws:cloudformation::439629811643:stack/ELBAutoScaleNginx/4e092dc7-1bcc-41e7-9c87-07ce0226db41 ELBAutoScaleNginx CREATE_COMPLETE Complete!
CAPABILITIES CAPABILITY_IAM
OUTPUTS URL http://ELBAutoS-ElasticLo-EAQYCU62XLKFQ-439629811643.lb.h-12.autoqa.qa1.eucalyptus-systems.com
OUTPUTS BucketName elbautoscalenginx-s3bucket-cely40rmyafxw
OUTPUTS StackRef arn:aws:cloudformation::439629811643:stack/ELBAutoScaleNginx-BucketStack-YCECUI0HNPKSN/48dbceb0-e4d2-4ae9-9a3c-655b1566ad05
OUTPUTS OutputFromNestedStack elbautoscalenginx-bucketstack-ycecui0hnp-s3bucket-tqbrqk5olar9o
PARAMETERS InstanceType m1.large
PARAMETERS UbuntuImageId emi-38abe905
PARAMETERS UserKeyPair admin-hspencer1
PARAMETERS MinSize 2
PARAMETERS MaxSize 4
PARAMETERS Zone two
STACKS 2015-01-24T03:57:55.927Z Cloudformation Example => Create bucket with public-read access control list (ACL) True 2015-01-24T03:58:04.169Z arn:aws:cloudformation::439629811643:stack/ELBAutoScaleNginx-BucketStack-YCECUI0HNPKSN/48dbceb0-e4d2-4ae9-9a3c-655b1566ad05 ELBAutoScaleNginx-BucketStack-YCECUI0HNPKSN CREATE_COMPLETE Complete!
CAPABILITIES CAPABILITY_IAM
OUTPUTS BucketName elbautoscalenginx-bucketstack-ycecui0hnp-s3bucket-tqbrqk5olar9o

Creation and deletion actions work as well.  For example, here is how you would create and delete an OSG bucket:

# aws --endpoint-url http://objectstorage.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ s3 mb s3://test-bucket
make_bucket: s3://test-bucket/
# aws --endpoint-url http://objectstorage.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ s3 ls
2015-01-23 07:03:07 elbautoscalenginx-bucketstack-mcbjidl0sa-s3bucket-kzk0iss16c3x6
2015-01-24 03:57:59 elbautoscalenginx-bucketstack-ycecui0hnp-s3bucket-tqbrqk5olar9o
2015-01-24 03:57:55 elbautoscalenginx-s3bucket-cely40rmyafxw
2015-01-24 04:08:18 test-bucket
2015-01-20 15:08:13 ubuntu-trusty-20150120
# aws --endpoint-url http://objectstorage.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ s3 rb s3://test-bucket
remove_bucket: s3://test-bucket/
# aws --endpoint-url http://objectstorage.h-12.autoqa.qa1.eucalyptus-systems.com:8773/ s3 ls
2015-01-23 07:03:07 elbautoscalenginx-bucketstack-mcbjidl0sa-s3bucket-kzk0iss16c3x6
2015-01-24 03:57:59 elbautoscalenginx-bucketstack-ycecui0hnp-s3bucket-tqbrqk5olar9o
2015-01-24 03:57:55 elbautoscalenginx-s3bucket-cely40rmyafxw
2015-01-20 15:08:13 ubuntu-trusty-20150120

As you can see, AWS CLI works really well with Eucalyptus 4.0/4.1.  Please remember that if a user is not authorized to do a certain service action through an IAM policy, then they will receive a 'not authorized' return message.  For additional information regarding AWS CLI, check out the following documentation resources from AWS:

 Note:  The AWS CLI version used in this article was version 1.7.0.  Currently, AWS CLI version 1.7.5 has an issue with Eucalyptus 4.0.x and greater.  The issue is around creating buckets using the S3 service API call.  This is due to following bug found in AWS CLI - https://github.com/aws/aws-cli/issues/1142.  This issue looks to be resolved in AWS CLI version 1.7.12 (https://github.com/aws/aws-cli/commit/b8b75b85819ad1a9fda74a7c284ee288c25aa7c7) 

Have more questions? Submit a request

Comments

Powered by Zendesk