Management of Eucalyptus Elastic Load Balancers

Follow

Eucalyptus Versions: 3.3.0 and Greater

Managing Eucalyptus Elastic Load Balancers

In a related KB article [1], we discussed how to Eucalyptus implemented Elastic Load Balancers [2].  This article will cover how to manage them as a cloud administrator. 

In certain situations, the cloud administrator will need to remove a cloud user's ELB.  Since Eucalyptus deploys each ELB as an AutoScaling group [3], the cloud administrator will need to delete the Autoscaling group instead of interacting with the ELB. 

Displaying ELBs

To display the ELBs on a given Eucalyptus cloud, use euscale-describe-auto-scaling-groups:

$ euscale-describe-auto-scaling-groups --region admin@

AUTO-SCALING-GROUP asg-euca-internal-elb-961915002812-Account11User51LB
lc-euca-internal-elb-961915002812-Account11User51LB-b6020253 LayinDaSmackDown 1 1 1 Default
INSTANCE i-E2AE423B LayinDaSmackDown InService Healthy lc-euca-internal-elb-961915002812-Account11User51LB-b6020253

If eulb-describe-lbs verbose is executed as the cloud administrator, the ELB will be seen, but only from the ELB service perspective:

$ eulb-describe-lbs verbose --region admin@

LOAD_BALANCER Account11User51LB Account11User51LB-961915002812.lb.euca-hasp.eucalyptus-systems.com 2014-02-16T01:16:56.898Z

Removing an ELB

To remove the ELB, use euscale-update-auto-scaling-group to set the following properties to 0 for the AutoScaling group:

  • Maximum group size (-M | --max-size)
  • Minimum group size (-m | --min-size)
  • Number of running instances the group should contain (--desired-capacity)

Below is an example of removing the load balancer using the example above:

$ euscale-update-auto-scaling-group --max-size 0 --min-size 0 --desired-capacity 0 asg-euca-internal-elb-961915002812-Account11User51LB --region admin@

Use euscale-describe-auto-scaling-groups to check the status of the AutoScaling group:

$ euscale-describe-auto-scaling-groups --region admin@

AUTO-SCALING-GROUP asg-euca-internal-elb-961915002812-Account11User51LB lc-euca-internal-elb-961915002812-Account11User51LB-b6020253 LayinDaSmackDown 0 0 0 Default

To check the status of the instance associated with the AutoScaling group, use euca-describe-instances, passing the 'verbose' keyword:

$ euca-describe-instances verbose --region admin@ 

RESERVATION r-F897408B 441445882805
INSTANCE i-E2AE423B emi-B41335F3 euca-172-17-120-240.eucalyptus.euca-hasp.eucalyptus-systems.com euca-172-17-120-240.eucalyptus.internal terminated 0 m1.small 2014-02-16T01:17:06.947Z LayinDaSmackDown eki-71583A4F eri-DD0D37F3 monitoring-enabled 172.17.120.240 172.17.120.240 instance-store paravirtualized 17a782a1-4c37-4319-bcab-b0e5efcd3372_LayinDaSmackDown_1 arn:aws:iam::441445882805:instance-profile/internal/loadbalancer/loadbalancer-vm

To delete the ELB, interact with the ELB service by using eulb-delete-lb:

$ eulb-describe-lbs verbose --region admin@

LOAD_BALANCER Account11User51LB Account11User51LB-961915002812.lb.euca-hasp.eucalyptus-systems.com 2014-02-16T01:16:56.898Z
$ eulb-delete-lb Account11User51LB-961915002812.lb.euca-hasp.eucalyptus-systems.com --region admin@

After deleting the ELB, the associated AutoScaling group will be removed as well:

$ euscale-describe-auto-scaling-groups asg-euca-internal-elb-961915002812-Account11User51LB --region admin@
$

The ELB has been successfully deleted. 

Troubleshooting

To help cloud administrators troubleshoot instance(s) responsible for providing the Elastic Load Balancing, the following cloud property can be set to the name of the SSH private key pair that allows cloud administrators to SSH into the instance(s):

loadbalancing.loadbalancer_vm_keyname

To set this property correctly, do the following:

  1. Create a keypair using euca-create-keypair as the cloud administrator:
    • euca-create-keypair euca-admin-elb > euca-admin-elb.priv
    • chmod 0600 euca-admin-elb.priv
  2. Set the cloud property 'loadbalancing.loadbalancer_vm_keyname' to the keypair name:
    • euca-modify-property -p loadbalancing.loadbalancer_vm_keyname=euca-admin-elb

This keypair will be used when any load balancer is created.  If cloud users report issues with their load balancer(s), the cloud administrator has the ability to SSH into the load balancer(s) and troubleshoot them accordingly.

Troubleshooting Tips

Typically, when a cloud user reports issues with ELB, the common problem is that the instances that have been registered with the load balancer not reporting to be 'InService' when eulb-describe-instance-health is executed.  For example,

eulb-describe-instance-health MyLoadBalancer --instances i-ac3e214b,i-8a823227 -I AKIOQS8WTUDCCYHAKBYQ -S vgT3kgPZHluMIgU4qyatUQsC7PY1geIhnUDIg4sY
INSTANCE     i-ac3e214b    OutOfService
INSTANCE     i-8a823227    OutOfService

There are a few places that can be checked to figure out what may be causing the issue:

  • Check NTP - SSH into the ELB instance, and confirm that NTP is synced
# euca-describe-instances
RESERVATION r-55ce44e7 266767588119 euca-internal-283851778068-BaseLoadBalancer
INSTANCE i-61dc752b emi-30a085a7 10.104.6.233 172.18.0.101 running euca-admin-elb 0 m1.small 2014-04-20T17:33:01.587Z GIRL monitoring-enabled 10.104.6.233 172.18.0.101 instance-store hvm 7e773ccd-5f19-4d56-ad6c-9153881021f9_GIRL_1 sg-1fb7bcb9 arn:aws:iam::266767588119:instance-profile/internal/loadbalancer/loadbalancer-vm-283851778068-BaseLoadBalancer
TAG instance i-61dc752b Name loadbalancer-resources
TAG instance i-61dc752b aws:autoscaling:groupName asg-euca-internal-elb-283851778068-BaseLoadBalancer
TAG instance i-61dc752b euca:node 10.104.1.188
root@odc-d-31 ~]# ssh -i euca-admin-elb.priv root@10.104.6.233                   [root@ip-172-18-0-101 ~]#  # ntpdate -u pool.ntp.org
22 Apr 00:39:00 ntpdate[1204]: adjust time server 96.44.142.5 offset -0.002369 sec
  • Check /var/log/load-balancer-servo/servo.log file - SSH into the ELB instance and review the messages in the /var/log/load-balancer-servo/servo.log file:
[root@odc-d-31 ~]# ssh -i euca-admin-elb.priv root@10.104.6.233                   [root@ip-172-18-0-101 ~]#  tail -16 /var/log/load-balancer-servo/servo.log
failed to find the loadbalancers
2014-04-20 19:21:06,227 servo [WARNING]:failed to find the loadbalancers
new health check config: healthcheck-config: interval=30, healthy_threshold=2, unhealthy_threshold=2, timeout=3, target=HTTP:80/
2014-04-20 19:21:22,232 servo [INFO]:new health check config: healthcheck-config: interval=30, healthy_threshold=2, unhealthy_threshold=2, timeout=3, target=HTTP:80/
starting to check i-8a823227
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
2014-04-20 19:21:22,233 servo [INFO]:starting to check i-8a823227
[Boto]
starting to check i-ac3e214b
2014-04-20 19:21:22,233 servo [INFO]:starting to check i-ac3e214b
failed to report the cloudwatch metrics: [Errno -2] Name or service not known
2014-04-20 19:21:31,361 servo [ERROR]:failed to report the cloudwatch metrics: [Errno -2] Name or service not known
  • Run health check from the Load Balancer - SSH into the load balancer, and manually run the health check configured (using curl/wget) to check the instances that are registered with the load balancer:
[root@odc-d-31 ~]# eulb-describe-instance-health BaseLoadBalancer --instances i-ac3e214b,i-8a823227 -I AKIOQS8WTUDCCYHAKBYQ -S vgT3kgPZHluMIgU4qyatUQsC7PY1geIhnUDIg4sY
INSTANCE i-ac3e214b OutOfService
INSTANCE i-8a823227 OutOfService
[root@odc-d-31 ~]# eulb-describe-lbs BaseLoadBalancer --show-long
LOAD_BALANCER BaseLoadBalancer BaseLoadBalancer-283851778068.lb.localhost {interval=30,target=HTTP:80/index.html,timeout=3,healthy-threshold=2,unhealthy-threshold=2} GIRL i-8a823227,i-ac3e214b {protocol=HTTP,lb-port=80,instance-protocol=HTTP,instance-port=80} {owner-alias=266767588119,group-name=euca-internal-283851778068-BaseLoadBalancer} 2014-04-20T17:32:54.327Z
[root@odc-d-31 ~]# euca-describe-instances i-ac3e214b i-8a823227 
RESERVATION r-a56cef10 283851778068 default
INSTANCE i-ac3e214b emi-7ab4d6ca 10.104.6.230 172.18.0.172 running user01 0 m1.xlarge 2014-04-19T20:18:36.650Z GIRL monitoring-disabled 10.104.6.230 172.18.0.172 instance-store hvm sg-4aaad23a
INSTANCE i-8a823227 emi-7ab4d6ca 10.104.6.231 172.18.0.181 running user01 1 m1.xlarge 2014-04-19T20:18:36.679Z GIRL monitoring-disabled 10.104.6.231 172.18.0.181 instance-store hvm sg-4aaad23a
[root@odc-d-31 ~]# ssh -i euca-admin-elb.priv root@10.104.6.233 
[root@ip-172-18-0-101 ~]# curl http://10.104.6.230/
<html><body><h1>It works!</h1>
<p>This is the default web page for this server.</p>
<p>The web server software is running but no content has been added, yet.</p>
</body></html>
[root@ip-172-18-0-101 ~]# curl http://10.104.6.231/
<html><body><h1>It works!</h1>
<p>This is the default web page for this server.</p>
<p>The web server software is running but no content has been added, yet.</p>
</body></html>

 

References

[1]  Knowledge Base Article - Eucalyptus Elastic Load Balancer Implementation
[2]  Eucalyptus User Guide - Using Elastic Load Balancing
[3]  Eucalyptus User Guide - Using Auto Scaling

 

 

Have more questions? Submit a request

Comments

Powered by Zendesk