Load Balancing with OpenStack

One of the key strength of Cloud applications is High Availability which ensures that the applications and services are accessible to users all the time with as little down time as possible. The backbone of High Availability is Load Balancing which also comes as a type of cloud model known as LBAAS(Load Balancing As A Service). In my earlier article  , I had explained how to configure your OpenStack setup to enable Load balancing. In this article , we will see how to load balance services in OpenStack taking some basic examples.

Lets begin with some concepts of Load balancing in OpenStack -
VIP - it is like a listener for clients which will receive requests on behalf of services being load balanced. It constitutes of an IP and port on which it can listen.

LB Pool - it is a pool of VMs which are being load balanced. LB pool is responsible for identifying which member will process the request coming to VIP. Each VIP has one pool.

Member - it is the application/server which is being load balanced.

Health Monitor - it is responsible for identifying which members are fit to process requests coming to VIP and which members are not. A pool can be associated with many monitors. Following type of health monitors are supported - PING, TCP, HTTP, HTTPS.

Below are high level steps used for load balancing in OpenStack -
1. Create a pool.
2. Create one or more members to be added in pool.
3. Create one or more health monitors to be associated with pool.
4. Add members to pool.
5. Associate health monitors with pool.
6. Create VIP
7. Associate VIP with pool.

Lets take an example of httpd service load balancing where in we will try to send wget or curl requests for VIP and will in return be getting host names of members being load balanced.

Create a Pool
Login to dashboard and navigate to Load Balancer section to create new pool.

While creating pool we can specify provider like HA Proxy, Subnet to be associated with pool , Protocol and load balancing method.

2. Add members to pool
In this step we will add member VMs to the pool .

In this I have added to members to the pool.


3. Create Health monitor 

The health monitor inspects periodically the members of pool to check their availability and health in order to process request.


4. Associate Health monitor with pool

5. Create VIP

Final step is to create Virtual IP address (VIP) which will handle traffic. The VIP should come from the same subnet as that of the pool. It may be allocated as Public IP so that it is reachable from external network.

We have now configured load balancer on 2 VMs each running httpd service in order to cater to requests coming to VIP.

Testing

Lets hit VIP (10.0.0.10) and see the responses

for i in {1..4} ; do curl -w "\n" 10.0.0.10 ; done
vm1
vm2
vm1
vm2

Hence we can see the responses of individual VM in a round robin manner.

Next you can disable one of the members of pool and try the above command again. It can be observed that despite of one VM going down, the requests to VIP are still fulfilled.

Hope it helped !

Configuring Load Balancers in OpenStack

After you install OpenStack through RDO (which is most widely used) , Load Balancer are not by default enabled. To enable Load balancer in OpenStack Havana or Ice House following steps need to be followed :-
Load balancing can be achieved by HA Proxy which is a fast and reliable solution for High availability and load balancing. Ha Proxy needs to be installed in order to enable Load balancing in OpenStack.
To install Ha Proxy on Centos5/RHEL 5 you need to add Epel packages via
# rpm -Uvh http://dl.fedoraproject.org/pub/epel/5/i386/epel-release-5-4.noarch.rpm

Next to install HA Proxy -
# yum install haproxy

Following changes are required in OpenStack configuration files :-
1.Edit /etc/neutron/neutron.conf and add the following in the default section:
      service_plugins = neutron.services.loadbalancer.plugin.LoadBalancerPlugin

   Also, comment out the line in the service_providers section:

 service_provider=LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default

2.Set Ha Proxy drivers  in /usr/share/neutron/neutron-dist.conf: 

service_provider = LOADBALANCER:Haproxy:neutron.services.loadbalancer.drivers.haproxy.plugin_driver.HaproxyOnHostPluginDriver:default

3.Edit  /etc/neutron/lbaas_agent.ini file and add HA Proxy device driver

device_driver = neutron.services.loadbalancer.drivers.haproxy.namespace_driver.HaproxyNSDriver

4.The interface_driver will depend on the core L2 plugin being used

For OpenVSwitch:
interface_driver = neutron.agent.linux.interface.OVSInterfaceDriver

For linuxbridge:
interface_driver = neutron.agent.linux.interface.BridgeInterfaceDriver

5. Make changes in /etc/openstack-dashboard/local_settings file so that Load balancer section starts appearing in dashboard 

OPENSTACK_QUANTUM_NETWORK = {
    'enable_lb': True

}

6. Restart the services 
sudo service neutron-lbaas-agent start
sudo service neutron-server restart
sudo service httpd restart

7.Login to dashboard to see Load Balancer section

VM Provisioning flow in OpenStack

In my earlier article OpenStack Components , I had given an overview of components that make up the OpenStack cloud . These components work in an integrated way to fulfill service requests like VM provisioning. In this article I will be explaining the backend steps that happen when a user launches an instance from OpenStack Dashboard or though CLI (Command line client).

Just to heads up , below command is used to boot an instance in OpenStack through CLI :

nova boot --image <<glance image id>> --flavor <<flavor id>> <<server name>> --nic net-id=<<network id to boot upon>>

When an instance is launched , OpenStack components work together in a cohesive manner to fulfill the request. Following steps happen in the backend :-

1. Dashboard or CLI sends authenticates the user credentials provided in request from Keystone.
2. Keystone returns an authentication token which will be used for interacting and authenticating with other components.
3. Request then goes to nova-api which make DB entry for the new instance to be created and lines up an entry in queue.
4. nova-api then forward the request to nova-scheduler which picks up the entry from queue . It then applies scheduler algorithms and filters to find a suitable host on which the instance will be created.
5. nova-scheduler return the entry from DB with updated host id in it .
6. nova-scheduler forwards the request to nova-compute for launching instance on host.
7.nova-compute passes the request to nova-conductor to find information about flavor (RAM,CPU required).
8. nova-conductor queries DB to find the required information and returns it to nova-compute.
9. nova-compute creates a REST call out of all the information and hits glance service to find the image details from which the instance will boot.
10. nova-compute then passes REST call to neutron/quantum to get IP address for the instance.
11. Next the request is forwarded to cinder service to attach volume to instance.
12. Finally the request is passed to hypervisor to create a new instance .

VM Placement Options in Openstack

In my last article, I wrote about different levels in which OpenStack cloud can be segregated  . These are hosts , host aggregates , availability zones and regions. In this article, I will be mentioning different options available to users for placing their virtual machine instances on these constructs based on suitability.

When a user creates an instance, the nova-scheduler service comes into picture and is responsible for selecting a target host for this requested instance. For doing this, all compute nodes publish their resources and other artifacts to scheduler which keeps the data handy and available . By default , scheduler is configured to choose hosts based on following criteria :-
1. Whether the host falls in requested Availability Zone
2. Whether the host has required RAM available
3. Whether the host has required CPU available

This selection criteria is configurable in many ways like filters. Lets check out various placement options :-

1. Placement on desired Availability Zone
    Availability zone is preferably created as host aggregate which is exposed to end users in form of availability zone. Below steps illustrate how to create aggregate and specify it while provisioning :

    - To create host aggregate , below command is used :
          nova aggregate-create <name> [<availability-zone>]
       Eg, nova aggregate-create Aggregate_1 AZ_1

    - Add host to it using below command :
          nova aggregate-add-host <aggregate> <host>
       Eg. nova aggregate-add-host Aggregate_1 Host_A
    - Now when an instance is created , created availability zone can be mentioned as a target for that instance.
          nova boot --flavor 1 --image 6116f510-18c5-44e0-8a81-868e1c9fc79f --availability-zone AZ_1                   testinstance

   Note : we can also specify particular host within availability zone in above command

       nova boot --flavor 1 --image 6116f510-18c5-44e0-8a81-868e1c9fc79f --availability-zone  AZ_1: Host_A testinstance

2. Placement on desired Host aggregate
    Host aggregates provide mechanism to group hosts that have a common capability like a group of hosts having SSD drives enabled. OpenStack scheduler is not by default enabled to support this and can be configured by making below changes in nova.conf file :-
    Lookout for property scheduler_default_filters and append AggregateInstanceExtraSpecsFilter filter to its     value.
     Selecting host aggregate is done through metadata associated with it. Metadata contains key-value pairs that can be matched while provisioning instances.
     Eg, we can create a ssd=true key-value pair and attach it to metadata of a host aggregate.
      nova aggregate-set-metadata Aggregate_1 ssd=true

    Next step is to create a flavor and add the same key-value pair to it's extra specs.
    Create flavor using below command :
       nova flavor-create ssd.large 6 512 10 1
    Add key value pair to it
       nova-manage instance_type set_key --name=ssd.large --key=ssd --value=true

    Finally when an instance is launched with this flavor, the nova-scheduler will match the host aggregate  based on the metadata of host aggregate and extra specs of flavor to determine the target host aggregate for new instance.

3. Filters 
    OpenStack provides filter scheduler which allows users to have their custom vm placement choices. There are many filters available in Openstack like ComputeFilter , DifferentHostFilter , RamFilter , JsonFilter etc.
    When a request is made to launch instance, Filter scheduler first lines up all hosts and filters them based on the filter provided in request . After making a subset of hosts , then wights are applied to determine the destination host.
   Filters are specified by --hint option in the nova boot command .
    For example , this filter will select hosts that have free RAM more that 4 GB available.
nova boot --image 58ae75f1-3840-4149-bb0b-e52badbd1d4c --flavor 1 --hint query='[">=","$free_ram_mb",4096]' server_correct --nic net-id=16630baf-7b7a-45bf-88a5-cbbcd5d22cab




 

       

OpenStack Cloud Partitioning(Availability Zones/Regions/Host Aggregates)

              

One of the key features that almost every cloud provider will provide is High Availability which ensures that you applications running on cloud are up and accessible all the time. Suppose there is a single cloud data center having all of infrastructure that the cloud hosts,  and there occurs a natural calamity like an earthquake which causes lot of harm to the data center. What would happen in this case? All the data contained in that data center would be lost and likely to be recovered also causing downtime for hosted applications. On the other hand if there would have been a distributed environment where data would be stored in not one but many data centers residing in different geographical location , there would be minimal impact of one data center going down. This is a very important aspect of cloud which every cloud provider must provide.

In OpenStack , spreading of data centers(if I can use a generic term for cloud data repository) is achieved via Regions, Availability Zones and Host aggregates. In this article , I will explain about each of these starting from lowest granular entity viz host to highest entity viz Region.

Host : a host is where your hypervisor runs , which has compute power or where your virtual machine instances are placed actually.A host has a nova-compute service running on it. In a single node setup there would be just one host containing compute and all other nodes which is not a good cloud implementation as it is more susceptible to breakdown.

Host Aggregate : host aggregates are like a collection of hosts that share a common property. This is more of logical grouping of hosts which share same set of capabilities. Host aggregates allow administrators to group hosts based on some common characteristic so that while launching instances , only those hosts are eligible for placement which qualify the requested capability. For example, an administrator can group hosts which have SSD drives enabled for faster disk I/O thus when a user requests for an instance having faster disk I/O , one of the host from that host aggregate can be selected for placement. The task of selecting appropriate host is done by Openstack scheduler. Host aggregates are only accessible to cloud administrators.

Availability Zone : an availability zone is next level of grouping and may contain one or more host aggregates and/or compute nodes. It is more of geographical division of OpenStack cloud and allows administrators to spread their compute nodes across geographies in order to achieve high availability.Availability zones are exposed to end users who can select a particular availability zone for VM placement while the instance is launched.

Region : a region is the biggest building block of Openstack cloud. It contains one or more availability zones . Each region within a cloud deployment will have its own set of Openstack services(network,compute,image,volume,storage etc) and endpoints except for keystone and dashboard. Within one cloud deployment , different regions will share one common keystone and dashboard.

OpenStack Components

                       OpenStack Components

In my last article Why Use OpenStack , I had given a very basic introduction to OpenStack and how to get started with it. In this article , I will be providing basic information about various OpenStack components and its architecture (for IceHouse Openstack release).

OpenStack has a modular architecture having various modules that work in an integrated manner to provider cloud services to users. These components may all be present on one machine or may reside on different modules thus providing loose coupling among them.

OpenStack deployment has following components :

Nova (compute) : the controller component in OpenStack and is responsible for managing compute resources like virtual machine instances, hosts, firewalls etc .It looks up from other components for spawning virtual machine instances and manages the life cycle of instances hosted in OpenStack.

Neutron/Quantum (network) : the networking component of OpenStack that is responsible for managing networks and IP addresses of virtual machines. The name of networking component was Quantum till Grizzly release and thereafter it was named Neutron. It provides features like creating networks, subnets, public IP pools .

Cinder (block storage) : this component provides mechanism for horizontally scaling disk capacity of instances and makes use of Logical Volume Manager for catering to such needs of additional volume requirements of virtual machines.

Horizon (dashboard) : the graphical user interface provided to OpenStack administartors and end users to manage the resources easily through a thin client browser. It also provides the resource utilization charts keeping the users well informed of their capacity and resource usage. It provides multilingual support for about 16 languages.

Keystone (identity) : the security component of OpensStack is known as Keystone and is responsible for validating each API request received. All components of OpenStack use it to authenticate and authorize request coming to them by means of an authorization token provided by Keystone. The token generated by Keystone also contains service catalog which contains API endpoints information for all components.

Glance (image) : virtual machine instances are spawned over operating system images like rhel , windows etc. Glance is the component that allows users to upload new images in some specified formats or create snapshots from existing instances. 

Swift (object storage) : it is the data repository for OpenStack and is responsible for managing storage capabilities . 

Cielometer (telemetry) : it is metering system of Openstack for managing customer billings and charge back purposes. 

Trove (dbaas) : it is the newest addition to family of OpenStack components and provides database as a service to users. 

Heat (orchestration) : automated scaling of instances in OpenStack is achieved via Heat component.

For more information on OpenStack , please visit http://www.openstack.org/

OpenStack Cloud Provider. What is is about?

                                OpenStack as a Cloud Provider

What is OpenStack?

OpenStack is a cloud platform which can be utilized as a private or public cloud. Within the world of cloud computing consisting of giant public cloud providers like Amazon and Windows Azure which offer paid services , OpenStack has come as an inexpensive solution to cater to infrastructure resource requirements for small to large scale business operations. 

It is Open Source

 Being an open source software , you may access the source code and customize it according to your needs. It is currently managed by OpenStack Foundation and is backed up by some of the big names like Rackspace , HP, at&t , redhat and more. Millions of developers world wide are contributing to OpenStack with a very fast release cycle of 3 months. Currently , OpenStack has come up with many releases like Folsom, Grizzly ,Havana and latest release of Ice House which has many interesting additions like Database as a Service .

Performance wise OpenStack is well reputed and can spin virtual images within couple of minutes(depending upon image size). It provides features such as visualization, creating and managing networks , load balancing, firewalls, auto scaling etc. OpenStack provides a very easy and thin client dashboard for managing your instances, networks , security and other infrastructure resources. You may upload various OS images like RHEL , Windows , Ubuntu in OpenStack specified formats and can have many flavors of these images depending upon your needs in terms of RAM , CPU and disk. 

Setting up your own OpenStack cloud

Setting up your OpenStack cloud can be as simple as running three commands and can even hardly take 15-20 minutes. All you need is a linux machine to setup OpenStack cloud and follow the RDO community guidelines. This community provides the easiest way of installing OpenStack in your lab.

Or, if you want to have a quick feel of how OpenStack works , you may use TryStack and spin some virtual machines and play around with it.