The entire intent of load balancing is to create a system that virtualizes
the "service" from the physical servers that actually run that service. A more
basic definition is to balance the load across a bunch of physical servers and
make those servers look like one great big server to the outside world. There
are many reasons to do this, but the primary drivers can be summarized as
"scalability," "high availability," and "predictability."
Scalability is the capability of dynamically, or easily, adapting to
increased load without impacting existing performance. Service virtualization
presented an interesting opportunity for scalability; if the service, or the
point of user contact, was separated from the actual servers, scaling of the
application would simply mean adding more servers or cloud resources which would
not be visible to the end user.
High Availability (HA) is the capability of a site to remain available and
accessible even during the failure of one or more systems. Service
virtualization also presented an opportunity for HA; if the point of user
contact was separated from the actual servers, the failure of an individual
server would not render the entire application unavailable. Predictability is a
little less clear as it represents pieces of HA as well as some lessons learned
along the way. However, predictability can best be described as the capability
of having confidence and control in how the services are being delivered and
when they are being delivered in regards to availability, performance, and so
on.
A Little Background
Back in the early days of the commercial Internet, many would-be dot-com
millionaires discovered a serious problem in their plans. Mainframes didn't have
web server software (not until the AS/400e, anyway) and even if they did, they
couldn't afford them on their start-up budgets. What they could afford was
standard, off-the-shelf server hardware from one of the ubiquitous PC
manufacturers. The problem for most of them? There was no way that a single
PC-based server was ever going to handle the amount of traffic their idea would
generate and if it went down, they were offline and out of business.
Fortunately, some of those folks actually had plans to make their millions by
solving that particular problem; thus was born the load balancing market.
In the Beginning, There Was DNS
Before there were any commercially available, purpose-built load balancing
devices, there were many attempts to utilize existing technology to achieve the
goals of scalability and HA. The most prevalent, and still used, technology was
DNS round-robin. Domain name system (DNS) is the service that translates
human-readable names (
www.example.com)
into machine recognized IP addresses. DNS also provided a way in which each
request for name resolution could be answered with multiple IP addresses in
different order.
Figure 1: Basic DNS response for redundancy
The first time a user requested resolution for
www.example.com, the DNS server would hand
back multiple addresses (one for each server that hosted the application) in
order, say 1, 2, and 3. The next time, the DNS server would give back the same
addresses, but this time as 2, 3, and 1. This solution was simple and provided
the basic characteristics of what customer were looking for by distributing
users sequentially across multiple physical machines using the name as the
virtualization point.
From a scalability standpoint, this solution worked remarkable well; probably
the reason why derivatives of this method are still in use today particularly in
regards to global load balancing or the distribution of load to different
service points around the world. As the service needed to grow, all the business
owner needed to do was add a new server, include its IP address in the DNS
records, and voila, increased capacity. One note, however, is that DNS responses
do have a maximum length that is typically allowed, so there is a potential to
outgrow or scale beyond this solution.
This solution did little to improve HA. First off, DNS has no capability of
knowing if the servers listed are actually working or not, so if a server became
unavailable and a user tried to access it before the DNS administrators knew of
the failure and removed it from the DNS list, they might get an IP address for a
server that didn't work.
Proprietary Load Balancing in
Software
One of the first purpose-built solutions to the load balancing problem was
the development of load balancing capabilities built directly into the
application software or the operating system (OS) of the application server.
While there were as many different implementations as there were companies who
developed them, most of the solutions revolved around basic network trickery.
For example, one such solution had all of the servers in a cluster listen to a
"cluster IP" in addition to their own physical IP address.
Figure 2: Proprietary cluster IP load
balancing
When the user attempted to connect to the service, they connected to the
cluster IP instead of to the physical IP of the server. Whichever server in the
cluster responded to the connection request first would redirect them to a
physical IP address (either their own or another system in the cluster) and the
service session would start. One of the key benefits of this solution is that
the application developers could use a variety of information to determine which
physical IP address the client should connect to. For instance, they could have
each server in the cluster maintain a count of how many sessions each clustered
member was already servicing and have any new requests directed to the least
utilized server.
Initially, the scalability of this solution was readily apparent. All you had
to do was build a new server, add it to the cluster, and you grew the capacity
of your application. Over time, however, the scalability of application-based
load balancing came into question. Because the clustered members needed to stay
in constant contact with each other concerning who the next connection should go
to, the network traffic between the clustered members increased exponentially
with each new server added to the cluster. The scalability was great as long as
you didn't need to exceed a small number of servers.
HA was dramatically increased with these solutions. However, since each
iteration of intelligence-enabling HA characteristics had a corresponding server
and network utilization impact, this also limited scalability. The other
negative HA impact was in the realm of reliability.
Network-Based Load balancing
Hardware
The second iteration of purpose-built load balancing came about as
network-based appliances. These are the true founding fathers of today's
Application Delivery Controllers. Because these boxes were application-neutral
and resided outside of the application servers themselves, they could achieve
their load balancing using much more straight-forward network techniques. In
essence, these devices would present a virtual server address to the outside
world and when users attempted to connect, it would forward the connection on
the most appropriate real server doing bi-directional network address
translation (NAT).
Figure 3: Load balancing with network-based
hardware
The load balancer could control exactly which server received which
connection and employed "health monitors" of increasing complexity to ensure
that the application server (a real, physical server) was responding as needed;
if not, it would automatically stop sending traffic to that server until it
produced the desired response (indicating that the server was functioning
properly). Although the health monitors were rarely as comprehensive as the ones
built by the application developers themselves, the network-based hardware
approach could provide at least basic load balancing services to nearly every
application in a uniform, consistent manner—finally creating a truly virtualized
service entry point unique to the application servers serving it.
Scalability with this solution was only limited by the throughput of the load
balancing equipment and the networks attached to it. It was not uncommon for
organization replacing software-based load balancing with a hardware-based
solution to see a dramatic drop in the utilization of their servers. HA was also
dramatically reinforced with a hardware-based solution. Predictability was a
core component added by the network-based load balancing hardware since it was
much easier to predict where a new connection would be directed and much easier
to manipulate.
The advent of the network-based load balancer ushered in a whole new era in
the architecture of applications. HA discussions that once revolved around
"uptime" quickly became arguments about the meaning of "available" (if a user
has to wait 30 seconds for a response, is it available? What about one
minute?).
This is the basis from which Application Delivery Controllers (ADCs)
originated.
The ADC
Simply put, ADCs are what all good load balancers grew up to be. While most
ADC conversations rarely mention load balancing, without the capabilities of the
network-based hardware load balancer, they would be unable to affect application
delivery at all. Today, we talk about security, availability, and performance,
but the underlying load balancing technology is critical to the execution of
all.
ps
Next Steps
Ready to plunge into the next level of Load Balancing? Take a peek at these
resources:
