Choosing the right load balancer (2024)

Authors: Priyanka Vergadia, Stephanie Wong

“Get Cooking in Cloud” is a blog and video series to help enterprises and developers build business solutions on Google Cloud. In this series we plan on identifying specific topics that developers are looking to architect on Google cloud. Once identified we create a mini series on that topic.

In this miniseries, we will go over Google Cloud load balancing.

1. Choosing the right load balancer (this article)
2. Application Capacity Optimizations with Global Load Balancing
3. Capacity Management with load balancing
4. Load Balancing to GKE network endpoint groups

In this article we will cover different load balancing options available in Google Cloud platform. Using an example use case, we will help you define the process of picking the right load balancing option.

• What is load balancing?
• Why is load balancing important?
• Different load balancing options
• How to choose the right load balancing option

Imagine you’re in charge of a complex website, Beyond Treat (your one stop shop for vegan dog treats). And it’s an online hit! It continues to face high amounts of traffic and you’re not sure your website backend can handle the amount of traffic from all over the world. You know you need to use load balancers, but the options are confusing. It can be hard to know exactly how to settle on a load balancing architecture that meets your needs, and figure out the prerequisites you need, for the best performance, without making too much of a dent in your wallet.

To start, you need to know what load balancing is and why it’s so important to the long-lasting success of your application.

Load balancing is the process of distributing traffic across your network of servers to ensure that the system does not get overwhelmed and all requests are handled easily and efficiently.

Modern high‑traffic websites serve hundreds of thousands, if not millions, of concurrent requests from users or clients and return the correct text, images, video, or application data, all in a fast and reliable manner. You’ve probably all experienced visiting your favorite website, only to get long wait times, connection timeout errors, or images and videos buffering. And a lot of the times, this is because the website backend is unable to cost‑effectively scale to meet these high volumes.

The logical answer here is to add more backend servers to help serve traffic. But the next question becomes, how do you distribute traffic to those backend servers based on capacity and health?

This is where load balancing makes a splash. Load balancing is the process of distributing traffic across your network of servers to ensure that the system does not get overwhelmed and all requests are handled easily and efficiently.

There are a number of reasons why load balancing is important. Load balancer lets you:

• Distribute load-balanced resources in single or multiple regions
• Meet your high availability requirements
• Scale your resources up or down with intelligent Autoscaling
• And use cloud content delivery network (CDN) for optimal content delivery

With Google Cloud Load Balancing, you can serve content as close as possible to your users, on a system that can respond to over 1 million queries per second!

To decide which load balancer best suits your implementation, you need to think about whether you need

1. Global or regional load balancing. Global load balancing means backend endpoints live in multiple regions. Regional load balancing means backend endpoints live in a single region.

2. External or internal load balancing

3. What type of traffic you are serving? HTTP, HTTPS, SSL, TCP, UDP etc.

So clearly, for Beyond Treat, there are many Load Balancing options depending on where exactly they need a load balancer in the architecture.

They would use external load balancers to distribute traffic coming from the internet to their Google Cloud network and internal load balancers to distribute traffic within their GCP network.

External load balancer

External load balancing includes four options:

1. HTTP(S) Load Balancing for HTTP or HTTPS traffic,
2. TCP Proxy for TCP traffic for ports other than 80 and 8080, without SSL offload
3. SSL Proxy for SSL offload on ports other than 80 or 8080.
4. Network Load Balancing for TCP/UDP traffic.

Network load balancer

While the global HTTP(S) load balancer is for Layer-7 traffic and is built using the Google Front End Engines at the edge of Google’s network, the regional Network Load Balancer is for Layer-4 traffic and is built using Maglev.

Network load balancer is for the Layer-4 traffic

What is Maglev?

Google built Maglev in 2008 to load balance all traffic that comes into the data centers, and distribute traffic to front-end engines at the network edges. The traffic is distributed to a set of regional backend instances.

Maglev was a break from traditional load balancers because it is software-based and operates in an active-active scale-out architecture. Maglev evenly distributes traffic over hundreds of backends and minimizes negative impact of unexpected faults on connection-oriented protocols. It’s great for lightweight L4-based load balancing where you want to preserve the client IP address all the way to the backend instance and perform TLS termination on these instances.

HTTP(S) load balancers

Global HTTP(S) load balancing if for layer-7 traffic

Google pushed load balancing out to the edge network on front-end servers, as opposed to using the traditional DNS-based approach. Thus, global load-balancing capacity can be behind a single Anycast virtual IPv4 or IPv6 address. This means you can deploy capacity in multiple regions without having to modify the DNS entries or add new load balancer IP address for new regions. So, it is clear that with global HTTP(S) load balancing, you get cross-region failover and overflow!

With global HTTP(S) load balancing, you get cross-region failover and overflow!

The distribution algorithm automatically directs traffic to the next closest instance with available capacity in the event of failure of or lack of capacity for instances in the region closest to end user. We’ll cover this more in the next couple articles.

Proxy based load balancers (TCP and SSL)

Google Cloud also offers proxy-based load balancers for TCP and SSL traffic, and they use the same globally distributed infrastructure.

Use TCP proxy load balancer when you are dealing with TCP traffic and do not need SSL offload.

Generally speaking, your decision to use them would depend on whether you require SSL offload or not. You can find out more in the links below.

Use SSL proxy load balancer when you are dealing with TCP traffic and need SSL offload.

Now, let’s get back to Beyond Treat. In Beyond Treat’s case, we are only going to receive HTTP and HTTPS traffic, so they will choose HTTPS load balancer. But, like most companies, the website also has private workloads, such as application servers, that need to be protected from the public internet. Those services need to scale and grow behind a private virtual IP that is accessible only by internal instances. For this their best option is the regional layer-7 internal Load Balancing based on Google’s Andromeda network virtualization stack.

Internal Load Balancer

Similar to the HTTP(S) Load Balancer and Network Load Balancer, Internal L7 load balancer is neither a hardware appliance nor an instance-based solution, and can support as many connections per second as you need since there’s no load balancer in the path between your client and backend instances.

Internal layer 7 load balancer can support as many connections per second as your need!

Over the next few articles, we’ll follow Beyond Treat as they grow and even experience unpredictable spikes in traffic…like when it’s international dog day and big bone treat orders are up the wazoo 😃. This makes their website an ideal candidate for explaining all the load balancing options you have at your disposal. But of course, you can apply the principles described in these articles to a wide range of workloads.

Well, there you have it, Load balancing deconstructed! Stay tuned for more articles in the Get Cooking in Cloud series and checkout the references below for more details.

• Follow this blog series on Google Cloud Platform Medium.
• Reference: Choosing a load balancer
• Follow Get Cooking in Cloud video series and subscribe to Google cloud platform YouTube channel
• Want more stories? Follow me on Medium, and on twitter.
• Enjoy the ride with us through this miniseries and learn more about more such Google Cloud solutions :)