What Is a Load Balancer? A Beginner’s Guide With Real Examples and Code

From traffic police to Anycast routing — hardware, software, and cloud load balancers explained with real examples, not textbook definitions.

What Is a Load Balancer? A Beginner’s Guide With Real Examples and Code

Case Study: Hotstar

It’s the last over of an IPL final. Two runs needed off one ball. You’re on Hotstar, phone tilted, thumb hovering, and right at that exact second — the video freezes. That little spinning circle just sits there mocking you while your WhatsApp group is already screaming.

Here’s the thing that used to bug me: at that same moment, roughly 30 crore other people are watching the exact same six-second clip on the exact same app. Thirty crore. That’s more than the entire population of most countries, all hammering the same set of servers, all wanting the same video frame, at the same instant.

So why doesn’t Hotstar just… die?

The answer is one unglamorous, rarely-talked-about piece of infrastructure sitting quietly at the front door of almost every app you use — Instagram, YouTube, your banking app, even the food delivery app you ordered dinner from an hour ago. It’s called a load balancer, and once you understand it, you’ll start seeing it everywhere.

Ever wondered what actually happens in the 200 milliseconds between you tapping “play” and the video starting?

First, let’s break the app without one

Imagine you built a small app. One server. Everything’s fine when 50 people use it a day. Then one day it goes viral, and suddenly 50,000 people hit it in the same hour.

What happens to that one server?

  • CPU usage climbs to 100%

  • Memory fills up

  • Response times crawl from milliseconds to seconds

  • Eventually — it crashes

I’ve seen this happen in real life with a college project turned side hustle. A friend built a simple event-registration site, posted it on a WhatsApp group of 200 people, and the server fell over within 20 minutes. Not because the code was bad — it just had nowhere to send the overflow. One server, one point of failure, zero flexibility.

This is exactly the problem a load balancer exists to solve.

So what actually is a load balancer?

Think of it as a traffic police officer standing at a busy intersection, except instead of directing cars, it’s directing requests.

                                    ┌──────────┐
                                    │  Server 1│
                                   ┌┴──────────┴┐
   User Request ──▶ Load Balancer ─┤  Server 2  │ 
                                   └┬──────────┬┘
                                    │  Server 3│
                                    └──────────┘

Your request never goes straight to “the server.” It goes to the load balancer first, and the load balancer decides — based on current load, location, or health of each server — exactly which server should handle it.

In plain terms:

A load balancer is a system that spreads incoming traffic across multiple servers, so no single server gets overwhelmed.

That’s it. That’s the whole idea. Everything else is just implementation detail.

Why does this actually matter, though? Three concrete reasons:

  1. No single point of meltdown — if traffic is spread across 10 servers instead of putting all pressure on 1, each one just does 10% of the work.

  2. Speed — a load balancer can route you to the nearest server, and distance directly affects how fast data reaches you. Less distance, less delay.

  3. Uptime — if one server crashes, the load balancer simply stops sending traffic there and routes everyone to the healthy ones. You, the user, never even notice.

Have you ever had an app stay up during a random server outage without realizing anything went wrong behind the scenes? That’s a load balancer quietly doing its job.

A tiny working example

Let’s not just talk about it — let’s see it. Here’s a stripped-down load balancer in Node.js using a technique called round robin (basically: take turns).

const http = require('http');
const httpProxy = require('http-proxy');
// Our "servers" - in real life these would be different machines
const servers = [
  'http://localhost:3001',
  'http://localhost:3002',
  'http://localhost:3003',
];
let currentIndex = 0;
const proxy = httpProxy.createProxyServer();
const loadBalancer = http.createServer((req, res) => {
  // Pick the next server in line, then wrap back to the start
  const target = servers[currentIndex];
  currentIndex = (currentIndex + 1) % servers.length;
  console.log(`Routing request to: ${target}`);
  proxy.web(req, res, { target });
});
loadBalancer.listen(3000, () => {
  console.log('Load balancer running on port 3000');
});

That’s genuinely most of what a basic load balancer does — it just picks the next available server on a rotation and forwards the request there. Real production ones (Nginx, HAProxy, AWS ELB) do the same core thing but add health checks, weighting, session stickiness, and a hundred other refinements on top.

Curious what happens if you kill one of those three backend servers mid-traffic? Try it — that’s exactly the “availability” benefit in action.

The three flavors of load balancers

When people (or interviewers) ask “what types of load balancers exist,” there are three real answers.

  1. Hardware load balancers A literal, physical, dedicated machine whose entire job is load balancing. Nothing else runs on it. Brands like F5 BIG-IP are the classic names here. Big enterprises and data centers — think Google-scale operations — use these because dedicating a whole machine to one job means maximum performance. The catch? It’s expensive, exactly because it’s not shared with anything else.

2. Software load balancers Instead of a dedicated box, you run load-balancing software on a regular server, VM, or Kubernetes cluster. Tools like Nginx, HAProxy, and Envoy fall here. Since the machine might be doing other things too, there’s marginally more overhead — but it’s far cheaper, and this is what most small-to-mid companies actually use.

3. Cloud load balancers This is what most modern apps use today, because most apps are hosted on the cloud anyway. AWS, Google Cloud, and Microsoft Azure each provide their own managed load balancer — Elastic Load Balancer (AWS), Cloud Load Balancing (GCP), Azure Load Balancer. You don’t manage hardware or install software; the cloud provider handles it for you.

If you’re prepping for interviews, these six names — F5/BIG-IP, Nginx, HAProxy, Envoy, AWS ELB, GCP/Azure LB — are worth remembering by name. They come up a lot.

Layer 4 vs Layer 7 — the part that trips people up

This is the bit most beginner explanations skip, and it’s exactly the bit interviewers love to probe.

You’ve probably heard of the OSI model — the seven layers networking is built on. Load balancers usually operate at one of two of those layers:

Layer 4 (Transport layer) Checks only the IP address and port number. It works at the TCP/UDP level. Fast, lightweight, doesn’t peek into the actual content of the request.

Layer 7 (Application layer) This is smarter. It can look at the URL, headers, cookies, and API routes — the actual content of what you’re asking for.

Here’s why that distinction matters with a real example: imagine Instagram’s backend. 

Reels (video) need very different handling than chat messages, which need very different handling than the upload flow. 

A Layer 4 load balancer can’t tell these apart — it only sees “a request came in on port 443.” A Layer 7 load balancer can look inside and say “ah, this is a /reels request, send it to the video-optimized servers" versus "this is a /chat request, send it to the messaging cluster."

The tradeoff? Layer 7 does more work, so it costs more compute. Layer 4 is cheaper and faster but dumber. Which one should you use? Depends entirely on whether you actually need that content-level routing intelligence, or whether basic distribution is enough.

Which one do you think your favorite app is running — and would you even be able to tell the difference as a user?

How the giants really do it (this is the part that blew my mind)

Here’s something that genuinely surprised me the first time I dug into it: massive companies like Google, YouTube, and Amazon don’t use one load balancer. They don’t even use one data center.

They run data centers in multiple regions across the world — India, USA, Europe, Australia, and so on. So the actual flow of your request looks more like this:

User → DNS → Global Load Balancer → Regional Load Balancer → Nearest Server

  1. DNS first hands you an IP address.

  2. A global load balancer does the first-level traffic distribution.

  3. It routes you to a regional load balancer based on where you physically are.

  4. That regional layer sends you to the nearest healthy server.

And the technique that makes step 3 almost magical is called Anycast. With Anycast, multiple servers around the world literally share the same IP address. You don’t need to figure out which one is closest — the network automatically routes your request to whichever server is nearest to you. No manual check-in, no extra lookup. It just happens.

On top of all this, load balancers often cache frequently-requested content right at the edge. Think about a new movie trailer dropping on YouTube — instead of every single viewer’s request traveling all the way to YouTube’s origin servers, the load balancer (or the CDN layer near it) just serves it directly from a cached copy sitting closer to you. Faster for you, less load on the actual servers.

Did you know your video buffered faster during a viral trailer drop because of caching, not despite the traffic spike? Kind of counterintuitive, right?

Wrapping this up

Here’s what stuck with me most after going down this rabbit hole: a load balancer isn’t some exotic, advanced piece of tech reserved for FAANG-scale companies. It’s a genuinely simple idea — spread the load, don’t let one point take the whole hit — dressed up in increasingly clever implementations as the scale increases.

A hobby project might get away with a single Nginx instance doing round robin. Hotstar during a World Cup final needs global load balancers, regional layers, Anycast routing, and aggressive caching, all working together. Same core idea, wildly different scale.

If there’s one thing I’d want a beginner to walk away with, it’s this: every “how does this app handle millions of users” question in system design starts with this exact component. Understand this deeply, and half of system design interviews suddenly feel a lot less intimidating.

So — next time your favorite app doesn’t crash during a moment when logically it really, really should, you’ll know exactly who to thank.

What’s a time an app did crash on you at the worst possible moment? Drop it in the comments — I want to hear the disaster stories.