Implementing a Basic Load Balancer Simulation using Round Robin (2/n)

Let us implement a basic load balancer simulation using Round Robin algorithm.

This is an oversimplified implementation of load balancer and is no way mark to the production.

Video reference - https://www.linkedin.com/feed/update/urn:li:activity:7418682309714882560/

Technologies used here
JavaScript, Node.js, express server, postman

In this, we will initialize three backend servers and distribute incoming requests across them using a Round Robin strategy.

Setup

1. Initialize a node repo by entering the following command in terminal and enter the values needed, this initializes a blank node repo

    npm init
   

2. Change repo type to module.
   Goto your project and open ./package.json file and add the following key value pair to it

    "type": "module",
   

3. Install dependencies

   Install express in your project

    npm i express
   

4. Folder Structure.

    your-project/
    ├─ server1/
    │  └─ app.js
    ├─ server2/
    │  └─ app.js
    ├─ server3/
    │  └─ app.js
    ├─ app.js
    ├─ fetchserver.js
    └─ package.json
   

   We will create three folders, one for each server, add app.js file in them for our server app

   Each server runs independently to simulate distributed backend services.

   We will create 2 files in our root app.js and fetchserver.js

   app.js is our express app for our load balancer whereas fetchserver.js contains our core which will return us next best available server

Code

Server Code

Under the app.js file of each server we will create a basic express server for serving basic health check route.
The basic health check route returns if is the server active or not(status)

import express from 'express';

const app = express();
const port = 8000; // port 8000 for server 1

app.get('/status', (req, res) => {
  res.json({ status: 'available' });
});

app.listen(port, () => {
  console.log(`Server1 is running on port ${port}`);
});

We will create all 3 servers in different ports, here we are taking 8000, 8001 and 8002 for server 1, 2 and 3 respectively.

Load Balancer Code

1. FetchServers

We will be using Round Robin algorithm for our load balancing here.

Round Robin Load Balancing is a simple algorithm that distributes incoming requests evenly across a group of servers in a circular order.

Each incoming request is assigned to the next server in a fixed sequence. Once the last server is reached, the algorithm starts again from the first server, forming a loop.

In the root fetchserver.js file contains the function fetchAvailableServer where we return the available server port and current serverIndex when available else return null and -1 and take last used server as our argument

const servers = [8000, 8001, 8002];

const fetchAvailableServer = async (lastUsedIndex) => {
  const totalServers = servers.length;

  for (let i = 0; i < totalServers; i++) {
    lastUsedIndex = (lastUsedIndex + 1) % totalServers;
    const serverPort = servers[lastUsedIndex];

    try {
      const res = await fetch(`http://localhost:${serverPort}/status`);
      const data = await res.json();

      if (data.status === "available") {
        return [serverPort, lastUsedIndex];
      }
    } catch (err) {
      console.log(`Server ${serverPort} is DOWN`);
    }
  }

  return [null, -1];
};

export { fetchAvailableServer };

2. Load Balancer app

import express from "express";
import { fetchAvailableServer } from "./fetchServer.js";

const port = 3000;
const app = express();
let lastUsed = -1;

app.get("/send", async (req, res) => {
  const [availServer, currentServer] = await fetchAvailableServer(lastUsed);
  if (availServer) {
    res.send(`Request forwarded to server: ${availServer}`);
    lastUsed = currentServer;
  } else {
    res.send("No available servers to handle the request");
    lastUsed = -1;
  }
});

app.listen(port, () => {
  console.log(`Server is listening on port ${port}`);
});

We will run the main load balancer app on Port:3000

Under the /send route, the load balancer first searches for an available server by calling the fetchAvailableServer() function.

If any servers are available, it routes the request to that server

Running

Let’s start all the server (1,2 and 3) on different terminals

cd server1; node app.js; cd ../
cd server2; node app.js; cd ../
cd server3; node app.js; cd ../

Let’s run the load balancer app

node app.js

Now let’s call the localhost:3000/send GET request

Output:

You can see that our request is forwarded to server 1 (port:8000), cause all the servers are currently available and server 1 is the first one among them

Again call the localhost:3000/send GET request

You can see that here our request is forwarded to server 2 (port:8001), cause our last used server was 1 and we send our next request to next available server in circular fashion

Similar thing happens when you send another request, it will be forwarded to server 3

After shutting down all the available servers the load balancer catches the error and resets the last used server to -1

Extensions

We can set up the load balancer as skip to next server whenever the request is taking more than 1000ms of request time
In this example, a slow server is treated as unavailable even if it eventually responds successfully.

Changes to the fetchServers.js file:

const servers = [8000, 8001, 8002];

const fetchAvailableServer = async (lastUsedIndex) => {
  const totalServers = servers.length;

  for (let i = 0; i < totalServers; i++) {
    lastUsedIndex = (lastUsedIndex + 1) % totalServers;
    const serverPort = servers[lastUsedIndex];

    const controller = new AbortController();
    const timeout = setTimeout(() => controller.abort(), 1000);

    try {
      const res = await fetch(
        `http://localhost:${serverPort}/status`,
        { signal: controller.signal }
      );

      clearTimeout(timeout);
      const data = await res.json();

      if (data.status === "available") {
        return [serverPort, lastUsedIndex];
      }
    } catch (err) {
      clearTimeout(timeout);
      console.log(`Server ${serverPort} is DOWN or SLOW`);
    }
  }

  return [null, -1];
};

export { fetchAvailableServer };

We can add setTimeout to our servers for our demo

import express from 'express';

const app = express();
const port = 8000;

app.get('/status', (req, res) => {
  setTimeout(() => {
    res.json({ status: 'available' });
  }, 1001);
});

app.listen(port, () => {
  console.log(`Server1 is running on port ${port}`);
});

Now when we call our load balancer our request is forwarded to server 2 (port:8001) Even though the next server in the sequence is server 1, the request is forwarded to server 2 because server 1 exceeds the timeout threshold of 1000ms.
In real systems, a slow server is often worse than a down server, because it blocks resources and increases latency.

In real-world systems, load balancers also handle request forwarding, connection pooling, retries, metrics, and security.

Disadvantages of Round Robin

It distributes load evenly because of which the current state of the servers are ignored.
Few servers can be big some might be small. Incase of a fast server is available it is not utilized enough

Code reference: https://github.com/Nis-chal-Jain/load_balancer_simulation