In this article, we will explore the Token Bucket algorithm and its implementation in Node.js for API rate limiting, in very simple terms. Rate limiting is what stops a single client, or a burst of traffic, from overwhelming your server, and the token bucket is one of the most widely used ways to do it.
What Is the Token Bucket Algorithm?
Token Bucket is an algorithm used to limit how much of your resources or server capacity a client can consume. The idea is that your server holds a finite number of tokens, and every incoming request spends one token to be fulfilled. When there are more requests than available tokens, the server either denies or holds the extra requests until the tokens are refilled.
The tokens can be refilled in a few different ways. Two common ones are:
- Returning the token back to the server whenever a request is fulfilled.
- Refilling the tokens after a specific interval of time.
The second approach, refilling on a fixed interval, is the one we will build here.
Implementing Token Bucket in Node.js
Step 1: Set Up the Project
Create a new npm project and set "type": "module" in your package.json so you can use ES module imports.
Step 2: Install Express
Step 3: Create a Basic Express Server
Create a file called index.js and write a basic Express server in it.
Now we will rate limit the requests to /. We will write a middleware to achieve this.
Step 4: Write the Rate Limiter Middleware
Create a new file called rateLimiter.js, write the middleware function skeleton, and export it.
Next, add the tokens and a time reference. Here tokens is the array of tokens a user can spend to make a request, and time stores the current time in milliseconds.
Now add a simple if/else. If a request comes in and the tokens array still has a token, a token is popped and the request is allowed. Otherwise the request is denied.
Once the tokens array becomes empty, we need to refill it. Here we refill back to 5 tokens once 20000 milliseconds (20 seconds) have passed.
Step 5: The Refill Problem
This code has a subtle bug. The time variable is initialized the moment the server starts. Suppose the first request only arrives 20 seconds later. With the logic above you could then make 10 requests inside a single 20 second window, because after the first 5 tokens are spent, the refill branch runs immediately and tops the array back up to 5. But what we actually want is only 5 requests per 20 seconds.
Step 6: Fixing the Refill Logic
A small change to the outer if condition fixes it. We require both that a token is available and that we are still inside the 20 second window. Only once the window has elapsed do we refill.
Now a request is served only if there is a token and the current window has not yet passed 20 seconds. Once the difference exceeds 20 seconds, the array is refilled if it holds fewer than 5 tokens.
Step 7: Attach the Middleware to the Route
Finally, plug the middleware into the route.
The Final Code
Here is the complete implementation across the two files.
Conclusion
That is a working token bucket rate limiter in Node.js. From here you can customize the bucket size and the refill interval to match the traffic your API is meant to handle. Once your endpoints enforce limits like this, it is worth writing tests that confirm the behavior holds under load, which is exactly the kind of thing automated API testing is good at. If you have any questions, leave a comment and I will try to help.
Frequently Asked Questions
What is the Token Bucket algorithm?
The Token Bucket algorithm is used for rate limiting to control the consumption of resources or server usage. It maintains a bucket of tokens, where each token represents permission to perform a specific action or request.
How does the Token Bucket algorithm work?
Requests consume tokens from the bucket. When the bucket is empty, further requests are either denied or queued until tokens are refilled. Tokens can be refilled at a fixed rate or based on certain conditions, such as after a set time interval.
What is the purpose of rate limiting in APIs?
Rate limiting prevents abuse or overload of APIs by restricting the number of requests a client can make within a given timeframe. It ensures fair usage of resources and protects the server from being overwhelmed.
How can I change how many requests are allowed?
You control the limit through two values: the number of tokens the bucket starts with and the length of the refill window. In this example, five tokens and a 20 second window allow five requests every 20 seconds. Adjust either value to loosen or tighten the limit.
