Javascript Throttling

Hello everyone! This post will be about a parallel computing trick I had to use.

So I needed to limit the number of requests which are fired towards a third party API at the same time because of low threshold for DDoS there. Requests were sent from several places but both written in typescript so that is what I am going to use here.

TL;DR;
Here is the final function if you just want to copy and paste this for typescript:

async function throttleAllAsync(
    maxParallelism: number,
    promiseProducers: (() => Promise<any>)[]) {
    const activePromises = promiseProducers
        .slice(0, maxParallelism - 1)
        .map(p => p());
    for (let i = maxParallelism - 1; i < promiseProducers.length; i++) {
        const activePromise: Promise<any> = promiseProducers[i]().then(() =>
            activePromises.splice(activePromises.indexOf(activePromise)));
        activePromises.push(activePromise);
        if (activePromises.length >= maxParallelism) {
            await Promise.race(activePromises);
        }
    }

    await Promise.all(activePromises);
}

Or this for javascript:

async function throttleAllAsync(maxParallelism, promiseProducers) {
    const activePromises = promiseProducers
        .slice(0, maxParallelism - 1)
        .map(p => p());
    for (let i = maxParallelism - 1; i < promiseProducers.length; i++) {
        const activePromise = promiseProducers[i]()
            .slice(0, maxParallelism - 1)
            .map(p => p());
        activePromises.push(activePromise);
        if (activePromises.length >= maxParallelism) {
            await Promise.race(activePromises);
        }
    }

    await Promise.all(activePromises);
}

You are still here? Good! Let’s retrace the steps I did while creating this function.

Keep in mind, I have only been doing js/ts for a couple of months, so a lot of things I just had to figure out from scratch.

First I needed a way to run all promises. That’s easy!

Promise.all([]);

And then the target function signature is

async function throttleAllAsync(
    maxParallelism: number,
    promises: Promise<any>[])

Unfortunately, there is no way to configure the maximum number of promises being currently executed for Promise.all. On top of that, as soon as you create a promise it already starts execution. The only thing that await or Promise.all do is that they wait for the result to be available. They have nothing to do with the actual execution.

With this in mind let’s change the signature to something which will not run the promises before we need it.

async function throttleAllAsync(
    maxParallelism: number,
    promiseProducers: (() => Promise<any>)[])

Here. Now the caller has to pass lambda’s or functions which create promises. Now. Let’s use Promise.all but only for a slice of the array.

async function throttleAllAsync(
    maxParallelism: number,
    promiseProducers: (() => Promise<any>)[]) {
    const activePromises = promiseProducers
        .slice(0, maxParallelism)
        .map(p => p());
    await Promise.all(activePromises);
}

This will run first maxParallelism of promises. Let’s make it so it runs all of them eventually.

async function throttleAllAsync(
    maxParallelism: number,
    promiseProducers: (() => Promise<any>)[]) {
    let i = 0;
    while (i < promiseProducers.length) {
        const activePromises = promiseProducers
            .slice(
                i,
                Math.min(promiseProducers.length - i, maxParallelism))
            .map(p => p());
        await Promise.all(activePromises);
    }
}

This is good but there is one issue. It doesn’t always run maxParallelism of promises. As soon as one finishes, then one less are running at the same time. We would want to be a bit more effective. But how do we add a promise as soon as one of them finishes? Well, for that we are going to use

Promise.race([]);

This function await till any one of the promises finishes. So we can wait for any one to finish and then add a new one to the array. We also need to remove the finished one. We can do this by using then on a promise to then also remove it.

async function throttleAllAsync(
    maxParallelism: number,
    promiseProducers: (() => Promise<any>)[]) {
    const activePromises = promiseProducers
        .slice(0, maxParallelism - 1)
        .map(p => p());
    for (let i = maxParallelism - 1; i < promiseProducers.length; i++) {
        const activePromise: Promise<any> = promiseProducers[i]().then(() =>
            activePromises.splice(activePromises.indexOf(activePromise)));
        activePromises.push(activePromise);
        await Promise.race(activePromises);
    }

    while (activePromises.length) {
        await Promise.race(activePromises);
    }
}

We are almost there! There are still two thing to improve here. Firstly, we don’t need to race them one by one at the end, we can just use the Promise.all that we saw earlier. Secondly, though Promise.race waits for any one promise to finish, several can in fact finish before we add more to the pool.

Including these two ideas brings us to the final code!

async function throttleAllAsync(
    maxParallelism: number,
    promiseProducers: (() => Promise<any>)[]) {
    const activePromises = promiseProducers
        .slice(0, maxParallelism - 1)
        .map(p => p());
    for (let i = maxParallelism - 1; i < promiseProducers.length; i++) {
        const activePromise: Promise<any> = promiseProducers[i]().then(() =>
            activePromises.splice(activePromises.indexOf(activePromise)));
        activePromises.push(activePromise);
        if (activePromises.length >= maxParallelism) {
            await Promise.race(activePromises);
        }
    }

    await Promise.all(activePromises);
}

Here!
Now we can run requests in parallel but no more then maxParallelism at a time. Thank you for reading and see you next time!