Why Is My Array Of Promises Running Before Calling Promise.all()?

I am trying to create an array of Promises, then resolve them with Promise.all(). I am using got, which returns a promise. My code works, but I don't fully understand how. Here it

Solution 1:

You're not starting them twice. Promises start running as soon as they're created - or as soon as the JS engine finds enough resources to start them. You have no control on when they actually start.

All Promise.all() does is wait for all of them to settle (resolve or reject). Promise.all() does not interfere with nor influence the order/timing of execution of the promise itself.

Solution 2:

Promises don't run at all. They are simply a notification system for communicating when asynchronous operations are complete.

So, as soon as you ran this:

promiseArray.push(got(url + param));

Your asynchronous operation inside of got() is already started and when it finishes, it will communicate that back through the promise.

All Promise.all() does is monitor all the promises and tell you when the first one rejects or when all of them have completed successfully. It does not "control" the async operations in any way. Instead, you start the async operations and they communicate back through the promises. You control when you started the async operations and the async operations then run themselves from then on.

---

If you break down your code a bit into pieces, here's what happens in each piece:

let promiseArray = [];
for (param of params) {
    promiseArray.push(got(url + param));
}

This calls got() a bunch of times starting whatever async operation is in that function. got() presumably returns a promise object which is then put into your promiseArray. So, at this point, the async operations are all started already and running on their own.

// Inspect the promises
for (promise of promiseArray) {
    console.log(JSON.stringify(promise));
    // Output: promise: {"_pending":true,"_canceled":false,"_promise":{}}
}

This loop, just looks at all the promises to see if any of them might already be resolved, though one would not expect them to be because their underlying async operations were just started in the prior loop.

Promise.all(promiseArray).then((results) => {
     // Operate on results - works just fine
}).catch((e) => {
    // Error handling logic
});

Then, with Promise.all(), you're just asking to monitor the array of promises so it will tell you when either there's a rejected promise or when all of them complete successfully.

Solution 3:

Promises "start" when they are created, i.e. the function that gives you the promise, has already launched the (often asynchronous) operations that will eventually lead into an asynchronous result. For instance, if a function returns a promise for a result of an HTTP request, it has already launched that HTTP request when returning you the promise object.

No matter what you do or not do with that promise object, that function (got) has already created a callback function which it passed on to an asynchronous API, such as a HTTP Request/Response API. In that callback function (which you do not see unless you inspect the source of got) the promise will be resolved as soon as it gets called back by that API. In the HTTP request example, the API calls that particular callback with the HTTP response, and the said callback function then resolve the promise.

Given all this, it is a bit strange to think of promises as things that "start" or "run". They are merely created in a pending state. The remaining thing is a pending callback from some API that will hopefully occur and then will change the state of the promise object, triggering then callbacks.

Solution 4:

Please note that fetching an array of urls with Promise.all has got some possible problems:

1. If any of the urls fail to fetch your resolve is never called (so one fails and your resolve function is never called.
2. If your array is very large you will clobber the site and your network with requests, you may want to throttle the maximum open requests and or requests made in a certain timeframe.

The first problem is easily solved, you process the failed requests and add them to the results. In the resolve handler you can decide what to do with the failed requests:

const got = require('got');

const url = 'myUrl';
const params = ['param1', 'param2', 'param3'];

constFail = function(details){this.details = details;};
Promise.all(
  params.map(
    param =>got(url + param)
      .then(
        x=>x,//if resolved just pass along the valuereject=>newFail([reject,url+param])
      )
  )
).then((results) => {
  const successes = results.filter(result=>(result && result.constructor)!==Fail),
  const failedItems = results.filter(result=>(result && result.constructor)===Fail);
}).catch((e) => {
    // Error handling logic
});

Point 2 is a bit more complicated, throttling can be done with this helper function and would look something like this:

... other code
const max5 = throttle(5);
Promise.all(
  params.map(
    param =>max5(got)(url + param)
      .then(
        x=>x,//if resulved just pass along the valuereject=>newFail([reject,url+param])
      )
  )
)