node.js - How can I hold on http calls for a while?

Question

I have the following problem and I would appreciate if someone could send me an idea, I have tried some, but it did not work. Consider the code:

 while (this.fastaSample.length > 0) {
      this.datainputService
        .saveToMongoDB(this.fastaSample.slice(0, batch))
        .subscribe();
    }

It supposes to solve the issue that I cannot send my data in a single http call, since it is too big, I was able to send 10% without issue, more than that, it does not work! So I thought, I should send smaller batches, and I have consulted sone Q&As here, and they helped me, but did not solve the problem.

I have tried to use await as I did in node, but it does not work; it sends all the http at once, it would be nice to stop/hold the code until the last http call is complete, that would be nice! Any suggestion?

Answer 1

I suppose you could make it all nice and rxjs by using from and concatAll:

_{untested code}

// first create batches by chunking the array
const batches = Array.from(
  { length: Math.ceil(fastaSample.length / batch) },
  (v, i) => fastaSample.slice(i * batch, i * batch + batch)
)

// Second go over these chunks using `from` and `concatAll`:
from(batches).pipe(
  map((batch) => this.data.inputService.saveToMongoDB(batch)),
  concatAll()
).subscribe();

This will make the calls consecutively. If it's possible to do the requests at the same time, you can do mergeAll().

---

But like @Mike commented, it seems like the issue should be handled in the MongoDB backend and accept a multipart request. This way you don't need to chunk stuff