Concurrency in async iterators?

[everett1992]

The async iterator helpers described in this spec are serial - await it.next(), then await mapper(value), then loop.

This example from the readme would only send one fetch request at a time.

const responses = await AsyncIterator.from(urls)
  .map(async (url) => {
    const response = await fetch(url);
    return response.json();
  })
  .toArray();

This example, using aws-sdk paginators, sends one query api call then one fetch at a time for each item in the page before starting the next page query.

await responses = paginateQuery({ client }, QUERY)
  .flatMap(page => page.Items)
  .map(item => fetch(item.url.S))

Operations like this can improve throughput with concurrent execution.

Could async iterator helpers support eagerly getting the next items and buffering them for the next call?
Could the helpers call their callback functions concurrently, before waiting for their results?

This could be implemented as arguments to each helper:

await responses = paginateQuery({ client }, QUERY)
 // buffer one item from the iterator, even before it is requested. 
 .flatMap(page => page.Items, { buffer: 1})
 // allow up to four pending iterator/callback promises
 .map(item => fetch(item.url.S), { parallel: 4 }) 

Or their own helper method

await responses = paginateQuery({ client }, QUERY)
 .flatMap(page => page.Items)
 // buffer one item from the iterator, even before it is requested. 
 .buffer(1)
 .map(item => fetch(item.url.S)                    
 // allow up to four pending iterator/callback promises
 .parallel(4)                  

Parallel could support both ordered and unordered execution.

I think this request could be extended to for await loops as well

for await (const item of asyncIterator; parallel = 2, buffer = 1) { ... }

[ljharb]

That seems like something wildly out of scope of this proposal.

[bakkot]

While the idea of iterators which eagerly start work early is interesting, and is something we've talked about some before (e.g. x and the following day), it's definitely not in scope for this particular proposal - there's a lot of design space here, and it would serve a pretty different function than the helpers here.

[everett1992]

What is the process for follow up proposals? Do I have to wait for this to land or reach a certain stage before submitting a stand alone proposal?

[bakkot]

The process for contributing is documented here. No need to wait for this to advance to put something together, though do note you'll need to get someone who serves on the committee to champion it.

[bakkot]

After thinking about this more, I think that leaving the door open for a follow-on proposal here would actually require a tweak to the semantics in the helpers currently in this proposal. I've opened #262 to track.

I like the idea of a helper to eagerly pull from an iterator multiple times and buffer results, like your .buffer. Assuming we tweak the semantics so that .next calls are forwarded eagerly in helpers like .map, such a helper would be all that would be needed to get parallelism; an additional .parallel isn't actually necessary.