-
Notifications
You must be signed in to change notification settings - Fork 34
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Making it based on generators makes it impossible to poll values in parallel for iterators that support it #128
Comments
It would be nice if That said, I think your example is doable by hacking around the problem somewhat, and having asyncMap synchronously return the promise rather than await it to be resolved, & awaiting/filtering latter:
This idea of
Gross. That would be unfortunate. |
this is definitely out of scope for this proposal (i wouldn't be against this kind of functionality existing in the language though) |
@devsnek is there a reason we want to base this proposal on generators? it feels like it was convenience rather than cause. i don't see that a re-implementation off generators would be challenging. it would be very nice to not have these iteration helpers be so constrained. |
the problem this issue is discussing is out of scope. the implementation of the methods in this proposal have nothing to do with that. |
i don't understand your assertion that the the problem is out of scope. how so? iteration-helpers describes itself as:
surely we want to do a good job making sure these iteration helpers work in asynchronous use cases? why would we exclude asynchronous operations from iteration-helpers functioning? edit: as i think this over some, i am becoming somewhat sympathetic to this being chalked up as an async-iteration issue, not a helpers issue. my second psuedocode example, of creating trying to work with the constraints we have, perhaps we need to move parallelization out of the middle of the pipeline:
|
As the one starting the original conversation I'd like to add that That said there are still plenty of options to introduce parallel processing into iterators, e.g. a |
This came up in someone proposing an alternate formulation of my promise scheduling proposal. (H/T @Jonasdoubleyou for the suggestion that would require this)
Async generators enforce synchrony:
However, some cases, like stateless
.map
and.filter
transforms, could work with an async iterator that allows being read concurrently, and the spec doesn't exactly write that out as a possibility - it's just a natural part of the contract with how async iterables are enumerated.The hypothetical operator here is
AsyncIterator.prototype.parallel
, and it would allow controlled parallel iteration of the async iterable, and this would come in handy with things like enumerating files in a glob stream, breaking once one of them reads a file. Think of it like this pseudocode:I'm personally neutral-negative on the idea, but I can see the value of doing it this way, and I feel that it should be addressed whether you all want to leave the door open to this or not.
The text was updated successfully, but these errors were encountered: