Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[x] http: client destroy stream #29192

Closed
wants to merge 1 commit into from

Conversation

ronag
Copy link
Member

@ronag ronag commented Aug 18, 2019

This PR tries to re-use the destroy logic from streams. One small step towards making ClientRequest more stream-like.

Based on #28683. Since it's close to being merged I don't want to complicate it further.

Unlike #28683. Doesn't currently swallow errors. But will only emit one error after deestroy().

Refs: #28686

Checklist
  • make -j4 test (UNIX), or vcbuild test (Windows) passes
  • tests and/or benchmarks are included
  • documentation is changed or added
  • commit message follows commit guidelines

@nodejs-github-bot nodejs-github-bot added the http Issues or PRs related to the http subsystem. label Aug 18, 2019
@ronag ronag force-pushed the http-client-destroy-stream branch 2 times, most recently from 595b846 to 4a82370 Compare August 18, 2019 13:03
@ronag
Copy link
Member Author

ronag commented Aug 18, 2019

@mcollina: A little guidance here? Merge #28683 first and then have this as a continuation? Or merge this into #28683 and do another round of reviews?

@ronag ronag force-pushed the http-client-destroy-stream branch 21 times, most recently from 9223798 to f904ea8 Compare August 18, 2019 14:14
lib/_http_client.js Outdated Show resolved Hide resolved
@ronag ronag force-pushed the http-client-destroy-stream branch 4 times, most recently from 5f4f9d6 to 19425c2 Compare August 18, 2019 15:13
@ronag
Copy link
Member Author

ronag commented Sep 18, 2019

@mcollina: Any further changes?

@Trott: This needs more reviews?

@mcollina mcollina added the semver-major PRs that contain breaking changes and should be released in the next major version. label Sep 20, 2019
Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

lib/_http_client.js Show resolved Hide resolved
@mcollina
Copy link
Member

cc @nodejs/tsc

@Trott
Copy link
Member

Trott commented Sep 22, 2019

(Needs a rebase.)

@ronag
Copy link
Member Author

ronag commented Sep 23, 2019

@Trott rebased

@nodejs-github-bot
Copy link
Collaborator

Copy link
Member

@mcollina mcollina left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add some more tests about this behavior change?

test/parallel/test-http-client-set-timeout.js Show resolved Hide resolved
test/parallel/test-http-client-close-event.js Show resolved Hide resolved
lib/_http_client.js Show resolved Hide resolved
@ronag
Copy link
Member Author

ronag commented Sep 24, 2019

Can you add some more tests about this behavior change?

Not sure what that would test. I'll think about it and give it a try.

} else if (this[kWritableState].socketPending) {
// In the event that we don't have a socket, we will pop out of
// the request queue through handling in onSocket.
this[kWritableState].destroyCallback = (er) => cb(er || err);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it's better to wait for the 'socket' event to be emitted instead of adding one more state parameter: https://nodejs.org/api/http.html#http_event_socket.

Copy link
Member Author

@ronag ronag Sep 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That won't work, the socket event is not always emitted. See the error case in oncreate, i.e. close to where your other comment is.

The error from oncreate needs to be properly forwarded to the _destroy callback.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

From a user perspective, I think we should be swallowing that error if destroy() was called before hand.

Do we test that calling destroy() before we have a socket actually calls socket.destroy()?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This can be further refactored if/once #29656 is merged. But it basically does the same thing.

Copy link
Member Author

@ronag ronag Sep 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should be swallowing that error if destroy() was called before hand.

This confuses me. Isn't it a bit inconsistent with the discussion in #29197? @lpinca might have input?

Do we test that calling destroy() before we have a socket actually calls socket.destroy()?

Not sure, I will check.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we test that calling destroy() before we have a socket actually calls socket.destroy()?

Test added

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The possibility of deadlocks/having a callback not called in the destroy process makes me nervous. The current logic is significantly simpler, even if less correct, and I feel this might bite us in the future.

Copy link
Member Author

@ronag ronag Sep 25, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The possibility of deadlocks/having a callback not called in the destroy process makes me nervous.

That's the way Node works? This is not the only case? What can we do to make you less nervous? More tests?

The current logic is significantly simpler, even if less correct, and I feel this might bite us in the future.

I strongly disagree with this. In my opinion this kind of mentality will stagnate the project. Shouldn't we strive after correctness and consistency?

We have similar scenarios in a lot of different places and I have pending PR's (e.g. #29656) with efforts on this.

If this (swallow error) is the way we go then I'd like to ask #29197 to be reconsidered (to "no error after destroy") in order to achieve some form of consistency. Otherwise, we will never have "correct" code even in the future.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't we strive after correctness and consistency?

No. We should strive to not break current users, and then consistency. Correctness it's extremely hard to measure because there are no specifications for any of this.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should strive to not break current users

How does this break current users?

@ronag
Copy link
Member Author

ronag commented Sep 24, 2019

I've tried adding more test for what make sense. However, most of these changes are refactoring in nature as well as re and proper use of existing framework.

@ronag ronag force-pushed the http-client-destroy-stream branch 3 times, most recently from 1016ea1 to 09d15d5 Compare September 24, 2019 19:28
@ronag
Copy link
Member Author

ronag commented Nov 24, 2019

This PR can probably be easier implemented with #29656

@ronag
Copy link
Member Author

ronag commented Dec 15, 2019

This is blocked by #29656

@ronag
Copy link
Member Author

ronag commented Dec 29, 2019

There are several pending PR's (.e.g #30623, #29656) that affects this which means this is not ready. I will close this for now and re-open when it becomes relevant again.

@ronag ronag closed this Dec 29, 2019
@ronag ronag changed the title http: client destroy stream [x] http: client destroy stream Dec 29, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
http Issues or PRs related to the http subsystem. semver-major PRs that contain breaking changes and should be released in the next major version.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

9 participants