Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Not to consume buf when decoding non_huff header name(which consumes the buf) succeeded but header value failed (DecoderError::NeedMore) #589

Merged
merged 2 commits into from
Feb 24, 2022

Conversation

hikaricai
Copy link
Contributor

Decoding error when processing continuation frames

@hikaricai
Copy link
Contributor Author

Anyone?

@olix0r
Copy link
Collaborator

olix0r commented Jan 11, 2022

@hikaricai it would probably be easier to review this if we had a test that failed without this change.

At a quick glance, this is going to incur an additional (unnecessary?) allocation on each call.

@hikaricai
Copy link
Contributor Author

@olix0r thanks, I will write a test case.

It seems that the "fn consum" is used only if the decoding is success, but the "fn decode_string" may only be part of decoding. For example, decoding both header name and header value.

I met decoding failure when receiving continuation header which just spilt header name and header value.

Decoding error when processing continuation header which contains normal
header name at boundary
@hikaricai
Copy link
Contributor Author

I have just updated the commit, could anyone helps to review?

@olix0r
Copy link
Collaborator

olix0r commented Jan 14, 2022

Not sure if @carllerche has bandwidth to review, but I think he knows this code best. Also I think @seanmonstar and @nox have worked on this module ~recently. If no one's available, I'll try to take a look at this but it will take me some time to load up context.

@hikaricai hikaricai changed the title Not to consume buf in the decode_string Not to consume buf when decoding non_huff header name(which consumes the buf) successed but header value failed (DecoderError::NeedMore) Feb 8, 2022
@hikaricai hikaricai changed the title Not to consume buf when decoding non_huff header name(which consumes the buf) successed but header value failed (DecoderError::NeedMore) Not to consume buf when decoding non_huff header name(which consumes the buf) succeeded but header value failed (DecoderError::NeedMore) Feb 8, 2022
@hikaricai
Copy link
Contributor Author

Could anyone helps to review? I need this patch to fix bug in my project.

@seanmonstar
Copy link
Member

Sorry about that, I'll take a look now.

Copy link
Member

@seanmonstar seanmonstar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would you mind helping me understand what sort of situation this happens in? So, you receive a HEADERS frame, and only the name and part of the value are in it, and the rest of the value is in a CONTINUATION frame? And currently, h2 doesn't handle that?

fn try_decode_string(
&mut self,
buf: &mut Cursor<&mut BytesMut>,
) -> Result<((usize, usize), Option<Bytes>), DecoderError> {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's hard to know what these values are supposed to be. What if instead of returning a tuple, it return some struct DecodedString { .. } that named the usizes and Option<Bytes>?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for reply.

I am using tonic as grpc client and would receive a large HEADERS frame(grpc-status-details-bin) whose name is non_huff format and value is partial from C++ server. When decoding this header, the name is consumed because it is not huff, and then the value decoding is failed for DecoderError::NeedMore, and then the decode function returns with error but the buf is consumed.

I see that the CONTINUATION header currently tested in CI but both name and value are huff_encoded.

My first patch is simply replacing consume() with buf copy, and @olix0r pointed that the copy can be optimized. So i tried to save the decoding context and consume the buf only if both header name and header buf is successfully decoded.

I will update my patch to easily show the meaning of those values.

@hikaricai
Copy link
Contributor Author

@seanmonstar Is the new commit clearly enough?

The error situation is shown in the new test case

Copy link
Member

@seanmonstar seanmonstar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great now, thank you! <3

@seanmonstar seanmonstar merged commit 85549fc into hyperium:master Feb 24, 2022
hawkw added a commit that referenced this pull request Mar 9, 2022
# 0.3.12 (March 9, 2022)

* Avoid time operations that can panic (#599)
* Bump MSRV to Rust 1.49 (#606)
* Fix header decoding error when a header name is contained at a continuation
  header boundary (#589)
* Remove I/O type names from handshake `tracing` spans (#608)
@hawkw hawkw mentioned this pull request Mar 9, 2022
seanmonstar pushed a commit that referenced this pull request Mar 9, 2022
# 0.3.12 (March 9, 2022)

* Avoid time operations that can panic (#599)
* Bump MSRV to Rust 1.49 (#606)
* Fix header decoding error when a header name is contained at a continuation
  header boundary (#589)
* Remove I/O type names from handshake `tracing` spans (#608)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants