Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[QUIC] Fixed disabled receives if buffer gets drained before RECEIVE returns #72439

Merged
merged 2 commits into from
Jul 19, 2022
Merged

[QUIC] Fixed disabled receives if buffer gets drained before RECEIVE returns #72439

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
83ec947
Fixed disabled receives if buffer gets drained before RECEIVE returns
ManickaP Jul 19, 2022
9f9c4aa
Feedback
ManickaP Jul 19, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions src/libraries/System.Net.Quic/src/System/Net/Quic/Internal/ReceiveBuffers.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ public void SetFinal()
}
}

public bool HasCapacity()
{
lock (_syncRoot)
{
return _buffer.ActiveMemory.Length < MaxBufferedBytes;
}
}

public int CopyFrom(ReadOnlySpan<QUIC_BUFFER> quicBuffers, int totalLength, bool final)
{
lock (_syncRoot)
Expand Down
2 changes: 1 addition & 1 deletion src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -509,7 +509,7 @@ private unsafe int HandleEventReceive(ref RECEIVE data)
_receiveTcs.TrySetResult();

data.TotalBufferLength = totalCopied;
return QUIC_STATUS_SUCCESS;
return _receiveBuffers.HasCapacity() ? QUIC_STATUS_CONTINUE : QUIC_STATUS_SUCCESS;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What will happen if the buffers are consumed from a read thread just a moment after you check HasCapasity? Will we still have - however small but still - a probability of a deadlock (if I understood your description correctly)?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The read thread will see _receivedNeedsEnable is true and call StreamReceiveSetEnabled. It'll get queued in msquic and processed after the RECEIVE event finished, re-enabling the reads.

runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs

Lines 310 to 318 in 0b90ae4

if (totalCopied > 0 && Interlocked.CompareExchange(ref _receivedNeedsEnable, 0, 1) == 1)
{
unsafe
{
ThrowHelper.ThrowIfMsQuicError(MsQuicApi.Api.ApiTable->StreamReceiveSetEnabled(
_handle.QuicHandle,
1),
"StreamReceivedSetEnabled failed");
}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. The (initial) problem happens when this line on Read thread

runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs

Line 310 in c6f538e

if (totalCopied > 0 && Interlocked.CompareExchange(ref _receivedNeedsEnable, 0, 1) == 1)

executes earlier than this line on MsQuic thread

runtime/src/libraries/System.Net.Quic/src/System/Net/Quic/QuicStream.cs

Line 506 in 83ec947

Volatile.Write(ref _receivedNeedsEnable, 1);

meaning we are able to catch that situation any time after Volatile.Write, including on return.

I have one more question:
Is it ok when we both return CONTINUE and also schedule EnableReceive afterwards? What do you think about including the same CompareExchange here as well:

Suggested change
return _receiveBuffers.HasCapacity() ? QUIC_STATUS_CONTINUE : QUIC_STATUS_SUCCESS;
return (_receiveBuffers.HasCapacity() && Interlocked.CompareExchange(ref _receivedNeedsEnable, 0, 1) == 1) ? QUIC_STATUS_CONTINUE : QUIC_STATUS_SUCCESS;

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmmm, let me check if that works without breaking anything.
That would prevent queuing StreamReceiveSetEnabled if we've already re-enabled receives via CONTINUE status --> -1 mindless P/Invoke.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So far, looks good, I'll update the PR.

}
private unsafe int HandleEventSendComplete(ref SEND_COMPLETE data)
{
Expand Down