Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ReadStreamAsync produces an error log entry if the result stream is not enumerated #72

Closed
alexeyzimarev opened this issue Sep 4, 2020 · 10 comments

Comments

@alexeyzimarev
Copy link
Member

Originated by: https://discuss.eventstore.com/t/deadlineexceeded-errors/2737

Code to reproduce the issue:

public async Task<bool> Exists(string stream) {
    await using var result = _client.ReadStreamAsync(Direction.Forwards, stream, StreamPosition.Start, 1);
    var state = await result.ReadState;
    return state != ReadState.StreamNotFound;
}

Since we're only interested in the read result and aren't enumerating the stream, gRPC client closes the stream after timeout exceeds.

Here is the log:

warn: Grpc.Net.Client.Internal.GrpcCall[7]
      gRPC call deadline exceeded.
fail: Grpc.Net.Client.Internal.GrpcCall[3]
      Call failed with gRPC error status. Status code: 'DeadlineExceeded', Message: ''.

The code still works but under load, such an app will produce lots of false warnings and errors in the logs.

The code snippet above disposes the enumerator but it doesn't seem to help.

@alexeyzimarev
Copy link
Member Author

The same error appears in the log if I get one event from the result steam that has more events, even if I pass one as the count to the ReadEventsAsync. The event is returned but after a second or two, the same error appears in the log.

@pbn4
Copy link

pbn4 commented Jun 17, 2021

@alexeyzimarev Any updates on this?

@ThomasBoettcherHolschen

Hi, got the same issue. The problem is that when this error occurs the whole communication to the server failed.

Client:
EventStore.Client.Grpc (21.2.0).

Server:
eventstore:21.6.0-bionic Container.

I could get arround with an ugly workarround. I removed the single instance of EventStoreClient and replaced it with one instance per lifetimescope in my Autofac-DI. But I fear that the System will go down when under heavier load or running a less potent system. Maybe this helps someone.

@rmfaheem
Copy link

rmfaheem commented Jul 2, 2021

Seems to be identical to: EventStore/EventStore#2973

@tlutsyk
Copy link

tlutsyk commented Aug 30, 2021

Any updates on this?

1 similar comment
@yurii-pelekh
Copy link

Any updates on this?

@gavinburge
Copy link

Any update on this?

@ylorph
Copy link
Contributor

ylorph commented Mar 2, 2022

In 22.0 release of the client, we no longer set a deadline on Readstream
see here : #194

@ylorph ylorph closed this as completed Mar 2, 2022
@brian-douglas
Copy link

brian-douglas commented Jul 31, 2022

I see the interface was changed, and I can set deadline on ReadStreamAsync(), but the problem still exists for me.

server version: 21.10.6.0
gRPC client version: 22.0.0

confirmation it was actually fixed with #194 ? If I downgrade the client back to 21.2.0, it's working again.

@KeesSchoutenEasySystems
Copy link

KeesSchoutenEasySystems commented Nov 29, 2022

I ran into the same issue but fixed it by disposing the readstate.

The following code did not dispose the readstate in case the readstate was not an Ok response, see:
image

Fix:
image

@thefringeninja pointed me in the right direction:
https://discuss.eventstore.com/t/deadlineexceeded-errors/2737/11

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

10 participants