TCP listener handler does not disconnect on EOF (netcat -N)

[fungs]

Things to check first

• I have searched the existing issues and didn't find my bug already reported there

• I have checked that my bug is still present in the latest release

AnyIO version

4.3.0

Python version

3.11.8

What happened?

While trying to implement a Python TCP receiver for a line based protocol using anyio, I have issues closing the stream with an EOF on the client side with netcat. This works with a pure Python synchronous implementation. The code uses the anyio backend.

How can we reproduce the bug?

import anyio
from anyio.streams.text import TextReceiveStream

async def read_lines(port: int):
    """Read lines from TCP socket and output."""
    
    # FIXME: GNU netcat does not deterministically shutdown after sending EOF (-N option), one has to use -w 0 to close on idle
    async def handle(client):
        async with client:
            receive_stream = TextReceiveStream(client)
            async for chunk in receive_stream:
                for line in chunk.splitlines():
                    print(f"Line: {line.rstrip()}")
    
    listener = await anyio.create_tcp_listener(local_port=port)
    await listener.serve(handle)


anyio.run(read_lines, 1234)

Now, the following command hangs on most, but not all executions (it might have to do with buffering):
echo 'hello world' | netcat -N localhost 1234

The following works, but uses an idle workaround
echo 'hello world' | netcat -N -w 0 localhost 1234

[agronholm]

For reference, what does the synchronous Python server implementation look like?

[agronholm]

Curiously, this works fine with the trio backend.

[agronholm]

Ok, I figured it out. Strange that this hasn't come up before. What happens here is that AnyIO's stream protocol signals about the EOF simply by setting the receive event, but if there's also data in the buffer, the receive() call just reads the data and clears the flag, thus erasing any evidence of the EOF condition.

[agronholm]

I now have a regression test that fails on asyncio and passes on trio.

[fungs]

Thanks @agronholm, that was way too fast for me to answer or provide more code :) I think it is not relevant anymore, but before I was using a more basic example without the TextReceiveStream wrapper, like in the documentation. It had the same issue.

[agronholm]

Could you review the PR that fixes it? That would help.

[fungs]

Could you review the PR that fixes it? That would help.

Did that, and adjusted the example in case somebody does copy-paste, because TextReceiveStream does iterate lines.