Enable container-to-container service communication

[danegsta]

Description

This PR implements a custom container network for an Aspire AppHost run and uses it to support direct container-to-container service communication without going through service proxies on the host. This should allow scenarios such as sidecar management containers to work with more container runtime environments. For now we're hardcoding container-to-container connections in individual sidecar resources until we design and implement an updated API for connection service discovery that can better support these scenarios.

Fixes #5584

Checklist

• Is this feature complete?
  • Yes. Ready to ship.
  • No. Follow-up changes expected.
• Are you including unit tests for the changes and scenario tests if relevant?
  • Yes
  • No
• Did you add public API?
  • Yes
    • If yes, did you have an API Review for it?
      • Yes
      • No
    • Did you add <remarks /> and <code /> elements on your triple slash comments?
      • Yes
      • No
  • No
• Does the change make any security assumptions or guarantees?
  • Yes
    • If yes, have you done a threat model and had a security review?
      • Yes
      • No
  • No
• Does the change require an update in our Aspire docs?
  • Yes
    • Link to aspire-docs issue:
  • No

Microsoft Reviewers: Open in CodeFlow

[davidfowl]

Errors here:

 Expected: myredis1:host.containers.internal:5001:0
[xUnit.net 00:00:05.16]       Actual:   myredis1:myredis1:6379:0

We need to update host names in the tests

[davidfowl]

Now the port is wrong 😄

[davidfowl]

Beyond the small changes to ports and host names, it seems like using the docker network slows down the startup of these containers or breaks logs in some way that causes tests to fail.

[mitchdenny]

I don't quite understand why these ports are changing for these tests?

[danegsta]

Container to container requires using the in-container target port instead of the public port for connections

[mitchdenny]

With the container network creation, is this something that might benefit from being persistent by default? What I'm thinking is that if we made it persistent it would allow developers to spin up containers manually and attach it to that network if they want to.

[mitchdenny]

Is now a good time to discuss Podman testing? At the moment all of our testing is done against Docker/Docker Desktop, but we say we support Podman but don't do any validation in our CI.

It shouldn't block our PR, I'll raise an issue for it specifically.

[danegsta]

With the container network creation, is this something that might benefit from being persistent by default? What I'm thinking is that if we made it persistent it would allow developers to spin up containers manually and attach it to that network if they want to.

Long term I think it may be cleaner to work out how we want to expose container networks as a primitive in the resource builder API and allow users to configure their networks (including persistent networks to support this container attach scenario) that way.

[danegsta]

My main concern is that the V1 approach in this PR is taking an extremely naive approach that depends on a lot of assumptions that could fall apart if we allow non-Aspire managed containers to attach to the network. Depending on tightly managed AppHost scoped networks makes the assumptions a bit more reasonable.

[davidfowl]

Other failures are gone! Thanks for the bug fix in dcp @danegsta!

@radical the failure is:

docker command 'CreateContainer' returned with non-zero exit code 1: command output: Stdout: '' Stderr: 'Unable to find image 'netaspireci.azurecr.io/library/mongo-express:1.0' locally\nError response from daemon: manifest for netaspireci.azurecr.io/library/mongo-express:1.0 not found: manifest unknown: manifest tagged by "1.0" is not found

How do we get container images on our test feed?

[radical]

@radical the failure is:

docker command 'CreateContainer' returned with non-zero exit code 1: command output: Stdout: '' Stderr: 'Unable to find image 'netaspireci.azurecr.io/library/mongo-express:1.0' locally\nError response from daemon: manifest for netaspireci.azurecr.io/library/mongo-express:1.0 not found: manifest unknown: manifest tagged by "1.0" is not found

I can add this one.

How do we get container images on our test feed?

https://github.com/radical/aspire/blob/tests-run-scripts/docs/updating-container-registry.md

[radical]

/azp run

[azure-pipelines]

Azure Pipelines successfully started running 1 pipeline(s).