Gateway Discovery for DB mode

[mflendrich]

Is there an existing issue for this?

• I have searched the existing issues

Problem Statement

GD (#702) today only works with DB-less mode (Kong/docs.konghq.com#5602)

Proposed Solution

Solve for DB-mode as well.

Additional information

No response

Acceptance Criteria

• All KIC deployment topologies (before this issue - only DB-less; after this issue - DB mode as well) support Gateway Discovery
• By our judgment, the sidecar model is ready for deprecation altogether

[randmonkey]

As a previous issue #3606, we could simply discover all clients and choose one to call admin APIs to configure each time when we sync the configuration.
While it is possible that multiple Kong gateway instances uses different DBs. Should we consider such situation?

[rainest]

Were we looking to achieve anything functionally different from normal DB mode with this? This feels like it needs more clarification on the background of why we want to do this and the end functionality goals. Discovery is a suite of several pieces of controller functionality, and not all are relevant for DB mode.

Discovery mode (functionally, enumerating all Kong instances in a Deployment and sending configuration to each of them) has no obvious way to fit into DB mode, where you must send configuration to one replica only, and the then replica handles distribution on the Kong side via the DB's broadcast mechanism.

You can already run a controller Deployment separate from the proxy Deployment in DB mode; it can send its configuration requests through the Service for its. There wasn't ever any need for the sidecar approach for DB mode, it was just simpler to use the same Deployment template as the DB-less default.

We do need to determine which mode to use, and ideally don't need configuration to toggle between them, but that can be handled by talking to the Service initially, checking the database mode, and then configuring the parser to either broadcast to all endpoints or to unicast to the Service.

One ancillary concern is that separate controller Deployments remove the ability to shield the admin API with a localhost-only listen. The overall discovery design does include automatic mTLS configuration, and we probably want that for default separate controller and proxy Deployments.

IIRC there's also some component of discovery used for Konnect integration to list individual replicas. I'm unsure if we support Konnect+KIC+DB mode and if we wanted to port that.

[rainest]

From discussion this morning:

• Our main goal is that separate KIC and proxy Deployments work with a DB-backed Kong instance.
• We do not need the actual endpoint enumeration and broadcast aspects of discovery. Unicast to the admin Service is fine.

We should reconcile the --kong-admin-svc and --kong-admin-url settings, --kong-admin-svc was originally added for discovery, but probably should be the standard configuration option now. We can build the admin API URL from the Service under normal circumstances, and it's one less config discrepancy between architectures users need to worry about. --kong-admin-url should remain for less typical configurations, e.g. where the the admin API is behind a path on the proxy or where the admin API actually resides outside the controller cluster (I have encountered the latter, but it still strikes me as incredibly bizarre).

As the admin API will no longer have security by local network restriction, we should also configure mTLS in these DB configurations.