[admin] Get leaders info request

[VadimPlh]

Cover letter

This debug API to understand what current node thinks about leaders.

Because it is debug request we only execute it on one core, without merging ans.

"path": "/v1/debug/partition_leaders_table",
            "operations": [
                {
                    "method": "GET",
                    "summary": "Get information about leaders for node",

Release notes

• New admin api request /v1/debug/partition_leaders_table

[dotnwat]

everything looks good, but I think it's worth discussing if this needs to go under the /debug path. this information is largely available already via Kafka's MetadataRequest so it is effectively already not merely debug-level information and could live at a more first-class path (if it isn't all already available via Kafka API).

just as a simple example (we don't need to do this exactly), GET /v1/partitions returns nearly the exact set of information for local partitions, but could be updated to return everything for GET /v1/partitions?query=all or GET /v1/cluster/partitions.

wdyt?

EDIT: I was thinking that Kafka's MetadataRequest didn't return the leader info for an all-topics request but @mmaslankaprv showed me where that's happening. That request uses the leader table info. So it may be sufficient for this case.

[jcsp]

I think there's value in having a debug endpoint that is specifically peeking at a particular data structure, asking "who does the leaders table think is leader?" rather than the more generic "who is leader?" that might be answered different ways, such as asking the local raft groups. In that context, maybe the endpoint should be /v1/debug/partition_leaders_table to reflect that it's really more of a direct data structure dump

[VadimPlh]

In that context, maybe the endpoint should be /v1/debug/partition_leaders_table to reflect that it's really more of a direct data structure dump

I like it. It is more clearly what will be returned

[VadimPlh]

everything looks good, but I think it's worth discussing if this needs to go under the /debug

I think it should be in /debug path.
It contains some internal info, as I understand. User should use kafka's MetadataRequestto get info about leaders IMO.

[dotnwat]

It contains some internal info

This is true of many admin APIs. Maybe it's worth while describing some of the options for discussion on slack? The only reason I'm paying this so much attention (I mean, /debug really is fine, too) is that we will always be adding admin APIs. And if we don't have some sort of design approach to the API then it's going to become messy.

EDIT:

As @jcsp says I think there's value in having a debug endpoint that is specifically peeking at a particular data structure, asking "who does the leaders table think is leader?" I think that is a reasonable way to characterize our approach to /debug. Although, I think it will be a fine line in some ways. For example /v1/partitions is primarily a dump of the partition table. But fair point that /debug is much less governed by nice-ness of API and geared more towards raw internal state even if a 'nice' API also is :)

[dotnwat]

lgtm. there is a field that is set twice in the admin server but other than that 👍

[dotnwat]

@VadimPlh what is previous leader and why is it exposed?

[VadimPlh]

It is from partition_leaders_table::leader_meta:

        // previous leader id, this is empty if and only if there were no leader
        // elected for the topic before

Do we have problems with this?

[dotnwat]

no it's ok, just surprising that we track it and that we want to expose it through http. thanks for explaining.