rpk: add missing loggers

[daisukebe]

Cover letter

This is related to the community report, #3772; rpk redpanda admin config log-level set all ... does not take account of all loggers.

I have checked what have been added and been removed since beginning via:

$ git -P log -p  | grep -e "^+.*ss::logger " -e "^-.*ss::logger "

, and added

archival-ctrl
json
metrics-reporter
offset_translator
storage-gc
tx

I've run rpk redpanda admin config log-level set --host localhost:9644 all -l trace -e 1 and confirmed the following loggers are enabled from the log.

INFO  2022-03-05 17:38:38,867 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {admin_api_server}: info -> trace
INFO  2022-03-05 17:38:38,867 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {archival}: info -> trace
INFO  2022-03-05 17:38:38,867 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {archival-ctrl}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {assert}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {cloud_storage}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {cluster}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {compaction_ctrl}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {compression}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {coproc}: info -> trace
INFO  2022-03-05 17:38:38,868 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {dns_resolver}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {exception}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {fault_injector}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {http}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {httpd}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {io}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {json}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {kafka}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {kvstore}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {metrics-reporter}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {offset_translator}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {pandaproxy}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {raft}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {redpanda::main}: info -> trace
INFO  2022-03-05 17:38:38,869 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {rpc}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {s3}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {scheduler}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {scollectd}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {seastar}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {security}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {storage}: info -> trace
INFO  2022-03-05 17:38:38,870 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {storage-gc}: info -> trace
INFO  2022-03-05 17:38:38,871 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {syschecks}: info -> trace
INFO  2022-03-05 17:38:38,871 [shard 0] admin_api_server - admin_server.cc:495 - Set log level for {tx}: info -> trace

Release notes

[jcsp]

@twmb what was the motivation for having this list hardcoded in rpk to begin with? This seems like something that's going to go stale again in future, maybe rpk should just be passing through here

[dotnwat]

@twmb what was the motivation for having this list hardcoded in rpk to begin with? This seems like something that's going to go stale again in future, maybe rpk should just be passing through here

Depending on what the answer is here, we could expose the set of available loggers via the admin http (ss::logger::logger_registry::get_all_logger_names()).

[emaxerrno]

@twmb what was the motivation for having this list hardcoded in rpk to begin with? This seems like something that's going to go stale again in future, maybe rpk should just be passing through here

Depending on what the answer is here, we could expose the set of available loggers via the admin http (ss::logger::logger_registry::get_all_logger_names()).

indeed! dynamic discovery is where is at.

and honestly, this is also a good candidate to sedt on the centralized config. (set logging on these nodes to this level for 10mins kind of query.

[twmb]

The hardcoded motivation was to avoid exec'ing redpanda to discover the list, which at the time (and still today) is the only way to discover all loggers. Perhaps rpk could be implemented to try execing, and if redpanda doesn't exist (local rpk talking to remote redpanda), fallback to the hardcoded list. But, better would be to move some of this into redpanda via http handlers.

[emaxerrno]

The hardcoded motivation was to avoid exec'ing redpanda to discover the list, which at the time (and still today) is the only way to discover all loggers. Perhaps rpk could be implemented to try execing, and if redpanda doesn't exist (local rpk talking to remote redpanda), fallback to the hardcoded list. But, better would be to move some of this into redpanda via http handlers.

I like the idea of doing both A & B