[cmd/mdatagen] add a way to generate configuration as part of README

[atoulme]

Description:
Add a way to generate configuration as part of README.

Link to tracking Issue:
Not yet.

Testing:
Unit tests

Documentation:
N/A

[dmitryax]

This is amazing!

Just one note that we have https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/cmd/configschema, which does a similar thing. The problem with the configschema is that it takes all the components as dependencies instead of parsing the config.go files as you are doing here. I believe the parsing is the best option, but I'm not sure it should be in mdatagen. I think mdatagen should generate stuff from metadata.yaml only. It'd be great if we could change the cmd/configschema to use this approach instead unless we want to define the config in the metadata.yaml in the first place and generate both docs and config.go -- which can be a pretty good option, just need to see how feasible is that.

I also suggested this format some time ago, but I like yours as well, it's just maybe harder to copy-paste for the end users. PTAL, what do you think.

[atoulme]

Thank you, I'll take a look. This is just a skunkworks weekend thing and it needs to be tendered to in the right way. All your suggestions help.

[dmitryax]

@atoulme #23061 is a good example of how I envisioned the configuration definition in yaml format

[atoulme]

Definitely seems like a good format to me. Especially because you'd then copy/paste this into your collector config, right?

I haven't had time to get back to this code yet.

[dmitryax]

Especially because you'd then copy/paste this into your collector config, right?

Exactly

[dmitryax]

I don't like that we introduce another package doing pretty similar to what configschema is doing. Maintaining them both is not ideal. I'd be happy to replace it, but we need a plan.

One suggestion:

1. Refactor configschema to use your approach with generate pragmas -- this will help the CI time a lot!
2. Make configschema to produce the yaml format embedded into the README instead of https://github.com/open-telemetry/opentelemetry-collector-contrib/blob/main/receiver/redisreceiver/config.md

Another suggestion:

1. Define the configuration schema as yaml (probably inside metadata.yaml under config), similar to what configschema used to generate. Use it as a source of truth and generate both the config.go and README.md files.

I like the second one, but it might be a lot of work.

[atoulme]

I get it. It’s a spike and I learned something from it: we will need to do more to get default values, reading the ast tree is not enough.

Sounds to me like configschema can generate all the yaml files, and we use those to generate config and README then. Eventually configschema can be deprecated and removed once we stabilize the yaml files approach.

[github-actions]

This PR was marked stale due to lack of activity. It will be closed in 14 days.

[github-actions]

Closed as inactive. Feel free to reopen if this PR is still being worked on.