feat: Allow TriggerAuthentication defaults via ENV variables for HashiCorp Vault

[BojanZelic]

This PR sets vault defaults if specified via ENV variables. When configuring the operator, you can set ENV variables so users don't have to redefine vault settings every time they use vault authentication;

example:

hashiCorpVault:                                     # Optional.
  address: {hashicorp-vault-address}                # Optional. set via ENV: $VAULT_ADDR
  namespace: {hashicorp-vault-namespace}            # Optional. set via ENV: $VAULT_NAMESPACE
  authentication: token | kubernetes                # Optional. set via ENV: $VAULT_AUTH
  role: {hashicorp-vault-role}                      # Optional. set via ENV: $VAULT_ROLE
  mount: {hashicorp-vault-mount}                    # Optional. set via ENV: $VAULT_MOUNT
  credential:                                       # Optional.
    token: {hashicorp-vault-token}                  # Optional. set via ENV: $VAULT_TOKEN
    serviceAccount: {path-to-service-account-file}  # Optional. set via ENV: $VAULT_JWT_PATH
  secrets:                                          # Required.
  - parameter: {scaledObject-parameter-name}        # Required.
    key: {hasicorp-vault-secret-key-name}           # Required.
    path: {hasicorp-vault-secret-path}              # Required.

Checklist

• Tests have been added
• Changelog has been updated and is aligned with our changelog requirements
• A PR is opened to update the documentation on (repo) (if applicable) docs: Allow TriggerAuthentication Defaults via ENV variables for HashiCorp Vault keda-docs#1262
• Commits are signed with Developer Certificate of Origin (DCO - learn more)

Fixes # #4965

Relates to #

[github-actions]

Thank you for your contribution! 🙏 We will review your PR as soon as possible.

While you are waiting, make sure to:

• Add an entry in our changelog in alphabetical order and link related issue
• Update the documentation, if needed
• Add unit & e2e tests for your changes
• GitHub checks are passing
• Is the DCO check failing? Here is how you can fix DCO issues

Learn more about:

• Our contribution guide

[JorTurFer]

I like this feature as it helps the vault integration (majority of the use cases have a single vault).
I'm not sure about the naming of the envs (I know that they are the hashicorp envs but KEDA supports more than it), maybe adding something like HASHICORP_* prefix we can be more clear (or even more, DEFAULT_HASHICORP_*).
WDYT @tomkerkhove @zroubalik ?

Could you open PR's to helm chart and docs explaining this feature?

[zroubalik]

I like this feature as it helps the vault integration (majority of the use cases have a single vault). I'm not sure about the naming of the envs (I know that they are the hashicorp envs but KEDA supports more than it), maybe adding something like HASHICORP_* prefix we can be more clear (or even more, DEFAULT_HASHICORP_*). WDYT @tomkerkhove @zroubalik ?

Could you open PR's to helm chart and docs explaining this feature?

I do understand the problem, though I would actually prefer if we solve this in general for all config - thus having some global config for KEDA. If we start adding this kind of config via ENV variables the list would growth a lot in the future.

[tomkerkhove]

I like this feature as it helps the vault integration (majority of the use cases have a single vault). I'm not sure about the naming of the envs (I know that they are the hashicorp envs but KEDA supports more than it), maybe adding something like HASHICORP_* prefix we can be more clear (or even more, DEFAULT_HASHICORP_*). WDYT @tomkerkhove @zroubalik ?
Could you open PR's to helm chart and docs explaining this feature?

I do understand the problem, though I would actually prefer if we solve this in general for all config - thus having some global config for KEDA. If we start adding this kind of config via ENV variables the list would growth a lot in the future.

I tend to agree on this

[BojanZelic]

@zroubalik @tomkerkhove I understand that the original design might lead to a some complexity long-term.

What are your thoughts on any of the following alternatives for setting user-specified default values?

Option 1) Mutating Webhook
Extend the existing validating webhook & use a mutating-webhook to set user-configurable defaults if they're not specified?

A user could specify a scaledObject, triggerAuthentication, ect... yaml in a configfile; The mutating webhook would load this config file and set defaults for custom resources that keda manages. It would be generic & there wouldn't be any extra maintenance burden since the values would only be declared once.

example:
Keda mutating-webhook would parse this yaml file from a configmap specified on startup:

apiVersion: keda.sh/v1alpha1
kind: TriggerAuthentication
spec:
  hashiCorpVault:
    address: https://my-vault-url
    authentication: kubernetes
    credential:
      serviceAccount: /var/run/secrets/kubernetes.io/serviceaccount/token
    mount: kubernetes
    role: my-role

and set defaults whenever the TriggerAuthentication CR is created to the above values. The defaults will be persisted on the custom resource itself.

Option 2) Dynamic ENV variables
Use something like this package https://github.com/caarlos0/env to add env flags to custom resources that we wish to override; Keda would parse the ENV flags and set default values if the ENV variable is defined whenever the resources are loaded from kubernetes. The defaults wouldn't be persisted on the custom resource itself. Every value that we wish to override, would have to have an env flag set;

example:

type HashiCorpVault struct {
	Secrets []VaultSecret `json:"secrets"`
	// +optional
	Address string `json:"address" env:"DEFAULT_HASHICORP_VAULT_ADDR"`
	// +optional
	Authentication VaultAuthentication `json:"authentication" env:"DEFAULT_HASHICORP_AUTH"`
	// +optional
	Namespace string `json:"namespace,omitempty" env:"DEFAULT_HASHICORP_NAMESPACE"`
	// +optional
	Credential *Credential `json:"credential,omitempty" env:"DEFAULT_HASHICORP_CREDENTIAL"`
	// +optional
	Role string `json:"role,omitempty" env:"DEFAULT_HASHICORP_ROLE"`
	// +optional
	Mount string `json:"mount,omitempty" env:"DEFAULT_HASHICORP_MOUNT"`
}

If you have any other suggestions/approaches please let me know; I'd really like to get this feature implemented

[tomkerkhove]

I think we should discuss this on the issue and take #5229 (from @majusmisiak) in to mind as they both have similar goal - Make it easier to pull configuration from outside the scaledjob/triggerauth. In this case, envFrom or so would be better.

Created #5233 to consolidate proposals

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed in 7 days if no further activity occurs. Thank you for your contributions.

[stale]

This issue has been automatically closed due to inactivity.