Vault UI will not display Recovery Keys or Initial Root Token if initialized with pre-configured pkcs11 backend

[MarkusBauerFS]

Describe the bug
When initializing via the UI, Vault does not display the recovery keys and initial root token when configured with a PKCS11 backend.

To Reproduce

1. Deploy Vault with the config below (pkcs11 seal stanza already filled with relevant information)
2. Initialize Vault via the UI - it does not matter which number of keys or key-shares you select.
3. Observe that Vault directly moves on to the Unsealing section and moments later to the login page (as auto-unsealing kicks in)

Expected behavior
A clear and concise description of what you expected to happen.

Environment:

• Vault Server Version (retrieve with vault status): 1.14.1+ent.hsm
• Vault CLI Version (retrieve with vault version): Vault v1.13.0 (not relevant as this is a UI problem but hey)
• Server Operating System/Architecture: Ubuntu 22_04 (x86/amd64)

Vault server configuration file(s):

cluster_addr         = "http://<node1-ip>:8201"
api_addr             = "http://<node1-ip>:8200"
disable_mlock        = true
ui                   = true
license_path         = "/etc/vault.d/vault.hclic"

log_format           = "json"
log_level            = "info"
log_file             = "/opt/vault/log/"
log_rotate_max_files = "0"


listener "tcp" {
  tls_disable         = true
  address             = "0.0.0.0:8200"
  cluster_address     = "0.0.0.0:8201"
}


seal "pkcs11" {
  lib            = "/usr/local/primus/lib/libprimusP11.so"
  slot           = "1"
  pin            = "<PKCS11 password>"
  key_label      = "HashiCorp-Vault"
  hmac_key_label = "HashiCorp_hmac"
  generate_key   = "true"
  mechanism      = 0x1087
}


storage "raft" {
  path                         = "/opt/vault/data"
  node_id                      = "<my-node-id>"
  autopilot_reconcile_interval = "10s"
  autpilot_update_interval     = "2s"
  cleanup_dead_servers         = true
  dead_server_last_contact_threshold = "24h"

  autopilot_upgrade_version    = "1.14.0.0"


  retry_join {
      leader_api_addr            = "http://<node1-ip>:8200"
  }

  retry_join {
      leader_api_addr            = "http://<node2-ip>:8200"
  }

  retry_join {
      leader_api_addr            = "http://<node3-ip>:8200"
  }
}

Additional context
The CLI will properly output the recovery keys and initial root token - so this looks like a UI problem.

[Monkeychip]

@MarkusBauerFS I'm investigating this bug, and trying to determine if it's a regression. Have you done this flow successfully using this same environment in a past Vault version and if so can you tell me what version?

[MarkusBauerFS]

@Monkeychip I don't have any notes on it, so let's assume that no I did not. I have done lots of lab experiments on 1.13.x and don't recall running into this issue but as I said, I have no notes, so I might have had it initialized manually once. I have the setup automated, so I can run the tests if you want me to.

[Monkeychip]

@MarkusBauerFS got it. We've tested in on 1.14.x and we can reproduce it, but we believe updating a related component has fixed it in 1.15.x—PR here. Unfortunately, we cannot backport this "fix" because of the number of files it changes. Is upgrading to 1.15.x a potential solution for you?

[MarkusBauerFS]

@Monkeychip Yes, that does work for me, thank you. Thank you for looking into it as well.

[Monkeychip]

This issue is fixed in 1.15.x.

[weshouman]

In my case I had to decrypt the seal keys first using base64 instead of xxd, here are a couple of ways to do that non-interactively

# From an env var
echo ${ENCRYPTED_KEY} | base64 -d | GPG_TTY=$(tty) gpg --batch --yes --pinentry-mode loopback --passphrase ${PASSPHRASE} --decrypt

# From a file
echo ${ENCRYPTED_KEY} | base64 -d | gpg --pinentry-mode loopback --passphrase-file ${PASSPHRASE_FILE} --decrypt