Skip to content
This repository has been archived by the owner on Jun 2, 2022. It is now read-only.

support encrypting/decrypting non-yaml files used with --set-file #93

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

support encrypting/decrypting non-yaml files used with --set-file #93

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 15 additions & 15 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,24 +103,24 @@ Available Commands:
view Print secrets decrypted
edit Edit secrets file and encrypt afterwards
clean Remove all decrypted files in specified directory (recursively)
install wrapper that decrypts secrets[.*].yaml files before running helm install
upgrade wrapper that decrypts secrets[.*].yaml files before running helm upgrade
lint wrapper that decrypts secrets[.*].yaml files before running helm lint
diff wrapper that decrypts secrets[.*].yaml files before running helm diff
install wrapper that decrypts secrets[.*] files before running helm install
upgrade wrapper that decrypts secrets[.*] files before running helm upgrade
lint wrapper that decrypts secrets[.*] files before running helm lint
diff wrapper that decrypts secrets[.*] files before running helm diff
(diff is a helm plugin)
```

By convention, files containing secrets are named `secrets.yaml`, or anything beginning with "secrets." and ending with ".yaml". E.g. `secrets.test.yaml` and `secrets.prod.yaml`.
By convention, files containing secrets are named `secrets.*`. Anything ending with ".yaml" will be treated as a yaml file and encrypted as a tree rather than a blob. E.g. `secrets.test.yaml` and `secrets.prod.yaml`.

Decrypted files have the suffix ".yaml.dec" by default. This can be changed using the `HELM_SECRETS_DEC_SUFFIX` environment variable.
Decrypted files have the suffix ".dec" by default. This can be changed using the `HELM_SECRETS_DEC_SUFFIX` environment variable.

#### Basic commands:
```
enc Encrypt secrets file
dec Decrypt secrets file
view Print decrypted secrets file
edit Edit secrets file (decrypt before and encrypt after)
clean Delete *.yaml-dec files in directory (recursively)
clean Delete *.dec files in directory (recursively)
```
Each of these commands have their own help.

Expand All @@ -132,7 +132,7 @@ Note: You need to run `gpg --import example/pgp/project{x,y}.asc` in order to su

##### Decrypt

The decrypt operation decrypts a secrets.yaml file and saves the decrypted result in secrets.yaml.dec:
The decrypt operation decrypts a secrets.* file and saves the decrypted result in secrets.*.dec:
```
$ helm secrets dec example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Decrypting example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Expand All @@ -152,17 +152,17 @@ example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml.dec is newer

##### Encrypt

The encrypt operation encrypts a secrets.yaml.dec file and saves the encrypted result in secrets.yaml:
The encrypt operation encrypts a secrets.*.dec file and saves the encrypted result in secrets.*:

If you initially have an unencrypted secrets.yaml file, it will be used as input and will be overwritten:
If you initially have an unencrypted secrets.* file, it will be used as input and will be overwritten:

```
$ helm secrets enc example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Encrypting example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Encrypted example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
```

If you already have an encrypted secrets.yaml file and a decrypted secrets.yaml.dec file, encrypting will encrypt secrets.yaml.dec to secrets.yaml:
If you already have an encrypted secrets.* file and a decrypted secrets.*.dec file, encrypting will encrypt secrets.*.dec to secrets.*:
```
$ helm secrets dec example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Decrypting example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Expand All @@ -171,13 +171,13 @@ Encrypting example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Encrypted example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml.dec to example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
```
##### View
The view operation decrypts secrets.yaml and prints it to stdout:
The view operation decrypts secrets.* and prints it to stdout:
```
$ helm secrets view example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
secret_sandbox_projectx: secret_foo_123
```
##### Edit
The edit operation will decrypt the secrets.yaml file and open it in an editor. If the file is modified, it will be encrypted again after you exit the editor.
The edit operation will decrypt the secrets.* file and open it in an editor. If the file is modified, it will be encrypted again after you exit the editor.

```
$ helm secrets edit example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml
Expand All @@ -193,13 +193,13 @@ $ helm secrets clean example/helm_vars/projectX/sandbox/us-east-1/java-app/
removed example/helm_vars/projectX/sandbox/us-east-1/java-app/secrets.yaml.dec
```

If you use git there is commit hook that prevents commiting decrypted files and you can add all *.yaml.dec files in you repository ```.gitignore``` file.
If you use git there is commit hook that prevents commiting decrypted files and you can add all *.dec files in you repository ```.gitignore``` file.

#### Summary

* Values/Secrets data are not a part of the chart. You need to manage your values, public charts contains mostly defaults without secrets - data vs code
* To use the helm-secrets plugin you should build your ```.sops.yaml``` rules to make everything automatic
* Use helm secrets <enc|dec|view|edit> for everyday work with you secret yaml files
* Use helm secrets <enc|dec|view|edit> for everyday work with you secret files
* Use version control systems like GIT to work in teams and get history of versions
* Everyday search keys is simple even with encrypted files or decrypt on-the-fly with git diff config included
* With example helm_vars you can manage multiple world locations with multiple projects that contain multiple environments
Expand Down
71 changes: 52 additions & 19 deletions secrets.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

# The suffix to use for decrypted files. The default can be overridden using
# the HELM_SECRETS_DEC_SUFFIX environment variable.
DEC_SUFFIX="${HELM_SECRETS_DEC_SUFFIX:-.yaml.dec}"
DEC_SUFFIX="${HELM_SECRETS_DEC_SUFFIX:-.dec}"

# Make sure HELM_BIN is set (normally by the helm command)
HELM_BIN="${HELM_BIN:-helm}"
Expand Down Expand Up @@ -51,9 +51,10 @@ enc_usage() {
cat <<EOF
Encrypt secrets

It uses your gpg credentials to encrypt .yaml file. If the file is already
encrypted, look for a decrypted ${DEC_SUFFIX} file and encrypt that to .yaml.
It uses your gpg credentials to encrypt a file. If the file is already
encrypted, look for a decrypted ${DEC_SUFFIX} file and encrypt that.
This allows you to first decrypt the file, edit it, then encrypt it again.
Recognizes *.yaml extensions and to encrypt as a tree instead of a blob.

You can use plain sops to encrypt - https://github.com/mozilla/sops

Expand All @@ -70,7 +71,7 @@ dec_usage() {
cat <<EOF
Decrypt secrets

It uses your gpg credentials to decrypt previously encrypted .yaml file.
It uses your gpg credentials to decrypt a previously encrypted file.
Produces ${DEC_SUFFIX} file.

You can use plain sops to decrypt specific files - https://github.com/mozilla/sops
Expand All @@ -87,7 +88,7 @@ EOF

view_usage() {
cat <<EOF
View specified secrets[.*].yaml file
View specified secrets[.*] file

Example:
$ ${HELM_BIN} secrets view <SECRET_FILE_PATH>
Expand Down Expand Up @@ -134,7 +135,7 @@ install_usage() {
Install a chart

This is a wrapper for the "helm install" command. It will detect -f and
--values options, and decrypt any secrets.*.yaml files before running "helm
--values options, and decrypt any secrets.* files before running "helm
install".

Example:
Expand All @@ -151,7 +152,7 @@ upgrade_usage() {
Upgrade a deployed release

This is a wrapper for the "helm upgrade" command. It will detect -f and
--values options, and decrypt any secrets.*.yaml files before running "helm
--values options, and decrypt any secrets.* files before running "helm
upgrade".

Example:
Expand All @@ -167,9 +168,9 @@ lint_usage() {
cat <<EOF
Run helm lint on a chart

This is a wrapper for the "helm lint" command. It will detect -f and
--values options, and decrypt any secrets.*.yaml files before running "helm
lint".
This is a wrapper for the "helm lint" command. It will detect -f,
--values, and --set-files options, and decrypt any secrets.* files before
running "helm lint".

Example:
$ ${HELM_BIN} secrets lint <HELM LINT OPTIONS>
Expand All @@ -185,8 +186,8 @@ diff_usage() {
Run helm diff on a chart

"diff" is a helm plugin. This is a wrapper for the "helm diff" command. It
will detect -f and --values options, and decrypt any secrets.*.yaml files
before running "helm diff".
will detect -f, --values, and --set-file options, and decrypt any secrets.*
files before running "helm diff".

Example:
$ ${HELM_BIN} secrets diff <HELM DIFF OPTIONS>
Expand All @@ -213,20 +214,24 @@ encrypt_helper() {
local yml=$(basename "$1")
cd "$dir"
[[ -e "$yml" ]] || { echo "File does not exist: $dir/$yml"; exit 1; }
local ymldec=$(sed -e "s/\\.yaml$/${DEC_SUFFIX}/" <<<"$yml")
local ymldec=$(sed -e "s/$/${DEC_SUFFIX}/" <<<"$yml")
[[ -e $ymldec ]] || ymldec="$yml"

if [[ $(grep -C10000 'sops:' "$ymldec" | grep -c 'version:') -gt 0 ]]
if [[ $(egrep -C10000 'sops["]?:' "$ymldec" | egrep -c 'version["]?:') -gt 0 ]]
then
echo "Already encrypted: $ymldec"
return
fi
if [[ "$yml" =~ .yaml$ ]]
then
local type_params="--input-type yaml --output-type yaml"
fi
if [[ $yml == $ymldec ]]
then
sops --encrypt --input-type yaml --output-type yaml --in-place "$yml"
sops --encrypt ${type_params} --in-place "$yml"
echo "Encrypted $yml"
else
sops --encrypt --input-type yaml --output-type yaml "$ymldec" > "$yml"
sops --encrypt ${type_params} "$ymldec" > "$yml"
echo "Encrypted $ymldec to $yml"
fi
}
Expand Down Expand Up @@ -265,17 +270,22 @@ decrypt_helper() {

__dec=0
[[ -e "$yml" ]] || { echo "File does not exist: $yml"; exit 1; }
if [[ $(grep -C10000 'sops:' "$yml" | grep -c 'version:') -eq 0 ]]
if [[ $(egrep -C10000 'sops["]?:' "$yml" | egrep -c 'version["]?:') -eq 0 ]]
then
echo "Not encrypted: $yml"
__ymldec="$yml"
else
__ymldec=$(sed -e "s/\\.yaml$/${DEC_SUFFIX}/" <<<"$yml")
__ymldec=$(sed -e "s/$/${DEC_SUFFIX}/" <<<"$yml")
if [[ -e $__ymldec && $__ymldec -nt $yml ]]
then
echo "$__ymldec is newer than $yml"
else
sops --decrypt --input-type yaml --output-type yaml "$yml" > "$__ymldec" || { rm "$__ymldec"; exit 1; }
if [[ "$yml" =~ .yaml$ ]]
then
sops --decrypt --input-type yaml --output-type yaml "$yml" > "$__ymldec" || { rm "$__ymldec"; exit 1; }
else
sops --decrypt "$yml" > "$__ymldec" || { rm "$__ymldec"; exit 1; }
fi
__dec=1
fi
fi
Expand Down Expand Up @@ -308,7 +318,12 @@ dec() {
view_helper() {
local yml="$1"
[[ -e "$yml" ]] || { echo "File does not exist: $yml"; exit 1; }
if [[ "$yml" =~ .yaml$ ]]
then
sops --decrypt --input-type yaml --output-type yaml "$yml"
else
sops --decrypt "$yml"
fi
}

view() {
Expand All @@ -324,7 +339,12 @@ view() {
edit_helper() {
local yml="$1"
[[ -e "$yml" ]] || { echo "File does not exist: $yml"; exit 1; }
if [[ "$yml" =~ .yaml$ ]]
then
exec sops --input-type yaml --output-type yaml "$yml" < /dev/tty
else
exec sops "$yml" < /dev/tty
fi
}

edit() {
Expand Down Expand Up @@ -411,6 +431,19 @@ EOF
fi
shift # to also skip option arg
;;
--set-file)
cmdopts+=("$1")
yml="${2#*=}"
if [[ $yml =~ ^(.*/)?secrets ]]
then
decrypt_helper $yml ymldec decrypted
cmdopts+=("${2%=*}=$ymldec")
[[ $decrypted -eq 1 ]] && decfiles+=("$ymldec")
else
cmdopts+=("$2")
fi
shift # to also skip option arg
;;
*)
cmdopts+=("$1")
;;
Expand Down