Shell quote support for Jinja macros

[maage]

Description:

Quote a string to safely use as in a POSIX shell.

Handling quoting shell is quite hard when there is complex macro
workflow. This filter allows to quote where ever variable is used.

Name mirrors respective feature in ansible.

I also sorted jinja filters alphabetically for them to be neat.

Rationale:

When ever I look at bash remediations, there is multiple ways things can go sideways if variables contain shell metacharacters. There is sometimes extra checks.

If this feature is used, it provides clear security benefit.

Review Hints:

For now there is no test suite or any template that uses this.

[openshift-ci]

Hi @maage. Thanks for your PR.

I'm waiting for a ComplianceAsCode member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[github-actions]

Start a new ephemeral environment with changes proposed in this pull request:

Fedora Environment

Oracle Linux 8 Environment

[Mab879]

/ok-to-test

[Mab879]

Do you have an example where this could be used?

[maage]

It should be used anywhere where there is shellscript and it is not absolutely sure jinja value does not have shell metacharacters.

From 10-bash.jinja bash_ensure_pam_module_options start, set used jinja variables as shell variables using quote and then use them as normal shell variables.

pamFile={{{ pamFile | quote }}}
if [ -e "${pamFile}" ] ; then

This pattern would clearly split parts of shell scripts where there is first part to make all jinja parameters as shell parameters and then rest of the script is just pure shellscript. For somewhat complex parts that is not so. But simple macros like bash_ensure_pam_module_options this kind of conversion would provide clarity. When coding you should need to think only one domain and only one set of quotation rules.

And essentially any string that is not meant to be regexp but used in regexp context in shell script should be handled like:

type_rx={{{ type | escape_regex | quote }}}
module_rx={{{ module | escape_regex | quote }}}
    if grep -q -P "^\\s*(?"'!'"${type_rx}\\s)[[:alnum:]]+\\s+[[:alnum:]]+\\s+${module_rx}" < "${pamFile}" ; then

But that would require also standardizing s delim character in sed parts etc and then ensuring it is also quoted in escape_regex.

Also I think shell scripts should use

set -u

to catch any typos in variable names.

Another example from: shared/templates/pam_options/ansible.template

- name: Check the presence of "{{{ arg['argument'] }}}" argument in "{{{ MODULE }}}" module
  shell: |
    set -o pipefail
    grep -E '^\s*'{{{ TYPE | quote }}}'\s+'{{{ CONTROL_FLAG | quote }}}'\s+'{{{ MODULE quote }}}'.*\s+'{{{ arg['argument'] | quote }}}'(=|\s|\s*$)' {{{ PATH | quote }}} || true
  register: check_pam_module_argument_result

But anyways, above should be rewritten using command + argv, example below.

Also what is lacking is same stuff with ansible shell: tasks (with {{ }}).

Like: shared/templates/rsyslog_logfiles_attributes_modify/ansible.template

- name: '{{{ rule_title }}} - Extract log files old format'
  ansible.builtin.shell: |
    set -o pipefail
    grep -oP '^[^(\s|#|\$)]+[\s]+.*[\s]+-?(/+[^:;\s]+);*\.*$' {{ item.1.path | quote }} | \
    awk '{print $NF}' | \
    sed -e 's/^-//' || true
  loop: "{{ rsyslog_config_files.results | subelements('files') }}"
  register: log_files_old
  changed_when: False

Though above should be rewritten as awk / sed and use command, then there is no need to have quote.

Just like this should be like:

- name: '{{{ rule_title }}} - Get include files directives'
  ansible.builtin.command:
    argv:
      - awk
      - >-
        /)/{f=0} /include\(/{f=1} f{nf=gensub("^(include\\(|\\s*)file=\"(\\S+)\".*","\\2",1); if($0!=nf){print nf}}
      - rsyslog_etc_config
  register: rsyslog_new_inc
  changed_when: False
  failed_when: False

In most cases ansible shell: tasks should be converted to one command task and then pure ansible to parse the output.

[maage]

Added documentation fragments for this and other filters. These could be improved though. I hope this was okay.

[codeclimate]

Code Climate has analyzed commit 2a9ae08 and detected 0 issues on this pull request.

The test coverage on the diff in this pull request is 71.4% (50% is the threshold).

This pull request will bring the total coverage in the repository to 52.4% (0.0% change).

View more on Code Climate.

[Mab879]

Thanks for the PR!