Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: rewrite snapshot.py as an Ansible module / add support for thin origins #58

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

trgill
Copy link
Collaborator

@trgill trgill commented May 20, 2024

Enhancement: rewrite snapshot.py as an Ansible module / add support for thin origins

Reason:
#43
#57

Result:
Referenced issues fixed

Issue Tracker Tickets (Jira or BZ if any):

@trgill trgill changed the title Module work rewrite snapshot.py as an Ansible module / add support for thin origins May 20, 2024
library/snapshot.py Fixed Show fixed Hide fixed
library/snapshot.py Fixed Show fixed Hide fixed
library/snapshot.py Fixed Show fixed Hide fixed
tasks/main.yml Outdated Show resolved Hide resolved

- name: Assert no changes for create snapset
assert:
that: not snapshot_cmd["changed"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check won't work. It's impossible to check if include_role resulted in a change in Ansible. There is no proper way to check idempotency other than looking at the playbook output.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@richm added a boolean response from the module to indicate if any action was taken. Rich, please let me know what changes I should make.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html#creating-a-module
The module should return a dict. That dict should have a field called changed which is true if the module changed something or false otherwise.

Copy link
Contributor

@richm richm May 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This check won't work. It's impossible to check if include_role resulted in a change in Ansible. There is no proper way to check idempotency other than looking at the playbook output.

But in this case, when the role calls this module, it uses register: snapshot_cmd, so the test playbook can check. Otherwise, you are correct - there is no way to get at the internal state. Several of the roles register the module output or otherwise use some sort of internal variable to indicate the changed state of the role for the purpose of testing for idempotency.

@@ -1,22 +1,146 @@
from __future__ import print_function
#!/usr/bin/python
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you plan to add python unit tests for this module?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes. I'll add some unit tests.

library/snapshot.py Fixed Show fixed Hide fixed
library/snapshot.py Fixed Show fixed Hide fixed
library/snapshot.py Fixed Show fixed Hide fixed
@trgill trgill changed the title rewrite snapshot.py as an Ansible module / add support for thin origins feat: rewrite snapshot.py as an Ansible module / add support for thin origins May 22, 2024
@spetrosi
Copy link
Contributor

[citest]

tasks/main.yml Outdated Show resolved Hide resolved
tasks/main.yml Outdated Show resolved Hide resolved
tasks/main.yml Outdated Show resolved Hide resolved
tasks/main.yml Outdated Show resolved Hide resolved
tasks/main.yml Outdated Show resolved Hide resolved
library/snapshot.py Outdated Show resolved Hide resolved


EXAMPLES = r"""
# Create Snapshots of all VGs
Copy link
Contributor

@richm richm May 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be an example of using the module directly, not using the role.
Note that using the module directly is not supported, but module scanning tools such as ansible-test and ansible-doc may throw an error if this documentation is not in the correct format.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok. The errors are a bit difficult to understand. I was following the example in the storage role. Currently I'm seeing a lot of errors for the docs: https://github.com/linux-system-roles/snapshot/actions/runs/9502379067/job/26190204498 - it looks like the storage role is using a different "-collection-version 1.78.2" vs the snapshot role is using "--collection-version 1.79.0"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I also tried following the examples in: https://docs.ansible.com/ansible/latest/dev_guide/developing_modules_general.html

But had trouble with errors with that format too.

tasks/main.yml Outdated
snapshot_lvm_mount_options: "{{ snapshot_lvm_mount_options | d(omit) }}"
snapshot_lvm_fstype: "{{ snapshot_lvm_fstype | d(omit) }}"
snapshot_lvm_mountpoint: "{{ snapshot_lvm_mountpoint | d(omit) }}"
snapshot_lvm_set: "{{ snapshot_lvm_set | to_json }}"
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@richm is there a better way to pass snapshot_lvm_set?

I'd like it to be None if it is not set, but the way I've done it here I'm getting "{}\0" when I read it in the python.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then if you define snapshot_lvm_set as a list of dict in the module, you can just use

        snapshot_lvm_set: "{{ snapshot_lvm_set | d(omit) }}"

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, on the module side, I will remove the code that expects JSON and use dict()?

When I define it as you suggested, the module gets snapshot_lvm_set set to a string, right? But the string is no longer JSON (the values have single quotes).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So, on the module side, I will remove the code that expects JSON and use dict()?

yes

When I define it as you suggested, the module gets snapshot_lvm_set set to a string, right?

No, the module should get snapshot_lvm_set as a dict.

But the string is no longer JSON (the values have single quotes).

in defaults/main.yml you should have

snapshot_lvm_set: {}

snapshot_lvm_snapset_name=dict(type="str"),
snapshot_lvm_mount_options=dict(type="str"),
snapshot_lvm_mountpoint=dict(type="str"),
snapshot_lvm_set=dict(type="str"),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should be a list of dict - see the storage role for examples e.g. https://github.com/linux-system-roles/storage/blob/main/library/blivet.py#L2131

        snapshot_lvm_set=dict(type="list", elements="dict",
                                                  options=dict(name=dict(type="str"),
                                                                         vg=dict(type="str"),
                                                                         lv=dict(type="str"),
                                                                         percent_space_required=dict(type="int"),
....

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry - not a list of dict but just a dict

@richm
Copy link
Contributor

richm commented Jun 10, 2024

ping - any updates?

@@ -14,7 +14,7 @@
__snapshot_failed_params.get('snapshot_lvm_percent_space_required')
}}"
snapshot_lvm_all_vgs: "{{
__snapshot_failed_params.get('snapshot_all')
__snapshot_failed_params.get('snapshot_lvm_all_vgs') | d(false)
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@richm how should I default this to false when snapshot_lvm_all_vgs is not defined. Currently it defaults to "" and that causes an error when the module expects it to be a boolean.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
__snapshot_failed_params.get('snapshot_lvm_all_vgs') | d(false)
__snapshot_failed_params.get('snapshot_lvm_all_vgs', false)

Update to ignore thinpool LVs and support think provisioned sources.

Signed-off-by: Todd Gill <[email protected]>
it returns rc, is_snapshot and should not be called as a boolean
function.

Signed-off-by: Todd Gill <[email protected]>
verbosity: 2

- name: Parse raw output
- name: Set result
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think you need to do this - I believe the register: snapshot_cmd in the snapshot task creates the global variable snapshot_cmd

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok - fixed.

@richm
Copy link
Contributor

richm commented Jun 18, 2024

@trgill This PR should fix all of those ansible-test (and flake8, pylint, black) issues. trgill#6 I didn't test it but hopefully the errors will be obvious

A big change was using module.run_command - which means most of the methods required the addition of the module argument - the way other modules handle this is by declaring a module class e.g. SnapshotModule(AnsibleModule) and putting all of the methods in this class - then you can just use self.module.run_command everywhere instead of having to pass module as a method argument.

fix various ansible-test, pylint, flake8, black issues
@richm
Copy link
Contributor

richm commented Jun 18, 2024

[citest]

@trgill
Copy link
Collaborator Author

trgill commented Jun 19, 2024

[citest bad]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants