Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

USHIFT-3463: Add microshift-tuned daemon for unattended TuneD profile activation #3656

Merged
merged 3 commits into from
Jul 31, 2024

Conversation

pmtk
Copy link
Member

@pmtk pmtk commented Jul 18, 2024

No description provided.

@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 18, 2024
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 18, 2024

@pmtk: This pull request references USHIFT-3463 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.17.0" version, but no target version was set.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 18, 2024
Copy link
Contributor

openshift-ci bot commented Jul 18, 2024

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 18, 2024
@pmtk pmtk force-pushed the low-latency/microshift-tuned branch from 689e9e4 to 6bc6446 Compare July 24, 2024 09:03
@openshift-merge-robot openshift-merge-robot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 24, 2024
@pmtk pmtk force-pushed the low-latency/microshift-tuned branch from 6bc6446 to 3267a0a Compare July 29, 2024 09:08
@openshift-merge-robot openshift-merge-robot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jul 29, 2024
@pmtk pmtk force-pushed the low-latency/microshift-tuned branch from 3267a0a to c45d7ff Compare July 29, 2024 09:08
@pmtk pmtk marked this pull request as ready for review July 29, 2024 11:43
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 29, 2024
@openshift-ci openshift-ci bot requested review from jerpeter1 and pliurh July 29, 2024 11:43
@pmtk pmtk force-pushed the low-latency/microshift-tuned branch 3 times, most recently from 4422b03 to 921fcb0 Compare July 29, 2024 11:59
@pmtk pmtk force-pushed the low-latency/microshift-tuned branch from 921fcb0 to 70f8737 Compare July 29, 2024 14:08
Requires=tuned.service

[Service]
ExecStart=/usr/bin/microshift-tuned
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This script may reboot host. Should we provide a mandatory command line for the script to actually do something? Otherwise, if someone runs it, they can end up rebooting accidentally.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You mean that dry run is default and not dry run is protected with --something?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a variable in the config which controls if reboot is allowed. Wouldn't it be slightly confusing if it was defined False but our service always had --reboot-if-allowed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We could call it --live-run or --execute.

# Later, when microshift-tuned.service is merged, we'll re-enable
# it to test the whole R4E flow.
enabled = ["microshift-test-agent", "tuned"]
enabled = ["microshift-test-agent", "tuned", "microshift", "microshift-tuned"]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we also need tuned explcitly enabled? Can we do it like with MicroShift and CRI-O services, so that enabling / starting microshift-tuned would also start tuned?

@@ -15,5 +15,14 @@ scenario_remove_vms() {
}

scenario_run_tests() {
run_tests host1 suites/tuned/
# Should not be ran immediately after creating VM because of
# microshift-tuned rebooting the node to activate the profile.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How do we time this after creating the VM, then?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should we add some kind of a loop checking that the profile is active?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a check to see if we have >= 2 boots. > so we can run the script repeatedly in local env.

Current Profile Should Be Hashed
[Documentation] We expect that microshift-tuned keeps a hash of the profile and variables.

SSHLibrary.File Should Exist /var/lib/microshift-tuned.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how the name / docs reflect the contents of the test. Could you explain?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it better now?

[Documentation] If profile is already active, but cache file is missing,
... we expect microshift-tuned to reactivate it, reboot, and store the hashes.

Command Should Work rm /var/lib/microshift-tuned.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Command Should Work rm /var/lib/microshift-tuned.yaml
Command Should Work rm -f /var/lib/microshift-tuned.yaml

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add a rationale for this change? I'm not opposing, just want to know full story

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometime rm is aliased in the root account as rm -i

# alias rm
alias rm='rm -i'


MicroShift-Tuned Requires Config To Function
[Documentation] Verify that missing configuration will be fatal.
Command Should Work rm /etc/microshift/tuned.yaml
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Command Should Work rm /etc/microshift/tuned.yaml
Command Should Work rm -f /etc/microshift/tuned.yaml

[Documentation] Verify that missing configuration will be fatal.
Command Should Work rm /etc/microshift/tuned.yaml
Command Should Work systemctl restart microshift-tuned.service
Sleep 5s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is potentially a race condition because we're checking a concrete state.
Can we add a loop here with a few retries?

[Documentation] TODO
${bootid}= Get Current Boot Id
Command Should Work systemctl restart microshift-tuned.service
Sleep 60s
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could we think of making this check a bit faster? Is there any other way we can get an indication that the microshift-tuned script "decided" not to reboot?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I changed it but I'm not sure about robustness, we'll see

@pmtk pmtk force-pushed the low-latency/microshift-tuned branch 2 times, most recently from 4272664 to 9ea0735 Compare July 31, 2024 10:08
@pmtk pmtk force-pushed the low-latency/microshift-tuned branch from 9ea0735 to 796c754 Compare July 31, 2024 11:52
Copy link
Contributor

openshift-ci bot commented Jul 31, 2024

@pmtk: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

description=f"""Daemon for unattended TuneD profile activation.

When program starts, it compares configuration and system state.
If the requested profile is not activate, it will be activated,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
If the requested profile is not activate, it will be activated,
If the requested profile is not active, it will be activated,

@ggiguash
Copy link
Contributor

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 31, 2024
Copy link
Contributor

openshift-ci bot commented Jul 31, 2024

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: ggiguash, pmtk

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-merge-bot openshift-merge-bot bot merged commit 2b6c959 into openshift:main Jul 31, 2024
7 checks passed
@pmtk pmtk deleted the low-latency/microshift-tuned branch August 1, 2024 07:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants