[Security Solutions] Make Cypress tests work independent of on-disk rules vs cloud rules #105857
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…it pulls from online
FrankHassanabad
added
v8.0.0
v7.15.0
release_note:skip
rw-access
reviewed
Jul 15, 2021
x-pack/plugins/security_solution/cypress/integration/detection_rules/prebuilt_rules.spec.ts
Show resolved
Hide resolved
x-pack/plugins/security_solution/cypress/tasks/alerts_detection_rules.ts
Show resolved
Hide resolved
💚 Build SucceededMetrics [docs]
History
To update your PR or re-run it, just comment with: |
MadameSheema
approved these changes
Jul 16, 2021
There was a problem hiding this comment.
Approving this since I it is not a good idea to have our tests blocking things but I would deeply appreciate @FrankHassanabad if we can have a quick call to have a better understanding why that missalignment exist. Thanks for taking care of this :)
|
Putting this in draft mode in favor of potentially one of the other options from: |
rw-access
approved these changes
Jul 19, 2021
|
Closing this since we went with a different option here: |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Whenever we forget to mirror a rule on disk on-disk compared to a fleet server being updated, the pre-packaged rules will activate and prefer to grab the rules from the online service which can and will cause issues with our tests. This PR decouples that logic so that the total count of rules comes from the pre-packaged rule endpoint instead of relying on-disk rules.
>=check instead of strict equals and splits out the regex of numbers from their text in the tests.prebuild_rules.spec.tsOne important CON to mention is that you're losing a bit of test safety that all the rules were installed correctly since we use a
>=even when checking strings around deleting rules.How to manual testing this situation:
Run the cypress tests and delete one of the rules from
And you should see Cypress still passing tests.
Checklist