Collect, snapshot and gracefully restore global objects in e2e test

[tnozicka]

Description of your changes:
This PR extends our e2e framework to be able to snapshot, collect and restore global resources like NodeConfigs or ScyllaOperator configs. This is very important to be able to debug the disruptive serial suites that use the same object in multiple tests and the final dump just shows the last test.

This also no longer requires the NodeConfig or Resource quotas to be missing before running the e2e suite an depending on the cleanup policy and whether the object has existed before, it will be restored after the test. This also help not to mess up local setups too much but more importantly it allows you to specify a cleanup policy to avoid deleting e.g. the NodeConfig used during the test.

Also this hooks extra dumps for the artifacts dir, e.g. e2e/cluster-0/cluster-scoped-per-ns/e2e-test-nodesetup-q6bhx-0-d7s4w/cluster-scoped/nodeconfigs.scylla.scylladb.com/cluster.yaml

Which issue is resolved by this Pull Request:
Resolves #2045

Requires

• Fix NodeConfig e2e race check with node setup daemon #2101

[scylla-operator-bot]

@tnozicka: GitHub didn't allow me to request PR reviews from the following users: tnozicka.

Note that only scylladb members and repo collaborators can review this PR, and authors cannot review their own PRs.

In response to this:

Description of your changes:

Which issue is resolved by this Pull Request:
Resolves #

/cc

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[tnozicka]

/hold
for #2101

[tnozicka]

/hold cancel
#2101 landed

[zimnx]

/assign rzetelskik

[rzetelskik]

/lgtm

[scylla-operator-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: rzetelskik, tnozicka, zimnx

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [tnozicka,zimnx]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[rzetelskik]

@tnozicka: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-gke-parallel-clusterip 00a1223 link true /test e2e-gke-parallel-clusterip
Full PR test history. Your PR dashboard.

#2061 (comment)
known manager flake
/retest