etcd-quorum-guard should not attempt to deploy 3 replicas if useUnsupportedUnsafeNonHANonProductionUnstableEtcd is set

[cgruver]

Description

When deploying a single node cluster with OKD 4.4, it is possible to set useUnsupportedUnsafeNonHANonProductionUnstableEtcd so that the bootstrap process will complete with only one Master node. However, the etcd-quorum-guard deployment is still set to three replicas, two of which cannot start, (for obvious reasons). The deployment can be scaled down, but the operator immediately restores the replicas value to 3.

This prevents the cluster from running in a healthy state, and prevents upgrades.

Steps to reproduce the issue:

1. Deploy a Single Node Cluster
2. As soon as the API is available from the Bootstrap node, set useUnsupportedUnsafeNonHANonProductionUnstableEtcd

oc patch etcd cluster -p='{"spec": {"unsupportedConfigOverrides": {"useUnsupportedUnsafeNonHANonProductionUnstableEtcd": true}}}' --type=merge

3. Wait for installation to complete, and observe that etcd-quorum-guard has two pending pods

OKD 4.4 release: 4.4.0-0.okd-2020-04-21-163702

[runcom]

@hexfusion @alaypatel07 ptal

[vrutkovs]

PR for fcos branch to support this: #1708

Not sure if its worth pursuing in master now, as it conflicts with openshift/enhancements#292

[enxebre]

To clarify and make sure we are in the same page and agree on the boundaries to provide the best experience @vrutkovs openshift/enhancements#292 is about manipulating compute resources while honouring PDBs.
It's not meant to manipulate the etcd-quorum-guard deployment or handle any of the etcd operational aspects in any manner. It rather just respect any PDB, including the one given by etcd-quorum-guard.
This should be transparent for openshift/enhancements#292.

[cgruver]

@vrutkovs This is resolved now, correct? Any reason it can't be closed?

[vrutkovs]

Fixed by #1708 in fcos branch, didn't send this upstream though