Port-Forwarding cleanups (#722)

As per minio/console#2539 , the websocket behavior integrated as part of Console 0.22.1 (minio/console#2419) seems to break port-forwarding behavior. There's no easy fix for this. NodePorts are a workaround, but slightly kludgy. Ingress is the better long-term solution, but requires more work. This is a stopgap: - For Operator, point users towards NodePorts if port-forwarding fails - For Tenant Console, simply drop port-forwarding entirely and point only at Ingress/LB Out of scope but in progress is Ingress guidance for Nginx and Traefik so we can close the loop on this.
minio · Feb 7, 2023 · 94d1faa · 94d1faa
1 parent da88409
commit 94d1faa
Show file tree

Hide file tree

Showing 10 changed files with 136 additions and 211 deletions.
diff --git a/source/includes/common-installation.rst b/source/includes/common-installation.rst
@@ -168,17 +168,8 @@ Console.
 .. start-local-jbod-single-node-desc
 
 MinIO strongly recommends direct-attached :abbr:`JBOD (Just a Bunch of Disks)`
-arrays with XFS-formatted disks for best performance.  
-
-- Direct-Attached Storage (DAS) has significant performance and consistency
-  advantages over networked storage (NAS, SAN, NFS). 
-
-- Deployments using non-XFS filesystems (ext4, btrfs, zfs) tend to have
-  lower performance while exhibiting unexpected or undesired behavior.  
-
-- RAID or similar technologies do not provide additional resilience or
-  availability benefits when used with distributed MinIO deployments, and
-  typically reduce system performance.
+arrays with XFS-formatted disks for best performance.
+Using any other type of backing storage (SAN/NAS, ext4, RAID, LVM) typically results in a reduction in performance, reliability, predictability, and consistency.
 
 Ensure all server drives for which you intend MinIO to use are of the same type (NVMe, SSD, or HDD) with identical capacity (e.g. ``12`` TB).
 MinIO does not distinguish drive types and does not benefit from mixed storage types. 

diff --git a/source/includes/common/common-install-operator-kubectl-plugin.rst b/source/includes/common/common-install-operator-kubectl-plugin.rst
@@ -67,23 +67,7 @@ The output resembles the following:
 4) Open the Operator Console
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-Run the :mc:`kubectl minio proxy` command to temporarily forward traffic from
-the :ref:`MinIO Operator Console <minio-operator-console>` service to your 
-local machine:
+.. include:: /includes/common/common-k8s-connect-operator-console.rst
 
-.. code-block:: shell
-   :class: copyable
-
-   kubectl minio proxy
-
-The command output includes a JWT token you must use to log into the
-Operator Console. 
-
-.. image:: /images/k8s/operator-dashboard.png
-   :align: center
-   :width: 70%
-   :class: no-scaled-link
-   :alt: MinIO Operator Console
+
 
-You can deploy a new :ref:`MinIO Tenant <minio-k8s-deploy-minio-tenant>` from
-the Operator Dashboard.
diff --git a/source/includes/common/common-k8s-connect-operator-console.rst b/source/includes/common/common-k8s-connect-operator-console.rst
@@ -0,0 +1,82 @@
+.. tab-set::
+
+   .. tab-item:: Port Forwarding
+
+      .. note::
+
+         Starting with Operator 4.5.8, the MinIO Operator Console implements websockets for improved performance.
+         This protocol may exhibit unpredictable behavior with Kubernetes port-forwarding, especially during the login sequence.
+
+         Users who encounter timeouts when using port forwarding should select from the remaining tabs to view alternatives for connecting to the Operator Console.
+
+      Run the :mc:`kubectl minio proxy` command to temporarily forward traffic from
+      the :ref:`MinIO Operator Console <minio-operator-console>` service to your 
+      local machine:
+
+      .. cond:: k8s and not openshift
+
+         .. code-block:: shell
+            :class: copyable
+
+            kubectl minio proxy
+
+      .. cond:: openshift
+
+         .. code-block:: shell
+            :class: copyable
+
+            oc minio proxy
+
+      The command output includes a JWT token you must use to log into the
+      Operator Console. 
+
+      .. image:: /images/k8s/operator-dashboard.png
+         :align: center
+         :width: 70%
+         :class: no-scaled-link
+         :alt: MinIO Operator Console
+
+      You can deploy a new :ref:`MinIO Tenant <minio-k8s-deploy-minio-tenant>` from
+      the Operator Dashboard.
+
+   .. tab-item:: Node Ports
+
+      Use the following command to identify the :kube-docs:`NodePorts <concepts/services-networking/service/#type-nodeport>` configured for the Operator Console.
+      If your local host does not have the ``jq`` utility installed, you can run the first command and locate the ``spec.ports`` section of the output.
+
+      .. code-block:: shell
+         :class: copyable
+
+         kubectl get svc/console -n minio-operator -o json | jq -r '.spec.ports'
+
+      The output resembles the following:
+
+      .. code-block:: json
+
+         [
+            {
+               "name": "http",
+               "nodePort": 31055,
+               "port": 9090,
+               "protocol": "TCP",
+               "targetPort": 9090
+            },
+            {
+               "name": "https",
+               "nodePort": 31388,
+               "port": 9443,
+               "protocol": "TCP",
+               "targetPort": 9443
+            }
+         ]
+
+      Use the ``http`` or ``https`` port depending on whether you deployed the Operator with Console TLS enabled via :mc-cmd:`kubectl minio init --console-tls`.
+
+      Append the ``nodePort`` value to the externally-accessible IP address of a worker node in your Kubernetes cluster.
+
+      Use the following command to retrieve the JWT token necessary for logging into the Operator Console:
+
+      .. code-block:: shell
+         :class: copyable
+
+         kubectl get secret/console-sa-secret -n minio-operator -o json | jq -r '.data.token' | base64 -d
diff --git a/source/includes/k8s/common-minio-kes.rst b/source/includes/k8s/common-minio-kes.rst
@@ -19,17 +19,9 @@ You can use either the MinIO Tenant Console or the MinIO :mc:`mc` CLI to enable
 
    .. tab-item:: MinIO Tenant Console
 
-      You can manually :ref:`port forward <create-tenant-operator-forward-ports>` the MinIO Tenant Console service to your local host machine for simplified access:
-
-      .. code-block:: shell
-         :class: copyable
-
-         # Replace 'minio-tenant' with the name of the MinIO Tenant
-         # Replace '-n minio' with the namespace of the MinIO Tenant
-
-         kubectl port-forward svc/minio-tenant-console 9443:9443 -n minio
-
-      Open the MinIO Console by navigating to http://127.0.0.1:9443 in your preferred browser and logging in with the root credentials for the deployment.
+      Connect to the :ref:`MinIO Tenant Console service <create-tenant-connect-tenant>` and log in.
+      For clients internal to the Kubernetes cluster, you can specify the :kube-docs:`service DNS name <concepts/services-networking/dns-pod-service/#a-aaaa-records>`.
+      For clients external to the Kubernetes cluster, specify the hostname of the service exposed by Ingress, Load Balancer, or similar Kubernetes network control component.
 
       Once logged in, create a new Bucket and name it to your preference.
       Select the Gear :octicon:`gear` icon to open the management view.
@@ -44,35 +36,19 @@ You can use either the MinIO Tenant Console or the MinIO :mc:`mc` CLI to enable
 
    .. tab-item:: MinIO CLI
 
-      You can manually :ref:`port forward <create-tenant-operator-forward-ports>` the ``minio`` service for temporary access via the local host.
-
-      Run this command in a separate Terminal or Shell:
+      Use the :ref:`MinIO API Service <create-tenant-connect-tenant>` to create a new :ref:`alias <alias>` for the MinIO deployment.
+      You can then use the :mc:`mc encrypt set` command to enable SSE-KMS encryption for a bucket:
 
       .. code-block:: shell
          :class: copyable
 
-         # Replace '-n minio' with the namespace of the MinIO deployment
-         # If you deployed the Tenant without TLS you may need to change the port range
-         
-         # You can validate the ports in use by running
-         #  kubectl get svc/minio -n minio
-
-         kubectl port forward svc/minio 443:443 -n minio
-
-      The following commands in a new Terminal or Shell window:
-
-      - Create a new :ref:`alias <alias>` for the MinIO deployment
-      - Create a new bucket for storing encrypted data
-      - Enable SSE-KMS encryption on that bucket
-
-      .. code-block:: shell
-         :class: copyable
-
-         mc alias set k8s https://127.0.0.1:443 ROOTUSER ROOTPASSWORD
+         mc alias set k8s https://minio.minio-tenant-1.svc.cluster-domain.example:443 ROOTUSER ROOTPASSWORD
 
          mc mb k8s/encryptedbucket
          mc encrypt set SSE-KMS encrypted-bucket-key k8s/encryptedbucket
 
+      For clients external to the Kubernetes cluster, specify the hostname of the service exposed by Ingress, Load Balancer, or similar Kubernetes network control component.
+
       Write a file to the bucket using :mc:`mc cp` or any S3-compatible SDK with a ``PutObject`` function. 
       You can then run :mc:`mc stat` on the file to confirm the associated encryption metadata.
 
@@ -90,7 +66,8 @@ MinIO requires that the |EK| for a given bucket or object exist on the root KMS
 You can use the :mc-cmd:`mc admin kms key create` command against the MinIO Tenant.
 
 You must ensure your local host can access the MinIO Tenant pods and services before using :mc:`mc` to manage the Tenant.
-You can manually :ref:`port forward <create-tenant-operator-forward-ports>` the ``minio`` service for temporary access via the local host.
+For hosts internal to the Kubernetes cluster, you can use the :kube-docs:`service DNS name <concepts/services-networking/dns-pod-service/#a-aaaa-records>`.
+For hosts external to the Kubernetes cluster, specify the hostname of the service exposed by Ingress, Load Balancer, or similar Kubernetes network control component.
 
 Run this command in a separate Terminal or Shell:
 

diff --git a/source/includes/k8s/steps-deploy-tenant-cli.rst b/source/includes/k8s/steps-deploy-tenant-cli.rst
@@ -18,8 +18,6 @@ To deploy a tenant from the command line, complete the following steps:
 
 :ref:`create-tenant-cli-access-tenant-console`
 
-:ref:`create-tenant-cli-forward-ports`
-
 .. _create-tenant-cli-determine-settings-required-options:
 
 1) Determine Values for Required Settings
@@ -234,47 +232,55 @@ In addition to access credentials, the output shows the service name and service
 5) Access the Tenant's MinIO Console
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
-To access the :ref:`MinIO Console <minio-console>` for the tenant, forward the tenant's port.
+The MinIO Operator creates services for the MinIO Tenant. 
 
-- If necessary, run ``kubectl get svc -n <namespace>`` to retrieve the tenant's port number.
-- Run the following to forward the tenant's port and access it from a browser:
+.. cond:: openshift
 
-  .. code-block:: shell
-     :class: copyable
+   Use the ``oc get svc -n TENANT-PROJECT`` command to review the deployed services:
 
-     kubectl port-forward svc/<tenant-name>-console -n <tenant-namespace> <localport>:<tenantport>
+   .. code-block:: shell
+      :class: copyable
 
-  - Replace ``<tenant-name>`` with the name of your tenant.
-  - Replace ``<tenant-namespace>`` with the namespace the tenant exists in.
-  - Replace ``<localport>`` with the port number to use on your local machine to access the tenant's MinIO Console.
-  - Replace ``<tenantport>`` with the port number the MinIO Operator assigned to the tenant.
+      oc get svc -n minio-tenant-1
 
-- Go to ``https://127.0.0.1:<localport>`` to Access the tenant's MinIO Console.
+.. cond:: k8s and not openshift 
 
-  Replace ``<localport>`` with the port number you used when forwarding the tenant's port.
+   Use the ``kubectl get svc -n NAMESPACE`` command to review the deployed services:
 
-- Login with the username and password shown in the tenant creation output and recorded in step 4 above.
+   .. code-block:: shell
+      :class: copyable
 
-.. _create-tenant-cli-forward-ports:
+      kubectl get svc -n minio-tenant-1
 
-6) Forward Ports
-~~~~~~~~~~~~~~~~
+.. code-block:: shell
 
-You can temporarily expose each service using the ``kubectl port-forward`` utility. 
-Run the following examples to forward traffic from the local host running ``kubectl`` to the services running inside the Kubernetes cluster.
+   NAME                               TYPE           CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
+   minio                              LoadBalancer   10.97.114.60     <pending>     443:30979/TCP    2d3h
+   minio-tenant-1-console             LoadBalancer   10.106.103.247   <pending>     9443:32095/TCP   2d3h
+   minio-tenant-1-hl                  ClusterIP      None             <none>        9000/TCP         2d3h
+   minio-tenant-1-log-hl-svc          ClusterIP      None             <none>        5432/TCP         2d3h
+   minio-tenant-1-log-search-api      ClusterIP      10.103.5.235     <none>        8080/TCP         2d3h
+   minio-tenant-1-prometheus-hl-svc   ClusterIP      None             <none>        9090/TCP         7h39m
 
-.. tab-set::
+- The ``minio`` service corresponds to the MinIO Tenant service. 
+  Applications should use this service for performing operations against the MinIO Tenant.
+
+- The ``*-console`` service corresponds to the :minio-git:`MinIO Console <console>`. 
+  Administrators should use this service for accessing the MinIO Console and performing administrative operations on the MinIO Tenant.
 
-   .. tab-item:: MinIO Tenant
+The remaining services support Tenant operations and are not intended for consumption by users or administrators.
+
+By default each service is visible only within the Kubernetes cluster. 
+Applications deployed inside the cluster can access the services using the ``CLUSTER-IP``. 
 
-      .. code-block:: shell
-         :class: copyable
+Applications external to the Kubernetes cluster can access the services using the ``EXTERNAL-IP``. 
+This value is only populated for Kubernetes clusters configured for Ingress or a similar network access service. 
+Kubernetes provides multiple options for configuring external access to services. 
 
-         kubectl port-forward service/minio 443:443
+.. cond:: k8s and not openshift
 
-   .. tab-item:: MinIO Console
-
-      .. code-block:: shell
-         :class: copyable
+   See the Kubernetes documentation on :kube-docs:`Publishing Services (ServiceTypes) <concepts/services-networking/service/#publishing-services-service-types>` and :kube-docs:`Ingress <concepts/services-networking/ingress/>` for more complete information on configuring external access to services.
+
+.. cond:: openshift
 
-         kubectl port-forward service/minio-tenant-1-console 9443:9443
+   See the OpenShift documentation on :openshift-docs:`Route or Ingress <networking/understanding-networking.html#nw-ne-comparing-ingress-route_understanding-networking>` for more complete information on configuring external access to services.
diff --git a/source/includes/openshift/deploy-minio-on-openshift.rst b/source/includes/openshift/deploy-minio-on-openshift.rst
@@ -153,20 +153,7 @@ You can find the MinIO Operator Interface from the :guilabel:`Operators` left-ha
 
 The MinIO Operator includes the Operator Console, a browser-based management interface for managed MinIO tenants.
 
-You can temporarily port-forward the Operator Console to your local host machine using the ``oc minio port-forward`` command:
-
-.. code-block:: shell
-   :class: copyable
-
-   oc minio port-forward
-
-The command returns a JWT token and a URL you can open in your browser.
-
-.. image:: /images/k8s/operator-dashboard.png
-   :align: center
-   :width: 70%
-   :class: no-scaled-link
-   :alt: Operator Dashboard
+.. include:: /includes/common/common-k8s-connect-operator-console.rst
 
 You can create a permanent routing rule by creating a :openshift-docs:`Route or Ingress <networking/understanding-networking.html#nw-ne-comparing-ingress-route_understanding-networking>` to allow access from external clients, such as your local computer browser. 
 

diff --git a/source/includes/openshift/steps-deploy-minio-tenant.rst b/source/includes/openshift/steps-deploy-minio-tenant.rst
@@ -93,25 +93,3 @@ See the Kubernetes documentation on
 :kube-docs:`Publishing Services (ServiceTypes) <concepts/services-networking/service/#publishing-services-service-types>` 
 and :kube-docs:`Ingress <concepts/services-networking/ingress/>` 
 for more complete information on configuring external access to services.
-
-4) Forward Ports
-~~~~~~~~~~~~~~~~
-
-You can temporarily expose each service using the ``oc port-forward`` utility. 
-Run the following examples to forward traffic from the local host running ``oc`` to the services running inside the Kubernetes cluster.
-
-.. tab-set::
-
-   .. tab-item:: MinIO Tenant
-
-      .. code-block:: shell
-         :class: copyable
-
-         oc port-forward service/minio 443:443
-
-   .. tab-item:: MinIO Console
-
-      .. code-block:: shell
-         :class: copyable
-
-         oc port-forward service/minio-tenant-1-console 9443:9443
diff --git a/source/operations/install-deploy-manage/deploy-minio-single-node-single-drive.rst b/source/operations/install-deploy-manage/deploy-minio-single-node-single-drive.rst
@@ -77,7 +77,7 @@ The following procedure deploys MinIO consisting of a single MinIO server and a
    :class: note
 
    MinIO's strict **read-after-write** and **list-after-write** consistency
-   model requires local drive filesystems (``xfs``, ``ext4``, etc.).
+   model requires local drive filesystems.
 
    MinIO cannot provide consistency guarantees if the underlying storage
    volumes are NFS or a similar network-attached storage volume.