@uppy/companion: document how to run many instances

.env.example

@@ -10,6 +10,7 @@ COMPANION_PROTOCOL=http
 COMPANION_PORT=3020
 COMPANION_CLIENT_ORIGINS=
 COMPANION_SECRET=development
+COMPANION_PREAUTH_SECRET=development2
 
 # NOTE: Only enable this in development. Enabling it in production is a security risk
 COMPANION_ALLOW_LOCAL_URLS=true

bin/companion.sh

@@ -11,6 +11,7 @@ else
     COMPANION_PORT=3020 \
     COMPANION_CLIENT_ORIGINS="" \
     COMPANION_SECRET="development" \
+    COMPANION_PREAUTH_SECRET="development2" \
     COMPANION_ALLOW_LOCAL_URLS="true" \
     nodemon --watch packages/@uppy/companion/src --exec node ./packages/@uppy/companion/src/standalone/start-server.js
 fi

packages/@uppy/companion/KUBERNETES.md

@@ -28,6 +28,7 @@ data:
   COMPANION_STREAMING_UPLOAD: true
   COMPANION_REDIS_URL: redis://:superSecretPassword@uppy-redis.uppy.svc.cluster.local:6379
   COMPANION_SECRET: "shh!Issa Secret!"
+  COMPANION_PREAUTH_SECRET: "another secret"
   COMPANION_DROPBOX_KEY: "YOUR DROPBOX KEY"
   COMPANION_DROPBOX_SECRET: "YOUR DROPBOX SECRET"
   COMPANION_BOX_KEY: "YOUR BOX KEY"

packages/@uppy/companion/env_example

@@ -9,6 +9,7 @@ COMPANION_STREAMING_UPLOAD=true
 COMPANION_PROTOCOL=https
 COMPANION_DATADIR=/mnt/uppy-server-data
 COMPANION_SECRET=
+COMPANION_PREAUTH_SECRET=
 COMPANION_SECRET_FILE=
 
 COMPANION_DROPBOX_KEY="dropbox_key"

packages/@uppy/companion/test/mockserver.js

@@ -17,6 +17,7 @@ const defaultEnv = {
   COMPANION_PROTOCOL: 'http',
   COMPANION_DATADIR: './test/output',
   COMPANION_SECRET: 'secret',
+  COMPANION_PREAUTH_SECRET: 'different secret',
 
   COMPANION_DROPBOX_KEY: 'dropbox_key',
   COMPANION_DROPBOX_SECRET: 'dropbox_secret',

website/src/docs/companion.md

@@ -295,6 +295,11 @@ export COMPANION_PERIODIC_PING_URLS="https://example.com/ping1,https://example.c
 export COMPANION_PERIODIC_PING_INTERVAL=60000
 # corresponds to the periodicPingStaticPayload option (JSON string)
 export COMPANION_PERIODIC_PING_STATIC_JSON_PAYLOAD="{\"static\":\"data\"}"
+
+# If you need to use `companionKeysParams` (custom OAuth credentials at request time),
+# set this variable to a strong randomly generated secret.
+# See also https://github.com/transloadit/uppy/pull/2622
+COMPANION_PREAUTH_SECRET="preauth secret"
 ```
 
 See [`.env.example`](https://github.com/transloadit/uppy/blob/main/.env.example) for an example environment configuration file.
@@ -481,6 +486,25 @@ app.use(companion.app({
 
 We have [a detailed guide on running Companion in Kubernetes](https://github.com/transloadit/uppy/blob/main/packages/%40uppy/companion/KUBERNETES.md) for you, that’s how we run our example server at <https://companion.uppy.io>.
 
+### Running many instances
+
+Two ways of running many concurrent Companion instances.
+
+#### Separate endpoints
+
+One option is to run many instances with each instance having its own unique endpoint. This could be on separate ports, (sub)domain names, or IPs. With this setup, the Companion client in Uppy will direct requests to companion in a way so that all requests of a particular upload will always be sent to the same endpoint, and hence process. You would then also typically configure a single instance (one endpoint) to handle all OAuth authentication requests, so that you only need to specify a single OAuth callback URL. See also `oauthDomain` and `validHosts`
+
+#### Behind a load balancer
+
+The other option is to set up a load balancer in front of many Companion instances. Then Uppy’s companion client will only see a single endpoint and send all requests to the associated load balancer, which will then distribute them between Companion instances. The companion instances will then coordinate their messages and events over Redis so that any instance can serve the client’s requests. Note that sticky sessions are **not** needed with this setup. Here are some requirements for this setup:
+
+* The instances need to be connected to the same Redis server.
+* You need to set `COMPANION_SECRET` to the same value on both servers.
+* if you use the `companionKeysParams` feature (Transloadit), you also need `COMPANION_PREAUTH_SECRET` to be the same on each instance.
+* All other configuration needs to be the same, except if you’re running many instances on the same machine, then `COMPANION_PORT` should be different for each instance.
+
+For more information about this setup [see this issue](https://github.com/transloadit/uppy/issues/3538).
+
 ### Adding custom providers
 
 As of now, Companion supports the [providers listed here](https://uppy.io/docs/companion/#Supported-providers) out of the box, but you may also choose to add your own custom providers. You can do this by passing the `customProviders` option when calling the Uppy `app` method. The custom provider is expected to support Oauth 1 or 2 for authentication/authorization.