[Flight Reply] Add Reply Encoding (#26360)

This adds `encodeReply` to the Flight Client and `decodeReply` to the
Flight Server.

Basically, it's a reverse Flight. It serializes values passed from the
client to the server. I call this a "Reply". The tradeoffs and
implementation details are a bit different so it requires its own
implementation but is basically a clone of the Flight Server/Client but
in reverse. Either through callServer or ServerContext.

The goal of this project is to provide the equivalent serialization as
passing props through RSC to client. Except React Elements and
Components and such. So that you can pass a value to the client and back
and it should have the same serialization constraints so when we add
features in one direction we should mostly add it in the other.

Browser support for streaming request bodies are currently very limited
in that only Chrome supports it. So this doesn't produce a
ReadableStream. Instead `encodeReply` produces either a JSON string or
FormData. It uses a JSON string if it's a simple enough payload. For
advanced features it uses FormData. This will also let the browser
stream things like File objects (even though they're not yet supported
since it follows the same rules as the other Flight).

On the server side, you can either consume this by blocking on
generating a FormData object or you can stream in the
`multipart/form-data`. Even if the client isn't streaming data, the
network does. On Node.js busboy seems to be the canonical library for
this, so I exposed a `decodeReplyFromBusboy` in the Node build. However,
if there's ever a web-standard way to stream form data, or if a library
wins in that space we can support it. We can also just build a multipart
parser that takes a ReadableStream built-in.

On the server, server references passed as arguments are loaded from
Node or Webpack just like the client or SSR does. This means that you
can create higher order functions on the client or server. This can be
tokenized when done from a server components but this is a security
implication as it might be tempting to think that these are not fungible
but you can swap one function for another on the client. So you have to
basically treat an incoming argument as insecure, even if it's a
function.

I'm not too happy with the naming parity:

Encode `server.renderToReadableStream` Decode: `client.createFromFetch`

Decode `client.encodeReply` Decode: `server.decodeReply`

This is mainly an implementation details of frameworks but it's annoying
nonetheless. This comes from that `renderToReadableStream` does do some
"rendering" by unwrapping server components etc. The `create` part comes
from the parity with Fizz/Fiber where you `render` on the server and
`create` a root on the client.

Open to bike-shedding this some more.

---------

Co-authored-by: Josh Story <josh.c.story@gmail.com>

fixtures/flight/package.json

@@ -17,6 +17,7 @@
     "babel-preset-react-app": "^10.0.1",
     "body-parser": "^1.20.1",
     "browserslist": "^4.18.1",
+    "busboy": "^1.6.0",
     "camelcase": "^6.2.1",
     "case-sensitive-paths-webpack-plugin": "^2.4.0",
     "compression": "^1.7.4",

fixtures/flight/server/region.js

@@ -33,6 +33,7 @@ if (typeof fetch === 'undefined') {
 
 const express = require('express');
 const bodyParser = require('body-parser');
+const busboy = require('busboy');
 const app = express();
 const compress = require('compression');
 
@@ -95,9 +96,8 @@ app.get('/', async function (req, res) {
 });
 
 app.post('/', bodyParser.text(), async function (req, res) {
-  const {renderToPipeableStream} = await import(
-    'react-server-dom-webpack/server'
-  );
+  const {renderToPipeableStream, decodeReply, decodeReplyFromBusboy} =
+    await import('react-server-dom-webpack/server');
   const serverReference = req.get('rsc-action');
   const [filepath, name] = serverReference.split('#');
   const action = (await import(filepath))[name];
@@ -108,9 +108,18 @@ app.post('/', bodyParser.text(), async function (req, res) {
     throw new Error('Invalid action');
   }
 
-  const args = JSON.parse(req.body);
-  const result = action.apply(null, args);
+  let args;
+  if (req.is('multipart/form-data')) {
+    // Use busboy to streamingly parse the reply from form-data.
+    const bb = busboy({headers: req.headers});
+    const reply = decodeReplyFromBusboy(bb);
+    req.pipe(bb);
+    args = await reply;
+  } else {
+    args = await decodeReply(req.body);
+  }
 
+  const result = action.apply(null, args);
   const {pipe} = renderToPipeableStream(result, {});
   pipe(res);
 });

fixtures/flight/src/actions.js

@@ -1,13 +1,5 @@
 'use server';
 
 export async function like() {
-  return new Promise((resolve, reject) =>
-    setTimeout(
-      () =>
-        Math.random() > 0.5
-          ? resolve('Liked')
-          : reject(new Error('Failed to like')),
-      500
-    )
-  );
+  return new Promise((resolve, reject) => resolve('Liked'));
 }

fixtures/flight/src/index.js

@@ -1,28 +1,28 @@
 import * as React from 'react';
 import {Suspense} from 'react';
 import ReactDOM from 'react-dom/client';
-import ReactServerDOMReader from 'react-server-dom-webpack/client';
+import {createFromFetch, encodeReply} from 'react-server-dom-webpack/client';
 
 // TODO: This should be a dependency of the App but we haven't implemented CSS in Node yet.
 import './style.css';
 
-let data = ReactServerDOMReader.createFromFetch(
+let data = createFromFetch(
   fetch('/', {
     headers: {
       Accept: 'text/x-component',
     },
   }),
   {
-    callServer(id, args) {
+    async callServer(id, args) {
       const response = fetch('/', {
         method: 'POST',
         headers: {
           Accept: 'text/x-component',
           'rsc-action': id,
         },
-        body: JSON.stringify(args),
+        body: await encodeReply(args),
       });
-      return ReactServerDOMReader.createFromFetch(response);
+      return createFromFetch(response);
     },
   }
 );

packages/react-client/src/ReactFlightClient.js

@@ -25,6 +25,8 @@ import {
   parseModel,
 } from './ReactFlightClientHostConfig';
 
+import {knownServerReferences} from './ReactFlightServerReferenceRegistry';
+
 import {REACT_LAZY_TYPE, REACT_ELEMENT_TYPE} from 'shared/ReactSymbols';
 
 import {getOrCreateServerContext} from 'shared/ReactServerContextRegistry';
@@ -495,6 +497,7 @@ function createServerReferenceProxy<A: Iterable<any>, T>(
       return callServer(metaData.id, bound.concat(args));
     });
   };
+  knownServerReferences.set(proxy, metaData);
   return proxy;
 }
 

packages/react-client/src/ReactFlightReplyClient.js

@@ -0,0 +1,278 @@
+/**
+ * Copyright (c) Meta Platforms, Inc. and affiliates.
+ *
+ * This source code is licensed under the MIT license found in the
+ * LICENSE file in the root directory of this source tree.
+ *
+ * @flow
+ */
+
+import type {Thenable} from 'shared/ReactTypes';
+
+import {knownServerReferences} from './ReactFlightServerReferenceRegistry';
+
+import {
+  REACT_ELEMENT_TYPE,
+  REACT_LAZY_TYPE,
+  REACT_PROVIDER_TYPE,
+} from 'shared/ReactSymbols';
+
+import {
+  describeObjectForErrorMessage,
+  isSimpleObject,
+  objectName,
+} from 'shared/ReactSerializationErrors';
+
+import isArray from 'shared/isArray';
+
+type ReactJSONValue =
+  | string
+  | boolean
+  | number
+  | null
+  | $ReadOnlyArray<ReactJSONValue>
+  | ReactServerObject;
+
+export opaque type ServerReference<T> = T;
+
+// Serializable values
+export type ReactServerValue =
+  // References are passed by their value
+  | ServerReference<any>
+  // The rest are passed as is. Sub-types can be passed in but lose their
+  // subtype, so the receiver can only accept once of these.
+  | string
+  | boolean
+  | number
+  | symbol
+  | null
+  | Iterable<ReactServerValue>
+  | Array<ReactServerValue>
+  | ReactServerObject
+  | Promise<ReactServerValue>; // Thenable<ReactServerValue>
+
+type ReactServerObject = {+[key: string]: ReactServerValue};
+
+// function serializeByValueID(id: number): string {
+//   return '$' + id.toString(16);
+// }
+
+function serializePromiseID(id: number): string {
+  return '$@' + id.toString(16);
+}
+
+function serializeServerReferenceID(id: number): string {
+  return '$F' + id.toString(16);
+}
+
+function serializeSymbolReference(name: string): string {
+  return '$S' + name;
+}
+
+function escapeStringValue(value: string): string {
+  if (value[0] === '$') {
+    // We need to escape $ prefixed strings since we use those to encode
+    // references to IDs and as special symbol values.
+    return '$' + value;
+  } else {
+    return value;
+  }
+}
+
+export function processReply(
+  root: ReactServerValue,
+  resolve: (string | FormData) => void,
+  reject: (error: mixed) => void,
+): void {
+  let nextPartId = 1;
+  let pendingParts = 0;
+  let formData: null | FormData = null;
+
+  function resolveToJSON(
+    this:
+      | {+[key: string | number]: ReactServerValue}
+      | $ReadOnlyArray<ReactServerValue>,
+    key: string,
+    value: ReactServerValue,
+  ): ReactJSONValue {
+    const parent = this;
+    if (__DEV__) {
+      // $FlowFixMe
+      const originalValue = this[key];
+      if (typeof originalValue === 'object' && originalValue !== value) {
+        if (objectName(originalValue) !== 'Object') {
+          console.error(
+            'Only plain objects can be passed to Server Functions from the Client. ' +
+              '%s objects are not supported.%s',
+            objectName(originalValue),
+            describeObjectForErrorMessage(parent, key),
+          );
+        } else {
+          console.error(
+            'Only plain objects can be passed to Server Functions from the Client. ' +
+              'Objects with toJSON methods are not supported. Convert it manually ' +
+              'to a simple value before passing it to props.%s',
+            describeObjectForErrorMessage(parent, key),
+          );
+        }
+      }
+    }
+
+    if (value === null) {
+      return null;
+    }
+
+    if (typeof value === 'object') {
+      // $FlowFixMe[method-unbinding]
+      if (typeof value.then === 'function') {
+        // We assume that any object with a .then property is a "Thenable" type,
+        // or a Promise type. Either of which can be represented by a Promise.
+        if (formData === null) {
+          // Upgrade to use FormData to allow us to stream this value.
+          formData = new FormData();
+        }
+        pendingParts++;
+        const promiseId = nextPartId++;
+        const thenable: Thenable<any> = (value: any);
+        thenable.then(
+          partValue => {
+            const partJSON = JSON.stringify(partValue, resolveToJSON);
+            // $FlowFixMe[incompatible-type] We know it's not null because we assigned it above.
+            const data: FormData = formData;
+            // eslint-disable-next-line react-internal/safe-string-coercion
+            data.append('' + promiseId, partJSON);
+            pendingParts--;
+            if (pendingParts === 0) {
+              resolve(data);
+            }
+          },
+          reason => {
+            // In the future we could consider serializing this as an error
+            // that throws on the server instead.
+            reject(reason);
+          },
+        );
+        return serializePromiseID(promiseId);
+      }
+
+      if (__DEV__) {
+        if (value !== null && !isArray(value)) {
+          // Verify that this is a simple plain object.
+          if ((value: any).$$typeof === REACT_ELEMENT_TYPE) {
+            console.error(
+              'React Element cannot be passed to Server Functions from the Client.%s',
+              describeObjectForErrorMessage(parent, key),
+            );
+          } else if ((value: any).$$typeof === REACT_LAZY_TYPE) {
+            console.error(
+              'React Lazy cannot be passed to Server Functions from the Client.%s',
+              describeObjectForErrorMessage(parent, key),
+            );
+          } else if ((value: any).$$typeof === REACT_PROVIDER_TYPE) {
+            console.error(
+              'React Context Providers cannot be passed to Server Functions from the Client.%s',
+              describeObjectForErrorMessage(parent, key),
+            );
+          } else if (objectName(value) !== 'Object') {
+            console.error(
+              'Only plain objects can be passed to Client Components from Server Components. ' +
+                '%s objects are not supported.%s',
+              objectName(value),
+              describeObjectForErrorMessage(parent, key),
+            );
+          } else if (!isSimpleObject(value)) {
+            console.error(
+              'Only plain objects can be passed to Client Components from Server Components. ' +
+                'Classes or other objects with methods are not supported.%s',
+              describeObjectForErrorMessage(parent, key),
+            );
+          } else if (Object.getOwnPropertySymbols) {
+            const symbols = Object.getOwnPropertySymbols(value);
+            if (symbols.length > 0) {
+              console.error(
+                'Only plain objects can be passed to Client Components from Server Components. ' +
+                  'Objects with symbol properties like %s are not supported.%s',
+                symbols[0].description,
+                describeObjectForErrorMessage(parent, key),
+              );
+            }
+          }
+        }
+      }
+
+      // $FlowFixMe
+      return value;
+    }
+
+    if (typeof value === 'string') {
+      return escapeStringValue(value);
+    }
+
+    if (
+      typeof value === 'boolean' ||
+      typeof value === 'number' ||
+      typeof value === 'undefined'
+    ) {
+      return value;
+    }
+
+    if (typeof value === 'function') {
+      const metaData = knownServerReferences.get(value);
+      if (metaData !== undefined) {
+        const metaDataJSON = JSON.stringify(metaData, resolveToJSON);
+        if (formData === null) {
+          // Upgrade to use FormData to allow us to stream this value.
+          formData = new FormData();
+        }
+        // The reference to this function came from the same client so we can pass it back.
+        const refId = nextPartId++;
+        // eslint-disable-next-line react-internal/safe-string-coercion
+        formData.set('' + refId, metaDataJSON);
+        return serializeServerReferenceID(refId);
+      }
+      throw new Error(
+        'Client Functions cannot be passed directly to Server Functions. ' +
+          'Only Functions passed from the Server can be passed back again.',
+      );
+    }
+
+    if (typeof value === 'symbol') {
+      // $FlowFixMe `description` might be undefined
+      const name: string = value.description;
+      if (Symbol.for(name) !== value) {
+        throw new Error(
+          'Only global symbols received from Symbol.for(...) can be passed to Server Functions. ' +
+            `The symbol Symbol.for(${
+              // $FlowFixMe `description` might be undefined
+              value.description
+            }) cannot be found among global symbols.`,
+        );
+      }
+      return serializeSymbolReference(name);
+    }
+
+    if (typeof value === 'bigint') {
+      throw new Error(
+        `BigInt (${value}) is not yet supported as an argument to a Server Function.`,
+      );
+    }
+
+    throw new Error(
+      `Type ${typeof value} is not supported as an argument to a Server Function.`,
+    );
+  }
+
+  // $FlowFixMe[incompatible-type] it's not going to be undefined because we'll encode it.
+  const json: string = JSON.stringify(root, resolveToJSON);
+  if (formData === null) {
+    // If it's a simple data structure, we just use plain JSON.
+    resolve(json);
+  } else {
+    // Otherwise, we use FormData to let us stream in the result.
+    formData.set('0', json);
+    if (pendingParts === 0) {
+      // $FlowFixMe[incompatible-call] this has already been refined.
+      resolve(formData);
+    }
+  }
+}