Updating the docs for remote / custom / external types

As discussed in #1865, this is how I think we should update our code for
remote / custom / external types:

- Make remote types a first-class feature.
- Make UDL generate blanket ffi trait impls for all UniFfiTags.
  Remove the `use_udl_*` since we don't need them anymore.
- Add `use_remote_type!` to handle the one case where we do need to
  forward ffi traits impls to another crate's tag.
- Use a macro to define custom types.
- Update the UDL external syntax so that `[Extern]` can work with any type.

Benefits are:
 - UDL and proc-macros will be consistent in their handling of UniFfiTag.
 - First-class remote types would enable things like using anyhow::Error in interfaces.
 - The custom type macro is easier for users to use then the current code.
   It allows us to hide the complexity of things like the `UniFffiTag`.
 - External types get a little less hacky.
 - 3rd party crates can provide built-in UniFFI support and implement
   the FFI traits for all their consumers.

CHANGELOG.md

@@ -21,6 +21,16 @@
   No consumers of any languages are impacted, only the maintainers of these language bindings.
   ([#2066](https://github.com/mozilla/uniffi-rs/issues/2066))
 
+### ⚠️ Breaking Changes ⚠️
+
+- Handling of remote/custom/external types has changed significantly:
+    - UDL users need to add the `[Remote]` attribute for remote types
+    - The `UniffiCustomTypeConverter` trait is no longer used.  Instead, use the `custom_type!` macro.
+    - The `use_udl_*` macros are no longer needed and have been removed.
+    - To share remote type implementations between crates, use the `use_remote_type` macro.
+    - The UDL syntax for external types in the UDL has been changed.
+    - See https://mozilla.github.io/uniffi-rs/udl/remote_custom_ext_types.html for details
+
 - The async runtime can be specified for constructors/methods, this will override the runtime specified at the impl block level.
 
 [All changes in [[UnreleasedUniFFIVersion]]](https://github.com/mozilla/uniffi-rs/compare/v0.27.1...HEAD).

docs/manual/src/SUMMARY.md

@@ -16,9 +16,8 @@
     - [Throwing errors](./udl/errors.md)
   - [Interfaces/Objects](./udl/interfaces.md)
   - [Callback Interfaces](./udl/callback_interfaces.md)
-  - [External Types](./udl/ext_types.md)
-    - [Declaring External Types](./udl/ext_types_external.md)
-    - [Declaring Custom Types](./udl/custom_types.md)
+  - [Remote and External Types](./udl/remote_ext_types.md)
+  - [Custom Types](./udl/custom_types.md)
   - [Docstrings](./udl/docstrings.md)
 - [Procedural Macros: Attributes and Derives](./proc_macro/index.md)
 - [Futures and async support](./futures.md)

docs/manual/src/internals/lifting_and_lowering.md

@@ -108,9 +108,10 @@ To work around this we do several things:
  - We generate a unit struct named `UniFfiTag` in the root of each UniFFIed crate.
  - Each crate uses the `FfiConverter<crate::UniFfiTag>` trait to lower/lift/serialize values for its scaffolding functions.
 
-This allows us to work around the orphan rules when defining `FfiConverter` implementations.
- - UniFFI consumer crates can implement lifting/lowering/serializing types for their own scaffolding functions, for example `impl FfiConverter<crate::UniFfiTag> for serde_json::Value`.  This is allowed since `UniFfiTag` is a local type.
+This allows us to work around the orphan rules when defining ffi trait implementations.
  - The `uniffi` crate can implement lifting/lowering/serializing types for all scaffolding functions using a generic impl, for example `impl<UT> FfiConverter<UT> for u8`.  "UT" is short for "UniFFI Tag"
- - We don't currently use this, but crates can also implement lifting/lowering/serializing their local types for all scaffolding functions using a similar generic impl (`impl<UT> FfiConverter<UT> for MyLocalType`).
+ - UniFFI consumer crates usually implement lifting/lowering/serializing types the same way.
+ - However, for remote types, they must only implement ffi traits for their local tag, for example `impl FfiConverter<crate::UniFfiTag> for serde_json::Value`.  This is valid since `UniFfiTag` is a local type.
+ - If other crates also want to use the same remote type implementation, the need to implement the ffi traits for their local tag as well.  This is what the `use_remote_type!` macro does.
 
 For more details on the specifics of the "orphan rule" and why these are legal implementations, see the [Rust Chalk Book](https://rust-lang.github.io/chalk/book/clauses/coherence.html#the-orphan-rules-in-rustc)

docs/manual/src/kotlin/configuration.md

@@ -10,7 +10,7 @@ The generated Kotlin modules can be configured using a `uniffi.toml` configurati
 | `cdylib_name`      | `uniffi_{namespace}`[^1] | The name of the compiled Rust library containing the FFI implementation (not needed when using `generate --library`). |
 | `generate_immutable_records` | `false` | Whether to generate records with immutable fields (`val` instead of `var`). |
 | `custom_types`      | | A map which controls how custom types are exposed to Kotlin. See the [custom types section of the manual](../udl/custom_types.md#custom-types-in-the-bindings-code)|
-| `external_packages` | | A map of packages to be used for the specified external crates. The key is the Rust crate name, the value is the Kotlin package which will be used referring to types in that crate. See the [external types section of the manual](../udl/ext_types_external.md#kotlin)
+| `external_packages` | | A map of packages to be used for the specified external crates. The key is the Rust crate name, the value is the Kotlin package which will be used referring to types in that crate. See the [external types section of the manual](../udl/remote_custom_ext_types.md#kotlin)
 | `android` | `false` | Used to toggle on Android specific optimizations
 | `android_cleaner` | `android` | Use the [`android.system.SystemCleaner`](https://developer.android.com/reference/android/system/SystemCleaner) instead of [`java.lang.ref.Cleaner`](https://docs.oracle.com/en/java/javase/21/docs/api/java.base/java/lang/ref/Cleaner.html). Fallback in both instances is the one shipped with JNA.
 

docs/manual/src/proc_macro/index.md

@@ -465,32 +465,7 @@ pub trait Person {
 // }
 ```
 
-## Types from dependent crates
+## Mixing UDL and proc-macros
 
-When using proc-macros, you can use types from dependent crates in your exported library, as long as
-the dependent crate annotates the type with one of the UniFFI derives.  However, there are a couple
-exceptions:
-
-### Types from UDL-based dependent crates
-
-If the dependent crate uses a UDL file to define their types, then you must invoke one of the
-`uniffi::use_udl_*!` macros, for example:
-
-```rust
-uniffi::use_udl_record!(dependent_crate, RecordType);
-uniffi::use_udl_enum!(dependent_crate, EnumType);
-uniffi::use_udl_error!(dependent_crate, ErrorType);
-uniffi::use_udl_object!(dependent_crate, ObjectType);
-```
-
-### Non-UniFFI types from dependent crates
-
-If the dependent crate doesn't define the type in a UDL file or use one of the UniFFI derive macros,
-then it's currently not possible to use them in an proc-macro exported interface.  However, we hope
-to fix this limitation soon.
-
-## Other limitations
-
-In addition to the per-item limitations of the macros presented above, there is also currently a
-global restriction: You can only use the proc-macros inside a crate whose name is the same as the
-namespace in your UDL file. This restriction will be lifted in the future.
+If you use both UDL and proc-macro generation, then your crate name must match the namespace in your
+UDL file. This restriction will be lifted in the future.

docs/manual/src/udl/custom_types.md

@@ -1,72 +1,95 @@
 # Custom types
 
-Custom types allow you to extend the UniFFI type system to support types from your Rust crate or 3rd
-party libraries. This works by converting to and from some other UniFFI type to move data across the
-FFI. You must provide a `UniffiCustomTypeConverter` implementation to convert the types.
+Custom types allow you to extend the UniFFI type system by converting to and from some other UniFFI
+type to move data across the FFI.
 
 ## Custom types in the scaffolding code
 
-Consider the following trivial Rust abstraction for a "handle" which wraps an integer:
+### custom_type!
+
+Use the `custom_type!` macro to define a new custom type.
 
 ```rust
-pub struct Handle(i64);
+
+// Some complex struct that can be serialized/deserialized to a string.
+use some_mod::SerializableStruct;
+
+/// `SerializableStruct` objects will be passed across the FFI the same way `String` values are.
+uniffi::custom_type!(SerializableStruct, String);
 ```
 
-In this trivial example, the simplest way to expose this is with a macro.
+By default:
+
+ - Values will be converted using `<SerializableStruct as Into<String>>` before being lowered
+ - Values will be converted using `<String as TryInto<SerializableStruct>>` after being lifted.
+ - The `TryInto::Error` type can be anything that implements `Into<anyhow::Error>`
+ - `<String as Into<SerializableStruct>>` will also work, since there is a blanket impl in the core libary.
 
+### custom_type! with manual conversions
+
+You can also manually specify the conversions by passing an extra param to the macro:
+
+```rust
+uniffi::custom_type!(SerializableStruct, String, {
+    from_custom: |s| s.serialize(),
+    try_into_custom: |s| s.deserialize(),
+});
 ```
+
+### custom_newtype!
+
+The `custom_newtype!` can trivially handle newtypes that wrap a UniFFI type.
+
+```rust
+/// handle which wraps an integer
+pub struct Handle(i64);
+
+/// `Handle` objects will be passed across the FFI the same way `i64` values are.
 uniffi::custom_newtype!(Handle, i64);
 ```
 
-Or you can define this in UDL via a `typedef` with a `Custom` attribute,
-defining the builtin type that it's based on.
+### UDL
+
+Define custom types in UDL via a `typedef` with a `Custom` attribute, specifying the UniFFI type
+followed by the custom type.
 
 ```idl
 [Custom]
 typedef i64 Handle;
 ```
 
-For this to work, your Rust code must also implement a special trait named
-`UniffiCustomTypeConverter`.
-
-An implementation is provided if you used the `uniffi::custom_newtype!()` macro.
-But if you use UDL or otherwise need to implement your own:
+**note**: UDL users still need to call the `custom_type!` or `custom_newtype!` macro in their Rust
+code.
 
-This trait is generated by UniFFI and can be found in the generated
-Rust scaffolding - it is defined as:
+## User-defined types
 
-```Rust
-trait UniffiCustomTypeConverter {
-    type Builtin;
+All examples in this section convert the custom type to a builtin type.
+It's also possible to convert them to a user-defined type (Record, Enum, interface, etc.).
+For example you might want to convert `log::Record` class into a UniFFI record:
 
-    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self>
-    where
-        Self: Sized;
-    fn from_custom(obj: Self) -> Self::Builtin;
-}
-```
+```rust
 
-where `Builtin` is the Rust type corresponding to the UniFFI builtin-type - `i64` in the example above. Thus, the trait
-implementation for `Handle` would look something like:
+pub type LogRecord = log::Record;
 
-```rust
-impl UniffiCustomTypeConverter for Handle {
-    type Builtin = i64;
+#[derive(uniffi::Record)]
+pub type LogRecordData {
+    level: LogLevel,
+    message: String,
+}
 
-    fn into_custom(val: Self::Builtin) -> uniffi::Result<Self> {
-        Ok(Handle(val))
+uniffi::custom_type!(LogRecord, LogRecordData, {
+    from_custom: |r| LogRecordData {
+        level: r.level(),
+        message: r.to_string(),
     }
+    try_into_custom: |r| LogRecord::builder()
+        .level(r.level)
+        .args(format_args!("{}", r.message))
+        .build()
+});
 
-    fn from_custom(obj: Self) -> Self::Builtin {
-        obj.0
-    }
-}
 ```
 
-Because `UniffiCustomTypeConverter` is defined in each crate, this means you can use custom types even
-if they are not defined in your crate - see the 'custom_types' example which demonstrates
-`url::Url` as a custom type.
-
 ## Error handling during conversion
 
 You might have noticed that the `into_custom` function returns a `uniffi::Result<Self>` (which is an
@@ -168,12 +191,32 @@ Here's how the configuration works in `uniffi.toml`.
   * `from_custom`: Expression to convert the custom type to the UDL type.  `{}` will be replaced with the value of the custom type.
   * `imports` (Optional) list of modules to import for your `into_custom`/`from_custom` functions.
 
-## Using Custom Types from other crates
+## Using custom types from other crates
 
-To use the `Handle` example above from another crate, these other crates just refer to the type
-as a regular `External` type - for example, another crate might use `udl` such as:
+To use custom types from other crates, use a typedef wrapped with the `[External]` attribute.
+For example, if another crate wanted to use the examples above:
 
 ```idl
 [External="crate_defining_handle_name"]
-typedef extern Handle;
+typedef i64 Handle;
+
+[External="crate_defining_log_record_name"]
+typedef dictionary LogRecord;
 ```
+
+## Remote custom types
+
+Custom types that convert [Remote types](./remote_ext_types.md#remote-types) defined in other crates require special handling.
+
+Specify `remote` param in the `custom_type!` macro:
+
+```rust
+
+uniffi::custom_type!(SerializableStructFromOtherCrate, String, {
+    remote,
+    from_custom: |s| s.serialize(),
+    try_into_custom: |s| s.deserialize(),
+});
+```
+
+To share the custom type implementation for the remote type with other crates, use the [remote_type! macro](./remote_ext_types.md#external+remote-types).