Bevy Text rework: TextBlock

[UkoeHB]

Continued from the Lorum Ipsum discord working group. This is a proposal for a new text API using required components and replacing Vec<TextSection> with a hierarchy of text entities.

See cart's comment for the current proposed design. Most of the discussion below remains relevant.

Objectives

• Improve the ergonomics of spawning basic one-line text.
• Break Text into separate components to take advantage of the new required components API. Deprecate existing text bundles.
• Facilitate text picking, including picking for regions of text that contain multiple sub-spans (e.g. a hyperlink that is partially bolded).
• Open a path to building text blocks that contain non-text elements (e.g. images, UI sub-trees, etc.) that are controlled by the text layout algorithm.
• Increase flexibility of constructing and managing complex text blocks. For example, bevy_mod_bbcode uses a custom multi-entity solution for text.

Background

Right now we use cosmic-text to control text layout. We feed cosmic-text an iterator over text sections, and it constructs a Buffer that knows how to organize the text contents. The buffer is then used to construct a TextLayoutInfo which has all the glyphs and their positions for rendering text in Bevy (both UI and 2d text).

When doing UI layout with taffy, a node with text on it is treated as having an opaque 'content block' (see the ContentSize component). Whenever a node has a content block, taffy asks the block for its computed size using different x/y bounds as part of the taffy layout algorithm (CSS grid/flexbox). For a text node, you compute that size by using the text's Buffer to recompute text layout.

Summary

Currently we store text sections as Vec<TextSection> in a single Text component that has all text layout and style information.

In this proposal, we break Text into a tree of text nodes. At the root of the tree are the following components:

• TextLayout: controls wrapping/line breaking
• TextBlock: contains the cosmic text buffer and caches the list of text section entities
• TextSpan: indicates the node should be tracked as part of text tree collection

Any node of the tree underneath a TextBlock (including the entity with TextBlock) can include the following components (if a child doesn't have TextSpan then we don't traverse into that child - only contiguous TextSpan nodes are considered):

• Text: contains a string
• TextStyle: font, font size, color
• TextSpan: indicates the node should be tracked as part of text tree collection

To delineate between UI and 2d we have:

• TextNode: marker component for UI text, requires components for setting up a UI node
• Text2d: marker component for 2d text, requires components for setting up a 2d node

The final bit of magic to make this ergonomic is using TextNode::new() and Text2d::new() to produce bundles instead of making just the components. This introduces a small learning curve but makes everything fit together nicely.

API

#[derive(Component)]
pub struct TextSpan;

#[derive(Component)]
pub struct TextLayout
{
    pub justify: JustifyText,
    pub break: BreakLineOn,
}

#[derive(Component)]
#[requires(TextStyle, TextSpan)]
pub struct Text(String);

impl Text
{
    pub fn new(text: impl Into<String>) -> Self
    {
        Self(text.into())
    }
}

#[derive(Component)]
pub struct TextStyle
{
    pub font: Handle<Font>,
    pub size: f32,
    pub color: Color,
}

pub struct TextEntity
{
    entity: Entity,
    /// Records the hierarchy depth of the entity within a `TextBlock`.
    depth: usize,
}

#[derive(Component)]
#[requires(TextLayoutInfo, TextLayout, TextSpan)]
pub struct TextBlock
{
    /// Buffer is private because buffer contents are always refreshed from ecs state when writing glyphs to
    /// `TextLayoutInfo`. If you want to control the buffer contents manually or use the `cosmic-text`
    /// editor, then you should manually implement the conversion to `TextLayoutInfo`.
    pub(crate) buffer: CosmicBuffer,
    pub(crate) entities: SmallVec<[TextEntity; 1]>,
    pub(crate) needs_recompute: bool,
}

impl TextBlock
{
    pub fn entities(&self) -> &[TextEntity]
    {
        &self.entities
    }
}

/// Renamed from `TextFlags`.
#[derive(Component)]
pub struct TextNodeFlags
{
    needs_measure_fn: bool
}

#[derive(Component)]
#[requires(TextBlock, TextNodeFlags, ContentSize, Node)]
pub struct TextNode;

impl TextNode
{
    // Returns a bundle for setting up UI text.
    pub fn new(text: impl Into<String>) -> (Self, Text)
    {
        (Self, Text::new(text))
    }
}

#[derive(Component)]
#[requires(TextBlock, TextBounds, Spatial2d)]
pub struct Text2d;

impl Text2d
{
  // Returns a bundle for setting up 2d text.
  pub fn new(text: impl Into<String>) -> (Self, Text)
  {
    (Self, Text::new(text))
  }
}

// 1.a Simple UI text node.
//
// Node, Buffer ["Hello"]
commands.spawn(TextNode::new("Hello"));

// 1.b Simple 2d text node.
//
// Buffer ["Hello"]
commands.spawn(Text2d::new("Hello"));

// 2.a UI multi-span text.
//
// Node, Buffer ["What a ", "wonderful ", "world"]
commands
  .spawn(TextNode::new("What a "))
  .with_children(|parent| {
    parent.spawn(Text::new("wonderful "));
    parent.spawn(Text::new("world"));
  });

// 2.b 2d multi-span text.
//
// Buffer ["What a ", "wonderful ", "world"]
commands
  .spawn(Text2d::new("What a "))
  .with_children(|parent| {
    parent.spawn(Text::new("wonderful "));
    parent.spawn(Text::new("world"));
  });

// 2.c UI multi-span text expanded from markup "[b]Hello, [i]World[\i][\b]"
//
// Node, Buffer ["Hello, ", "World"]
commands
  .spawn(TextNode)
  .with_children(|parent| {
    parent.spawn((
        Text::new("Hello, "),
        TextStyle::from_font(fonts.get(FontFamily::new("Fira Sans").bold()))
    ));
    parent
        .spawn(TextSpan)
        .with_children(|parent| {
          parent.spawn((
              Text::new("World"),
              TextStyle::from_font(fonts.get(FontFamily::new("Fira Sans").bold().italic()))
          ));
  });

// 3. Multi-node UI
//
// Node
// - Node, Buffer ["Hello"]
// - Node, Buffer ["World"]
commands
  .spawn(Node)
  .with_children(|parent| {
    parent.spawn(TextNode::new("Hello"));
    parent.spawn(TextNode::new("World"));
  });

// 4. (Invalid, no TextBlock. Warning emitted.)
commands
  .spawn(Text::new("Hello"))
  .with_child(|parent| {
    parent.spawn(Text::new("World"));
  });

Implementation details

• When Text or TextStyle changes on entity, iterate upwards to find TextBlock then set its needs_update flag. This flag is used by text2d and UI text to decide if the buffer should be rebuilt.
• In TextPipeline::update_buffer, instead of taking in a &[TextSection], take in impl Iterator<Item = (&Text, &TextStyle)>. This iterator should internally traverse the entity hierarchy and rebuild the TextBlock::entities vector in-line. This is an effective way to keep the entities vector accurate without any extra hierarchy traversal/management code.
• For text picking, use the text section already stored in TextLayoutInfo. The section index can be used with TextBlock::entities to look up the correct entity for picking events.
• Add an OnRemove observer for Text that sets the TextBlock::needs_update flag on the nearest TextBlock ancestor.
• In this proposal CosmicBuffer is embedded in TextBlock and should not be edited by users since any edits will be overwritten by the internal systems that filter for TextBlock. However, buffer construction, ContentSize measurements, and TextLayoutInfo assembly will be opt-in via the TextBlock, TextNode, and Text2d components. A user can instead manually construct and update ContentSize and TextLayoutInfo by adding a custom component/code for managing cosmic-text buffers, and simply not including those opt-in components. The TextPipeline should be refactored a bit to facilitate this (split up TextPipeline::queue_text so you can construct TextLayoutInfo without needing bevy-specific text sections/blocks).
  • UI: you'd need Node, ContentSize, TextLayoutInfo, CustomCosmicBuffer + code to manage these
  • 2d: you'd need Spatial2d, TextLayoutInfo, CustomCosmicBuffer + code to manage these

Open questions

• On taffy, Visibility, and Transform traversal, we want to ignore the child Text entities. One solution would be a SkipChildren marker component required by TextBlock. Or perhaps SkipChildren<T> and then require SkipChildren<Node>, SkipChildren<Visibility>, SkipChildren<Transform>.

Alternatives

• @viridia has proposed adding a Display::Flow setting to the UI Style which would have behavior similar to TextBlock.
• @cart has proposed only having a Text component, and auto-detecting the root node of text blocks (possibly with Display::Flow playing a part).

Problems with the alternatives:

• Auto-detection of block roots means all uses of Text need to be aware of each other for proper auto-setup (e.g. 2d and UI).
• The behavior of Display::Flow is also desirable for 2d.
• The behavior we need is satisfied by required components so it would be weird to add special magic to get that for text.
• We don't always want Text on all the nodes of a text block. This isn't a hard blocker to the alternatives since they could use TextSpan like I do in this proposal.
• Display::Flow would increase the complexity of Style, which is already quite overly-complected.

[alice-i-cecile]

My core criteria here:

• text trees are trees of entities
• spawning a single section of text is very easy
• mixed-formatting paragraphs and rich text are possible
• markdown-style text formatting is feasible to add on top
• it's easy to swap font, font style (bold / italic) and font color

Overall, this proposal meets my requirement and is well reasoned and explained!

I prefer the distinct components over a monolithic Text component: I think it's easier to reason about / makes for clearer APIs.

I think I'd prefer using Node + Text over a dedicated TextNode marker component, but I really don't feel strongly. The lack of symmetry with Text2d is annoying, so I can see why you chose the design you did.

The behavior of Display::Flow is also desirable for 2d.

Agreed: that's the strongest argument against it. I would also really like to not make Style even more complex.

[UkoeHB]

I think I'd prefer using Node + Text over a dedicated TextNode marker component, but I really don't feel strongly.

With required components it's necessary to have a dedicated type for pulling in components for the root node.

[UkoeHB]

Looking for sign-off from the following contributors before attempting an implementation:

• @cart
• @alice-i-cecile
• @viridia
• @rparrett
• @nicoburns
• @ickshonpe
• @Olle-Lukowski

[alice-i-cecile]

@rparrett for bestRanar :)

[cart]

I’ll definitely want to weigh in here when I get back. I’ll be back at work tomorrow

[ickshonpe]

Intuitively entity-per-text-section has always seemed the right direction, my worry with this design is that it could be very awkward to manage the text sections after they've been spawned (unless you just despawn the whole text tree and spawn a new one every time you make a change).

Like from the example:

commands
  .spawn(TextNode)
  .with_children(|parent| {
    parent.spawn((
        Text::new("Hello, "),
        TextStyle::from_font(fonts.get(FontFamily::new("Fira Sans").bold()))
    ));
    parent
        .spawn(TextSpan)
        .with_children(|parent| {
          parent.spawn((
              Text::new("World"),
              TextStyle::from_font(fonts.get(FontFamily::new("Fira Sans").bold().italic()))
          ));
  });

Suppose that you want to change the text to only display the word "World". Ideally you'd just want to be able to delete the "Hello, " text entity. But it's the head of the tree so that would break the text hierarchy. You could despawn the "World" entity, and change the "Hello, " string to "World" but then you also have to remember to update the font, and you have to understand the structure of the text tree and locate the world-text entity.

And that's only a very trivial construction. It's not very clear to me how to make the ergonomics as simple as with a vec of sections.

[nicoburns]

It's not very clear to me how to make the ergonomics as simple as with a vec of sections.

My take it that we shouldn't be optimising this for ergonomics, because I don't think it's possible to make an API that involves manually spawning entities ergonomic for text. We should optimise for implementation convenience, and convenience as a target format for macros, DSLs, and crates which I would expect game authors to actually be using.

[UkoeHB]

A big reason I went for hierarchy-based instead of a flat list is to unify the 'single section' and 'multi-section' use-cases. With single section you want to just spawn one entity that holds the buffer, while with multi-section you want to spawn multiple child entities that are collected into the buffer.

Then there's the important transition between single and multi-section, when a user goes from having one section to suddenly wanting to add more. With this proposal it's trivial - just add a new child with Text.

If we wanted a flat list then I'd recommend splitting into two separate root node types: Text (single-entity) and then separately TextBlock (one root node with no text component, and a flat list of children with TextSection). Then there would be Text2d, TextNode, TextBlock2d, and TextBlockNode.

[cart]

TextSpan / TextStyle / TextLayout

I'm thinking the proliferation of components might be overly complicated. I propose we merge TextLayout into TextBlock for conceptual simplicity.

pub break: BreakLineOn,

Sadly break is a reserved word so this won't be possible. I propose a rename of the type to something like LineBreak and for the field to match that (ex: line_break).

The final bit of magic to make this ergonomic is using TextNode::new() and Text2d::new() to produce bundles instead of making just the components. This introduces a small learning curve but makes everything fit together nicely.

I'm pretty strongly in opposition to this, as it makes this all needlessly complicated. It essentially makes a "one to many" constructor required for reasonable ergonomics, which I consider to be unexpected / unidiomatic in the new scene system / a missed ergonomic opportunity. I believe this approach came from a desire to share Text across contexts, which is totally understandable. But I don't think that is particularly helpful or necessary, relative to keeping init straightforward / predictable / ergonomic. I think we can simplify this by creating a new text component type for each context.

I do like your proposal to make the "cosmic data" / buffer component explicitly encoded via required components, rather than implicitly computing what needs it at runtime.

I propose the following, which I think will make this all fit more pleasantly in peoples' heads:

#[derive(Component)]
pub struct TextStyle {
    pub font: Handle<Font>,
    pub size: f32,
    pub color: Color,
}

/// This is always an (empty by default) "final destination" for text. It is where you "put" spans of text,
/// it is not a span of text itself.
#[derive(Component)]
#[requires(TextLayoutInfo, ComputedTextBlock)]
pub struct TextBlock
{
    pub justify: JustifyText,
    pub line_break: LineBreak,
}

#[derive(Component)]
pub struct ComputedTextBlock {
    pub(crate) buffer: CosmicBuffer,
    pub(crate) entities: SmallVec<[TextEntity; 1]>,
    pub(crate) needs_recompute: bool,
}

/// This is the top-level UI text component. If a string is specified, it behaves as if it has a "first" TextSpan child.
#[derive(Component)]
#[require(TextBlock, TextStyle, Node, ContentSize, TextNodeFlags)]
pub struct Text(String);

/// This is the top-level 2D text component. If a string is specified, it behaves as if it has a "first" TextSpan child
#[derive(Component)]
#[require(TextBlock, TextStyle)]
pub struct Text2d(String);

/// This will contribute its text to the parent TextBlock (or ancestor in a TextSpan tree leading to a TextBlock)
#[derive(Component)]
#[require(TextStyle)]
pub struct TextSpan(String);



// 1.a Simple UI text node.
//
// Node, Buffer ["Hello"]
commands.spawn(Text("Hello"));

bsn! {
    Text("Hello")
}

// 1.b Simple 2d text node.
//
// Buffer ["Hello"]
commands.spawn(Text2d("Hello"));

bsn! {
    Text2d("Hello")
}


// 2.a UI multi-span text.
//
// Node, Buffer ["What a ", "wonderful ", "world"]
commands
  .spawn(Text("What a "))
  .with_children(|parent| {
    parent.spawn(TextSpan("wonderful "));
    parent.spawn(TextSpan("world"));
  });

bsn! {
    Text("Hello") [
        TextSpan("wonderful "),
        TextSpan("world"),
    ]
}


// 2.b 2d multi-span text.
//
// Buffer ["What a ", "wonderful ", "world"]
commands
  .spawn(Text2d("What a "))
  .with_children(|parent| {
    parent.spawn(TextSpan("wonderful "));
    parent.spawn(TextSpan("world"));
  });

bsn! {
  Text2d("What a ") [
    TextSpan("wonderful "),
    TextSpan("world"),
  ]
}

// 2.c  Simple multi-span UI text
//
// Text, Node, Buffer ["Hello, ", "World"]
commands
    .spawn(Text::default())
    .with_children(|parent| {
        parent.spawn(TextSpan("Hello, "));
        parent.spawn(TextSpan("World, "));
    });

bsn! {
    Text [
        TextSpan("Hello, "),
        TextSpan("World, "),
    ]
}

// 2.d UI multi-span text expanded from markup "[b]Hello, [i]World[\i][\b]"
//
// Text, Node, Buffer ["Hello, ", "World"]
commands
  .spawn(Text::default())
  .with_children(|parent| {
    parent
        .spawn((
            TextSpan("Hello, "),
            TextStyle {
                font: fonts.get(FontFamily::new("Fira Sans").bold(),
                ..default()
            })
        ));
    parent
        .spawn(TextSpan::default())
        .with_children(|parent| {
          parent.spawn((
              TextSpan("World"),
              TextStyle {
                font: fonts.get(FontFamily::new("Fira Sans").italic())
              },
          ));
  });

bsn! {
    Text [
        (TextSpan("Hello, "), TextStyle { font: {fonts.get(FontFamily::new("Fira Sans").bold()} })
        TextSpan [
            (TextSpan("World"), TextStyle { font: {fonts.get(FontFamily::new("Fira Sans").italic()} }
        ]
    ]
}

// 3. Multi-node UI
//
// Node
// - Node, Buffer ["Hello"]
// - Node, Buffer ["World"]
commands
  .spawn(Node)
  .with_children(|parent| {
    parent.spawn(Text("Hello"));
    parent.spawn(Text("World"));
  });

bsn! {
    Node [
        Text("Hello"),
        Text("World"),
    ]
}

// 4. Invalid, no TextBlock. Warning emitted.
commands
  .spawn(TextSpan("Hello"))
  .with_child(|parent| {
    parent.spawn(TextSpan("World"));
  });

bsn! {
    TextSpan("Hello") [
        TextSpan("World")
    ]
}

// 5. Valid: This is a normal text node with a child Text node. They are each their own TextBlock
commands
  .spawn(Text("Hello"))
  .with_child(|parent| {
    parent.spawn(Text("World"));
  });

bsn! {
    Text("Hello") [
        Text("World")
    ]
}

[UkoeHB]

Thanks @cart, these look like good changes to me.

One issue I can see is bevy_text will need to have an optional dependency on bevy_ui in order to pull in the Text component for assembling cosmic buffers (is this a circular dependency that will cause problems?). That was part of the motivation behind the original separation between TextNode/Text2d and Text/TextSpan - so text sections can be assembled independent of context.

[viridia]

Thanks @cart, these look like good changes to me.

One issue I can see is bevy_text will need to have an optional dependency on bevy_ui in order to pull in the Text component for assembling cosmic buffers (is this a circular dependency that will cause problems?). That was part of the motivation behind the original separation between TextNode/Text2d and Text/TextSpan - so text sections can be assembled independent of context.

Move the Text component into bevy_ecs? There's not much there, other than TextSection, which is going away.

[UkoeHB]

Move the Text component into bevy_ecs? There's not much there, other than TextSection, which is going away.

Required components bring in Node and ContentSize.

[cart]

Text should be able to live in bevy_ui. Ex: if we can write "text update" systems in bevy_text that are generic on "a component that provides a string", then we can just register the proper instance of that system for Text in bevy_ui. All of the other components should be able to live in bevy_text.