Coordinate Systems (and should we change Bevy's defaults?)

[inodentry]

I am creating this post, to have a consolidated place for any discussion about the tradeoffs/merits/drawbacks regarding different coordinate systems, and what would be best to use in Bevy by default. There have been many scattered discussions about this on Discord, so we clearly need a better place to keep this information.

Status Quo (Bevy 0.5)

(explanation in Bevy Cheatbook)

Currently, Bevy uses a Right-Handed Y-Up coordinate system in all domains: 2d, 3d, UI.

This makes the coordinate system identical/consistent everywhere (for better or for worse).

One upside to this is that the implementations of 2D and UI things can converge / be very similar. They can have the same assumptions, the same rendering, the same camera configuration. One downside is that it can make things non-intuitive and confusing to work with.

When working with UI

This leads to the common confusion/pitfall: that the Y/vertical axis is inverted -- it behaves opposite to common intuition, other toolkits, and the flexbox model (as it behaves in web browsers, where it originates).

When working with 2D

This allows layers to be represented as positive +Z coordinates, with the background at 0. All of the axes behave intuitively.

However, an issue with the current implementation (IDK how much that is directly dependent on the coordinate system per se, but it needs to be addressed) is that the camera needs to be placed far away. The default bundle constructor creates a Transform with Z=9999, close to the clipping plane. It is a common pitfall -- if users override the default transform and don't know that this is necessary.

When working with 3D

The coordinate system matches 2D. This could work well, for example, in side-scrolling platform games that imitate 2D gameplay but use 3D graphics.

However: it is less intuitive for other applications, is inconsistent with important open-source software like Blender, and is different from the typical 3D conventions in other domains (CAD, architecture, mathematics).

Alternative Proposals

To keep the discussion organized/structured, I will create "answer" posts to this thread, for each aspect of the discussion. This should allow people to respond in comments to those sub-posts and make everything less chaotic.

---

Note that, while Bevy does not provide an API to easily change the default coordinate system, it is possible (and seems to work OK) to use other coordinate systems, by creating a camera with a custom projection.

[inodentry]

Proposal: Z-Up for 3D

To quote the arguments made by @Keavon on Discord:

Most 3D games are predominantly horizontal (across the floor or terrain of a world), not vertical, so putting X and Y next to each other allows you to address them easily together and worry about the up dimension as the last dimension. For example, when updating values in Unity for the transforms of objects, I have to click the X entry box, hit tab twice, then change the Z value to move it along the floor. It's an ergonomic annoyance, but it also extends into writing code: you expect the floor to have contiguous coordinates, not X and Z separated by the Y up axis, so it makes coding graphics also less ergonomic. Z is also just the canonical "up" direction in the real life world, it's the direction people refer to and think about. Architecture for centuries, and more recently 3D printing, and pretty much everything except when somebody stared at a monitor mounted in front of them and started working with 2D graphics and then expanded that into the foray into 3D graphics but didn't change their coordinate system to the one that makes sense, conflating screen/normalized device coordinates with world coordinates. Plus there's an argument to be made that, being an open source project, it's preferable to meld with the open source segment of the 3D ecosystem and Blender (sensibly) uses Z up.

I do really think the Z is up and 2D games having their camera point down is best (which is how architecture diagrams, people drawing math on a piece of paper on their desk, etc. already works). But I'm all for choice if it isn't too late to add such fundamental changes. I know in Blender's case, it is an intractable amount of work to support adding a coordinate system setting because of how deeply coupled the entire software is with its coordinate assumptions. If that's not the case with Bevy, I think the ideal case would be providing people a choice of handedness and up-ness (with right-handed Z up as the default!).

Just remember, when you write on a piece of paper with a 2D coordinate system and label your X and Y axes, you are doing so on a table where Z is up. Except when the teacher's writing it on the whiteboard. 2D is ultimately just a fundamentally different concept, but since us humans have this thing called terrain and gravity, we think of lateral movements and up/down movements as very fundamentally separate things, and XY has been the lateral movement dimension for centuries (because it makes sense to keep the coordinates contiguous, X and Y not X and Z).

As far as game engines go, Unity and Godot are the only major engines that use Y as up. (Although Unreal is the real odd-ball out using a left-handed coordinate system even though Z is up, while literally everything else with Z up uses right-handed.) Unreal, CryEngine, and Source (that's all of the popular engines I can think of) use Z up.

I find when I'm programming with 3D, I always mix up coordinates when the engine uses Y up (Unity). In fairness that's partly because I work in Blender more than in Unity. But the other reason is that I innately expect that the up and down direction doesn't insert itself between the two horizontal axes in the coordinate system. On Earth, we have the ground plane and we have the gravity vector, it's such a fundamental thing to human intuition, the up coordinate is fundamentally different from the ground plane coordinates and that's where the intuition comes into play.

[scottmcm]

Might as well add another one:

(From https://twitter.com/FreyaHolmer/status/644881436982575104)

A quick search suggests Vulkan is Y-is-down?

[jpetkau]

Neither Y nor Z is up or down (in the sense this issue is talking about) in OpenGL, DirectX, or Vulkan. These APIs are agnostic to what you use as your world coordinate convention. (They do pick a camera convention, e.g. cameras looking down the -Z axis with Y up in the camera viewport, but that has nothing to do with which coordinates correspond to "the two axes on which you can walk around" vs "the coordinate aligned with gravity").

[TotalKrill]

These pictures are quite interesting. Not 100% accurazy but the trend seems to say that tools primarily focused or originating in 2D workflows ( picture editing, movies, or simple flat games ) seems to prefer the "Y is up".

While project that originated for 3D workflows ( 3D-CAD software and game engines focused on 3D experiences ) seem to prefer "Z is up"

Im only guessing, but if you only have 2 axis, Y tends to represent the "UP" direction, as if the user is looking down on their creations, as if they were on a table.

[Krahos]

In technical drawing, architecture etc, x is length, y is width and z is height, so It comes more natural to me to think this way.
Also in 2D I see nothing wrong to think up as z, or even better let people decide: sometimes when doing 2D you're representing the world as viewed from the side (so z up makes sense), other times you're representing it as viewed from the top (then having x and y makes more sense).
As far as compatibility with other tools is concerned, they usually have a way to export files in other coordinates systems, so I don't think of it as that big of a deal once you've set up your workflow.

[TotalKrill]

Even Autodesk own help for Maya gets confused when using XYZ, and having Z as "depth": https://knowledge.autodesk.com/support/maya/learn-explore/caas/CloudHelp/cloudhelp/2015/ENU/Maya/files/Interface-overview-3D-coordinates-htm.html

[inodentry]

Proposal: Camera facing down in 2D (if using Z-Up for 3D)

If we switch to using Z-Up in 3D, but still want to use the same coordinate system in 2D and 3D, then the camera would have to be rotated to look downwards at the XY plane.

This adds an additional requirement onto the camera transform. This further adds to the issue/pitfall mentioned in the OP -- with requiring users to know how to correctly orient/position the camera, if overriding the default transform.

[inodentry]

Proposal: Using different coordinate systems for 2D (Y-up) and 3D (Z-up)

This could allow the default configuration to be intuitive to work with, in each respective domain.

However, it also means that the coordinate systems are inconsistent, and it further separates 2D and 3D logic and rendering.

[inodentry]

Proposal: Using Y-down in UI and Y-up in 2D

Break the consistency between UI and 2D, to make working with UI match common intuition from other UI toolkits and layout systems.

[minecrawler]

I am all for having a Y-down UI. I don't think there is any major UI framework out there which works otherwise. Since Bevy wants to be able to provide application UI needs (e.g. the editor), I think it would be important to provide the default here. Applications, just like text, usually start at the top and extend downward indefinitely, with the visible area always starting at the top.

For 2D, I am not so sure, though. I think there are many good reasons to have Y-up, but at the same time I think that Y-down might make sense for some games. It would be great to allow choice, but choice might be hard to implement in Bevy...

[inodentry]

Proposal: Make coordinate systems easily configurable

This would let users pick whatever they like / whatever works well for their application.

The drawbacks include:

• possible compatibility issues and confusion due to mismatching/inconsistent coordinate systems in different projects
• Bevy needs to take care to not make internal assumptions about the coordinate system, making development more difficult

[mockersf]

This is already mostly the case for 2D/3D, the only assumptions I can think of are:

• default cameras
• UI placement
• a few methods on Transform (

  bevy/crates/bevy_transform/src/components/transform.rs

  Line 135 in cf221f9

  pub fn left(&self) -> Vec3 {

  and friends)

[inodentry]

Yeah, as I said in the OP:

Note that, while Bevy does not provide an API to easily change the default coordinate system, it is possible (and seems to work OK) to use other coordinate systems, by creating a camera with a custom projection.

You can do it, but it's definitely unsupported/unofficial territory.

The question here is whether this should have the "official seal of approval", by making it project policy to always have consideration for these use cases, and ideally providing a nice/easy API for users to choose the coordinate system they want.

[mockersf]

I think having the camera you want in 3d is very supported
a Z-up left handed camera? here:

PerspectiveCameraBundle {
    transform: Transform::from_translation(-Vec3::Y)
        .looking_at(Vec3::ZERO, Vec3::Z),
    ..Default::default()
}

[scottmcm]

Assertion: There is no truly intuitive coordinate system

Instead, there's only "what someone is already used to". The might be Y-up from using OpenGL. That might be left-handed from DirectX. That might be Y-down from UIs. Etc. But they all work, and anyone importing models from different places may well need to deal with the mismatch no matter what Bevy uses.

[CharlesWoodhill]

Voting for "right hand" and Z up

i personally never understood why the "industry" stick to that 2 "systems", although i know why they there (historically).
and that they want to be "compatible" with at least some of some :)
in my opinion both are not intutive.

i dont like the terms left/right hand.
for me a cordinate system has only 1 property: are the axis clockwise or coutner clockwise.
So when using the picture of mokersf left hand => clockwise, right hand => counter clockwise.

it has effects even down into for example culling (triangle culling clockwise/counter clock)
and god knows where else but idk (matrices?, lookAt?, mesh import?,...)

the second property often used is: what axis points up,
in my opinon this is no axis property, its a property of the camera.
afaik, opengl/directx/vulkan dont care about that.

but its the thing that i think is most confusing, because in my mindset z is up!
but this can be (easily?) "customized" by a upvector on the camera (engine camera AND the editor camera),
i really dont care if then x or y is front/right, but it would be neat to have that customizable too.
ofc things like hardcoded skyboxes need an upvector prop then,
which anyway would be cool if you think a planet is a sphere and not a plate :)

I asked some people (who where not yet polluted by existings (programmesrs,artists...) ) how they would do the coordinates.
And, as i expected they almost all said,
X = (+)forward / (-)backward
Y = (+)right / (-)left
Z = (+)up / (-)down
which results in a "left hand" coordinate system.

but this doesnt work well with 2d, i also think 3d should be 2d + Z-axis
which requires "right hand" to have x pointing right and have a positive height (Z).

thats why i vote for "right hand" and Z up

and let the "identity"-rotation look along the positive y axis.
then ye could also get rid of that "the camera looks into the negative z axis" thing
which produces confusing logic ive often seen in some places (example: copied from a random camera script) :
(transform is cameras transform)

    let local_z = transform.local_z();
    let forward = -Vec3::new(local_z.x, 0., local_z.z);
    let right = Vec3::new(local_z.z, 0., -local_z.x);

could be?:
let forward = transform.local_y;
let right = transform.local_x;

[kvfreedom]

As a game engine, Y-up + "left hand" is the best.
2D game: X-axis left/right, Y-axis jump/fall.
3D game: X-axis left/right, Y-axis jump/fall, Z-axis forward/backward

[Mokuzzai]

+1 for rh Z-up

I've always thought 3D as "first two describe floor position and the last one describes height". The floor coordinates are similar so should go next to each other while the height is more special (humans move on the floor more easily than vertically). I think code tends to be nicer with rh Z-up.

Also I don't think using g the argument: "3D should be 2D + 3rd axis" doesn't work for a few reasons:

1. There are two main types of 2D games: top-down or Z-up and sidescroller or Y-up (if we are talking about screenspace coordinates). So if we simply add a third dimension we end up with two different 3D games: Y-up and Z-up.
2. 3D and 2D games are inherently different. Most 2D games have their camera aligned with the Z-axis looking at the world from the outside the 2D plane, while 3D games have their camera inside the 3D world, I've yet to see a 3D game where the camera is along the W-axis (and I've only seen like one 2D game where the camera is inside the world)

[TotalKrill]

I agree, and something I noticed when working with an application that switches between representing the world in 2D and 3D constantly.

Think build mode and then action gameloop etc, I have to rotate 2D objects I wish to overlay in the world etc. Due to the default of x,y meaning movement on the screen in 2D, and then in 3D that is represented in x-z.

And then when I have UI elements, again there's another coordinate system, I have to keep track of.

It's honestly a bit of a mess, and I feel that had it been more coherent, I would have had more oversight with whatever way was chosen, currently it feels a bit all over the place.

Before even any start could be done, since this would now be a massive effort. Someone with some kind of say so would have to give an indication that it would be a worthwhile change.

[EmiOnGit]

I want to give my insights with using the current system for 2d top-down games.

To start off, the current system is not great because using z for layering is not intuitive and makes a lot of problems.

To name problems I encountered:

y-ordering of sprites is a common need for top down games. Currently, this is only possible by updating the z component constantly. If you would want to introduce layergroups, such as a background and foreground layer, the current system would become even more troublesome.

I also noticed that the current system plays badly with other crates that rely on the transform component.
For example, I use bevy_rapier_2d for collisions. The colliders are behaving weirdly whenever changing the z layer directly. While this is no direct concern for bevy, it shows that it is easy to mess that up.

I'm not really sure how exactly a solution would look like but I'd love to get rid of the z for layering.
A component like

struct Layer {
    layer_group: u8,
    layer: f32
}

[micycle8778]

I agree with this. I recently ran into a bug when making a Unity game stemming from a 3d transform being used in a 2d context.
When drawing a vector from one entity to another, the z axis can sneak up on you and cause somewhat unintuitive logic errors.

[JoaoVictorVP]

Vote for Y as up-down axis, because Y on the screen looks like an arrow pointing downwards and because it does not matter in the end, and whatever it is changed will piss people off: so why not just let it go as it is then?
In the end, if you want to invert Y with Z you can literally make a new type and replace it when using transform, the cost will be negligible (and probably will be optimized away when used in place) and nobody would care.

[marstaik]

I am in agreement with the fact that the coordinate system should be configurable by the user. There is no "best" coordinate representation globally, but there is a "best" coordinate representation for the task at hand.

Maybe I can give some background in my experience on how coordinate spaces are handled.
There are generally two ways games/programs/etc define world coordinate systems:

1. You define only Left Hand/Right Hand and an Up Vector (or inversely gravity vector)

• Thus because the notion of "world forwards" is not constrained, there are technically infinite isometries that work for this definition (any arbitrary rotation about the up vector holds, in addition to translation, but we can ignore translation for the most part). In practice, there are only 4 sane possibilities (one of the other 2 axes and their negations arbitrarily defined as forwards) in order to have no precision loss at a mathematical level.
• The program (bevy in this case) cannot define any notion of Forward, Left, Up in Transform classes, because we have only one constraint.
• Importing data is always ambiguous, as we have no world alignment between the import format and the application

2. You define Left/Right Hand and an Up AND Forward Vector

• You now have a fully "constrained" world space. There is only one possibility for the world space alignment. You either cross Up and Forward to get Left for a Right Handed System, or you cross Up and Forward to get Right for a Left Handed System.
• Imports from standards that have a well-defined coordinate system are unambiguous.
  • GLTF is +Z Forwards, +Y Up
  • Blender is -Y Forwards, +Z Up (you can determine what blender "forwards" is by entering front orthographic view hitting Numpad 0
  • For these well-defined cases, we have a unique transform from their space to our application space
• The program (bevy in this case) can define "Forward, Left, Up" vectors, but most likely not in the Transform class itself, as these vectors can no longer be pulled trivially from their column vectors (not to mention sign change). These might need to be in some other Resource struct.

Just for clarity, I want to provide some context on cameras and how to potentially implement custom coordinate spaces on bevy side with the least amount of friction possible.

Traditionally, we have the 4 spaces:

1. Model/Object space: The coordinates system used when making the model.
   • Local coordinate space of the object (think raw mesh vertices)
2. World space: The coordinates in the world.
   • This is the matrix that is represented by GlobalTransform. It is the "world-position" representation.
3. Camera space: The coordinates from the point of view of the camera
   • This is the matrix that represents the world from the point of view of the camera
   • It is aligned with the representation your back-end driver expects (usually either -Z forwards for 3D graphics applications, or +Z for image manipulation/computer vision applications)
   • This is where you can inject your custom coordinate system math
4. Screen space (Sometimes called "Window space" or "Device space"): The coordinates for the screen.

Now there are two (common) approaches to get to Camera Space.

Camera Local space is tied to your Renderer Implementation

The first, is the technique Blender & GLTF uses (and others). In these programs world coordinate spaces, the cameras are defined with a local transform (Object space) different than that of their world space, in this case OpenGL convention where cameras look down (forward vector) is -Z.

To further clarify this point, if you add in an "Identity" camera in Blender, it will point "straight down". This is because the local convention of the camera is to look down -Z, and world up is +Z.

Then, to get to camera space, you perform:

view_matrix = camera_world_space.inverse();
camera_space = view_matrix * projection_matrix;

In this case, the view matrix is simply the inverse of the cameras world matrix. The math is simple.
However, there is a giant pain point/ergonomics problem when it comes to this approach - the cameras world matrix, and therefore the world matrix basis vectors, do not match the cameras forward vectors.

If we look at the blender image as an example, we can say that blender is -Y Forwards, +X Left, +Z Up.
Given an identity matrix, the forward, left, up getters would look like this:

let world_rot =
[1 0 0]
[0 1 0]
[0 0 1]

forwards() -> -world_rot .column[1] //  -Y
left() -> world_rot .column[0] // +X
up() -> world_rot .column(2) // +Z

However, look at the image above! The camera in blender is at the identity, and it looks down -Z! Our call to .forwards() on the transform on this camera would not give us the correct forwards vector of the actual camera! 😱

If we wanted to support a user-provided world space coordinate system, we would need to apply a change of basis to the view matrix on the left hand side, such that the camera is "framed correctly" in our custom system. The math for this isn't difficult per-say, but perhaps not as intuitive. The second approach also must do this, but the change of basis you provide is different.

change_of_basis = Mat4[...] // Change of basis for our custom coordinate system that maps the space from a `-Z` forwards in bevy frame, to a `-Z` forwards in our frame
let view = change_of_basis * camera_world_space.inverse();
camera_space = view_matrix * projection_matrix;

Camera Local space is tied to your World Space

This is a technique where your camera local space is exactly the same as your world system coordinate conventions.

If Blender was theoretically in this mode, then your camera at the identity would look like this:

The ergonomics of this make a bit more sense for 3D programming - the coordinate system of your world and the coordinate system of your camera match. It means that for generic transform code, the transform of your camera and the transform of your models have the same symbolic meaning when it comes to forward vector. If you look at the image above, the identity matrix of the camera, and the identity matrix of the model both have them "looking" the same direction.

The camera space math then looks like this:

change_of_basis = Mat4[]; // The matrix that takes you from your world space, to the renderers camera space (-Z) forwards
view_matrix = change_of_basis * camera_world_space.inverse()
camera_space = view_matrix * projection_matrix

Ignoring custom world spaces for a second, you may see that there is now always a change of basis matrix, where the previous method did not! That is because our cameras are defined in our world space, not the renderers. Before we give it to the renderer, we must put it into the space it understands!

However, with a custom coordinate space, you would only have to be concerned with changing that single change of basis, not caring about the world space convention used by Bevy/User code.

As an example, according to https://bevy-cheatbook.github.io/fundamentals/coords.html, Bevy uses -Z forwards for the world space.
Note that this is different than GLTF space, which formally defines +Z as world forwards.

This means that bevy's world space is the exact same as renderer space, so identity transform!

Thus, the change of basis matrix for bevy's current coordinate system would be the identity.
That would look like this:

change_of_basis = Matrix::identity();
view_matrix = change_of_basis * camera_world_space.inverse();
camera_space = view_matrix * projection_matrix;

However, let's say you wanted to use a more mathematical/ISO defintion of world space, such that your up vector is +Z, forwards is +X, and left is +Y:

world_rot =
[ 1 0 0 ]
[ 0 1 0 |
[ 0 0 1 ]

forwards() -> world_rot .column[0] //  +X
left() -> world_rot .column[0] // +Y
up() -> world_rot .column(2) // +Z

Then, to convert from our world space convention to renderer convention, our change of basis from +X F/+Z U to -Z F/+Y U looks like this:

// homogenous
X_FORWARD_TO_NEG_Z_FORWARD = [
   0, -1,  0,  0,
   0,  0,  1,  0,
  -1,  0,  0,  0,
   0,  0,  0,  1,
];

change_of_basis = X_FORWARD_TO_NEG_Z_FORWARD;
view_matrix = change_of_basis * camera_world_space.inverse();
camera_space = view_matrix * projection_matrix;

How would I do this in Bevy?

IIRC I believe Bevy is currently both #1 and #2 at the same time. It defines the camera convention as world space, and that world space happens to be renderer space as well (-Z forwards). I believe the most sane approach would be to go down the #2 path and provide users with the ability to change the basis of the view matrix.

Thus, I think the cleanest solution would be something along the lines of removing directional getters in Transforms (left, up, forwards, right, etc) and all dependencies on them (they shouldn't be needed at all IMO).

Introducing a Resource such as WorldConvention that allows you to provide the change of basis from the now arbitrary world space of bevy to the renderer's convention of camera space -Z forwards.

The WorldConvention resource should be used during import pipelines to correctly pre-bake the convention used by the application on imports such as GLTF.

Using this resource when computing the view matrix.
Everything should just magically work.