Color: Add 'colorSpace' argument for getters/setters

[donmccurdy]

Related

• Color Management: Documentation #23430 (edited)
• Add THREE.ColorManagement #22346
• WebGLRenderer: Set sRGBEncoding outputEncoding by default #22275
• Washed out colors w/ color-encoding despite conversion #19169
• Enhancement: add renderer.outputEncoding #11337

Description

Certain ways of expressing colors — hexadecimal and CSS strings — can safely be assumed to be in the sRGB color space. Hexadecimal syntax implies sRGB by convention, and CSS colors are always sRGB except where otherwise specified in the (very recent) CSS Color Module Level 4.

However, colors passed to three.js shader must be Linear-sRGB. Because hexadecimal and CSS colors are not (by convention or specification) in the required color space, I believe we should take the step of converting them from sRGB to Linear-sRGB on the user's behalf. Importantly, this ensures that hexadecimal colors from a color picker, or CSS colors from a stylesheet, can be used directly in three.js APIs. They will appear correctly without manual conversion, assuming .outputEncoding = sRGBEncoding is also set.

The changes provided by this PR are disabled by default (ColorManagement.legacyMode = true), but can be enabled by default (in a future release) at the same time as we change the default value of renderer.outputEncoding to sRGBEncoding, minimizing impact of the output encoding change on the remaining users who haven't yet enabled it.

// current defaults
THREE.ColorManagement.legacyMode = true;
renderer.outputEncoding = LinearEncoding;

// future defaults
THREE.ColorManagement.legacyMode = false;
renderer.outputEncoding = sRGBEncoding;

As discussed in #23614, .outputEncoding should perhaps also be renamed to .outputColorSpace at this time.

RGB and HSL components (e.g. color.r) continue to be Linear-sRGB ("Linear"), and can be manipulated exactly as before. I understand this may seem like an inconsistency, but it's exactly how Blender's color picker works as well, and ensures that lerping and keyframe animation continue to execute in Linear-sRGB color space.

// No change for RGB or HSL input.
color.r = color.g = color.b = 0.5;
color.setRGB( 0.5, 0.5, 0.5 );
color.setHSL( 0.0, 1.0, 0.5 );
console.log( color.r ); // → 0.5

// CSS input is sRGB by specification, and converted to Linear-sRGB by this PR.
color.setStyle( 'rgb( 127 127 127 )' );
console.log( color.r ); // → 0.214041140

// Hex input is sRGB by convention, and converted to Linear-sRGB by this PR.
color.setHex( 0x808080 );
console.log( color.r ); // → 0.214041140

// Default input color space can be overridden.
color.setStyle( 'rgb( 127 127 127 )', LinearSRGBColorSpace );
color.setHex( 0x808080, LinearSRGBColorSpace );
console.log( color.r ); // → 0.5

Changes

• Add SRGBColorSpace and LinearSRGBColorSpace enums
• Add colorSpace argument to THREE.Color getters/setters
  • Hexadecimal and CSS input are considered sRGB by default
  • All other inputs are considered Linear-sRGB by default
  • sRGB input will be converted to Linear-sRGB, as required by the renderer
• Add ColorManagement API. Only its .legacyMode flag is intended as a public API at this time. Other methods or properties may be added to support wide-gamut workflows in the future.

Effects

• Frameworks like A-Frame and R3F already enable color management as a layer on top of three.js, treating user-provided colors as sRGB by default. These changes allow vanilla three.js users to provide colors in the same way.
• Many examples already use renderer.outputEncoding=sRGB, and so hard-coded colors (like background and ground colors) are implicitly Linear-sRGB... These will need to be updated (or their appearance will change) with color management enabled.
• Fog and background color are excluded from output encoding. As a result, the two lines below result in different colors, and that doesn't seem ideal:
  • basicMaterial.color.setHex( ..., sRGBColorSpace )
  • background.setHex( ..., sRGBColorSpace )

Next steps

An upcoming PR will add documentation explaining what is meant by "working color space", and how that relates to choices of input and output color spaces. That documentation is meant to be a significantly expanded version of the content at https://www.donmccurdy.com/2020/06/17/color-management-in-threejs/.

/cc @gkjohnson @WestLangley

[donmccurdy]

Adding a few additional notes in comments here... I'll go into more detail in an upcoming documentation PR.

[Mugen87]

Would you please explain why the Color class needs a different approach for handling color spaces than Texture?

Textures allows to define the color space per texture. Why can't we do this for Color? TBH, I still like the Color.encoding approach^^.

The PR looks great, I'm just asking for a better understanding.

[gkjohnson]

Really glad to see how this has come along! A few thoughts other thoughts:

1. In docs it should be very explicitly noted that the Color.r/g/b members are not in a reliable color space. Not only because of this legacy mode support but in the future if other wider gamut color spaces are supported then it will depend on the current working color space.

2. Should functions like Color.convertLinearToSRGB log a deprecation warning when the legacy color path is enabled since they're inconsistent with the new color storage model?

3. I assume we'll only be changing WebGLRenderer.outputEncoding to sRGBEncoding by default once this legacy mode is disabled by default?

Overall I'm happy with what this has evolved in to. The API surface is consistent (aside from the Color.r/g/b fields) which was my original concern when improving the color spaces for the project.

[donmccurdy]

Would you please explain why the Color class needs a different approach for handling color spaces than Texture? Textures allows to define the color space per texture. Why can't we do this for Color? TBH, I still like the Color.encoding approach^^.

Let me come back to this question once I've opened the documentation PR, I think my answer will not be very clear without that additional context.

[WestLangley]

I assume we'll only be changing WebGLRenderer.outputEncoding to sRGBEncoding by default

@gkjohnson I would rename the property:

renderer.outputEncoding -> renderer.outputColorSpace

[donmccurdy]

Update —

• Color Management: Documentation #23430

@Mugen87 to your question earlier, I believe that almost anything a user will 'do' with a Color instance should done in the working color space (Linear-sRGB, as defined in link above), and that's not the color space users will find in a color picker or their CSS stylesheets, which are usually sRGB.

If we define color.encoding or color.colorSpace and most colors get marked sRGB — since that is what users initialize them with! — there's very little you can safely do with the RGB components of that Color instance. Instead I think it's best to specify the color space of the input when constructing a color, and immediately convert to Linear-sRGB at that point.

[donmccurdy]

As there are a few steps here, my suggested grouping of PRs would be:

r138

• Color: Add 'colorSpace' argument for getters/setters #23392
• Color Management: Documentation #23430
• color management disabled by default, begin opt-in testing

r139

• rename texture.encoding → texture.colorSpace
• rename renderer.outputEncoding → renderer.outputColorSpace
• change default renderer output to sRGB
• color management enabled by default

Wouldn't need to be r138 and r139 specifically of course, but basically I think these changes should be spread over at least two releases, with the breaking changes grouped together.

[WestLangley]

The Color class is NOT associated with a color space.

As I have said, the API to follow is:

color.convertSRGBToLinearSRGB();

color.copySRGBToLinearSRGB( source_color );

You need a source color space and a destination color space.

There are not that many variations, so hardwiring a separate method for each pair seems OK to me.

//

Alternatively, you can provide:

color.convert( fromColorSpace, toColorSpace );
color.copy( source_color, fromColorSpace, toColorSpace );

[donmccurdy]

The Color class is NOT associated with a color space.

This feels like a decision we need to make, not a foregone conclusion — I believe there are advantages to associating the RGB components of the Color class with the working color space (currently Linear-sRGB), as described above. Could you help me understand what advantages you see in the API you are suggesting, or the disadvantages in this PR? Am I correct in understanding that you'd like to have these convert/copy methods and not have these parameters on setters/getters?

If we only offer methods that convert a color from A to B, it seems like we are still expecting all Color instances to be provided on materials in Linear-sRGB, but we are making it harder for users to do that — because each needs to be explicitly converted from sRGB. I would like this syntax to 'just work', without needing to be told the hex color is sRGB:

const material = new MeshStandardMaterial( { color: 0x502020 } );

[WestLangley]

The Color class is NOT associated with a color space.

This feels like a decision we need to make, not a foregone conclusion

The statement is true. But, yes, the implication of my statement is it should stay that way. Just my humble opinion. :-)

Am I correct in understanding that you'd like to have these convert/copy methods and not have these parameters on setters/getters?

Yes.

I would like this syntax to 'just work', without needing to be told the hex color is sRGB:

IMO, your approach results in too much magic behind the scenes. That is just my view. I know your concept is proposed with good-intention.

//

Good news, thanks to recent work by @gkjohnson:

1. Loaders now work correctly and convert colors to scene-referred linear. Users just need to set the proper output color space for rendering. That is a matter of copying the three.js pattern.

2. If a user is using a color picker, then the user needs to know something about color space conversion. Again, it is a matter of copying the three.js patterns in the examples -- or adding appropriate examples if they do not exist.

[gkjohnson]

I don't feel that enforcing color storage in a common color space is all that magic. If we're going to hope to ensure that a color is in our working color space by the type they get to a shader then we need to have some reliable way of knowing what space a color instance is in.

Another option for enabling that is to store colorSpace on the Color instance itself but that requires converting the color to a common color space whenever an operation is done (lerp, add, etc) and converting before uploading to the shader. But that feels unnecessarily complicated.

[donmccurdy]

Another option for enabling that is to store colorSpace on the Color instance itself ... But that feels unnecessarily complicated.

Yeah, unfortunately I think this way would get too complicated.

IMO, your approach results in too much magic behind the scenes ...

I don't feel that enforcing color storage in a common color space is all that magic ...

There'd be no magic in saying, "all three.js methods and properties using Color instances assume the color's RGB components are Linear-sRGB". It's not totally true yet — see scene.fog and scene.background — but mostly true and we could fix the rest if we want to.

The harder question is whether using sRGB for getters/setters (hex, CSS, HSL) and Linear-sRGB for floating-point RGB components is too much magic. Personally I think it's the right choice — A-Frame and R3F read/write sRGB hex colors, and internally convert to Linear-sRGB for THREE.Color instances. As far as I can tell Filament makes the color space a required argument when setting a color on a material. Users seem to manage well enough; it helps that it's hard to find a hex color that isn't already sRGB.

But either way, it's a big change — see implications under Working with THREE.Color Instances in #23430. I'm hoping more people will weigh in on that part of the API.

[WestLangley]

There'd be no magic in saying, "all three.js methods and properties using Color instances assume the color's RGB components are Linear-sRGB".

There will be confusion, IMO.

Vertex colors in glTF are linear multipliers, while the base color in glTF is assumed to be encoded with the sRGB transfer function.

You are trying to enforce a particular color space with THREE.Color. THREE.Color is not associated with a color space.

[donmccurdy]

Vertex colors in glTF are linear multipliers...

The glTF specification's phrasing of "linear multipliers" is something I'd prefer to leave out of three.js. It's clearer to say that both three.js vertex colors, material.color, light.color, etc. are expected to be given in Linear-sRGB space. The same is true of almost any role colors may fill in three.js. If we want to say that a loose/unused THREE.Color does not have a color space that's fine, but a Color used by a material or a light must be Linear-sRGB.

If a user is using a color picker, then the user needs to know something about color space conversion...

The current state of three.js is that every time the user provides a color from virtually any other software, they must immediately do an explicit sRGB → Linear-sRGB conversion. The same is true for our own examples using dat.gui color pickers. It's good for users to know something about color space conversion, but requiring boilerplate conversion on every color input is not a good status quo, and not (IMO) standard practice in a graphics library. The objective of this PR is to ensure that a correct color workflow (with sRGB → Linear-sRGB conversion) is also the easiest color workflow. If this requires adding a stated assumption that certain THREE.Color instances contain RGB components in Linear-sRGB, I think we should do that.

[WestLangley]

@donmccurdy OK. I do not want to be a blocker. Your approach deserves a try. If others are happy with it, that is great. :-)

[donmccurdy]

@WestLangley thank you for being open on this. I don't want to force this through if it's divisive. Perhaps let's leave the PR up for a while and see if we can gather more feedback.

The <color> datatype in CSS is another interesting reference, as they're slowly adding color spaces beyond sRGB.

Pulling in a few people from #11337 — @bhouston and @zomboh do either of you have preferences here, or examples from experience in other tools?

[bhouston]

This makes sense and solves an issue I ran into with Three.js back in 2016: #8827

I think that in the future, we will likely start to support more output encodings rather than just sRGB, specifically HDR outputs, but that requires browser support first, which is coming: https://twitter.com/BenHouston3D/status/1481979159077339143

To speed up the transition to ColorManagement = true, we should likely have a warning for when it is off. I think that we should aim to actually just remove the idea of ColorManagement.enabled in the future because supporting multiple paths is annoying and this one is clearly right. If people want the old behaviour just use the ColorSpace specificers when setting/getting Color values.

[bhouston]

// current defaults
THREE.ColorManagement.enabled = false;
renderer.outputEncoding = LinearEncoding;

// future defaults
THREE.ColorManagement.enabled = true;
renderer.outputEncoding = sRGBEncoding;

The above confuses me. I am so confused that renderer.outputEncoding is linear by default. I thought it was sRGB for the longest time? outputEncoding linear is simply wrong and will lead to screwed up color output. I guess over here at @Threekit, we would just always set outputEncoding = sRGB and forgot that Three.js was using the wrong default?

[bhouston]

Ah, I understand the mistake. "Encodings" and "ColorSpaces" are being mixed together.

Originally we thought of these options:

• ColorSpace (Linear, sRGB, AdobeRGB, Gamma20, Gamma22)
• Encoding (None, RGBE, RGBM, RGBD, LogLUV, CMYK) parameter.

Per: #11076 (comment)

But I think that the above dichotomy isn't completely accurate. RGBE, RGBM, LogLUV are the same linear HDR colorspace but represented in 4 bytes (which some clipping - maybe that makes them different colorspaces?.) Where as CMYK is an LDR color space without Alpha.

I think we need better clarify on ColorSpace and Encoding usage.

[donmccurdy]

There are fewer encodings left now (see #23392 (comment)), so I agree with @WestLangley's previous suggestion that sRGBEncoding should be removed and replaced with *ColorSpace alternatives.

outputEncoding linear is simply wrong and will lead to screwed up color output...

I think we've all agreed for a while that the default should be changed to sRGB – but I'm at least partly responsible for why that hasn't happened yet, as I've been pushing to wait and change the output defaults and the input defaults (for hex and CSS colors) at the same time: #22275 (comment).

[DennisSmolek]

but that requires browser support first, which is coming: https://twitter.com/BenHouston3D/status/1481979159077339143

From what I can understand it's already here to some extent: https://developer.chrome.com/blog/new-in-chrome-94/#canvas-colorspace
The canvas can now be set to at least P3 and I've read it said it can be set to Rec. 2020 too and only depends on if the display/browser/os can support it.
This Webkit page is pretty great too and has detectors that work in real time. My iMac supports P3 but as I drag to other monitors it adjust and a warning appears.

[DennisSmolek]

After being sparked by Don's tween earlier I've (yet again) spent far too long going down the color rabbit hole..

In the UI/Design side we've had color pains in the web resulting in some amazing efforts to not only educate. but also provide solutions.

Even with all that I still struggle to really get what's going on or know what the "right" action is to take.. Converting in/blending/how it'll react with lights etc. On top of all the other things a person faces when trying something new.

So in regards to if you should convert with setHex() etc:

I appreciate that Three is adding more documentation to help (me) and others to get what's happening, but I also think it a GREAT idea to just take care of some of it by default.

I think it smart to lower the barrier to entry where you can, but not obscure or hide what's going on, rather I'd suggest offering the Progressive Enhancement of diving into the nitty gritty of it all when the user is ready/curious/problem solving.

And for those who've already got the experience & skillset they can use the overrides to dial their workflow to their EXACTING specifications..

[donmccurdy]

From what I can understand [wide gamut canvas is] already here to some extent...

Unfortunately limited to Canvas 2D contexts – no support for WebGL yet. But eventually it will happen, in WebGPU if not in WebGL.

I appreciate that Three is adding more documentation to help (me) and others to get what's happening, but I also think it a GREAT idea to just take care of some of it by default. ... I think it smart to lower the barrier to entry where you can, but not obscure or hide what's going on, rather I'd suggest offering the Progressive Enhancement ...

Thanks @DennisSmolek! Improving defaults (for both correctness and ease of use) is a goal of this PR, along with the related changes to renderer.outputEncoding. The aim of progressive enhancement is a helpful one, e.g. any options related to wide-gamut rendering should be fully "opt-in". But, similar to Blender's requirement of marking color and non-color textures, I think certain concepts can't fully be hidden here.

[donmccurdy]

NOTE: I've just rewritten the first few paragraphs of this PR's description, to reflect the feedback and changes to the PR. Specifically, calling out the fact that only hexadecimal and CSS input are considered sRGB and converted.

---

@Mugen87 after some more discussion and mulling over, I think I'd like to delay the question of adding color.colorSpace until some of the other changes of #23614 are in place. I'm not sure about parts of it yet. This PR will function the same way with or without that change.

---

In my opinion, this PR brings THREE.Color getter/setter APIs to where they ought to be in a linear workflow, while accommodating conventions our users expect from experience with CSS and Blender. Certain naming choices (e.g. NonColorData and ColorManagement.legacyMode) could perhaps use a bit more fine-tuning, but functionally I believe this is the right solution. @mrdoob — this should be ready for your review and feedback. Thanks!

[mrdoob]

Does @WestLangley approve? 😅

[gkjohnson]

Regarding adding Color.colorSpace I think it's worth noting that I don't think it simplifies any other part of the API and just adds overhead and complexity imo.

• A global color space must still be set for shaders and color operations but now colors must be converted to and from the working color space when performing math operations and updating shader uniforms.
• The constructor, setter, and getter APIs must still all have an argument to specify a target color space to retrieve and set color data in.

I guess I'm just not sure I see the benefit.

Another thing that I don't think we've really thought about, though I suppose it's longer term, is how vertex colors are handled. It's easy when we say Linear sRGB is the working color space and they're stored as such but what happens when P3 or ACES are the working color space? If vertex colors are going to be interpolated correctly in a shader or with animations then they must have either a color space associated with them or be implicitly converted to the correct color space. I think a solution for that can wait but I thought I'd mention it in the interest of being thorough.

edit: I see now we're waiting on Color.colorSpace field

[donmccurdy]

I don't think [color.colorSpace] simplifies any other part of the API...

A few quick thoughts. If we added such a property, its default should certainly be the working color space, Linear-sRGB (not sRGB). Automatic conversions should be generally avoided — if an operation is valid on the given color space it should run, otherwise the operation should probably warn or fail — to avoid a whole mess of complexity. For instances managed by three.js (like material.color) it would not be valid to change the color space to anything other than the working color space as used in the shader.

I do think use good cases exist — e.g. working with LightProbes or LUTs that don't need to be constrained to the sRGB gamut, perhaps? But I don't feel sure about those use cases yet, and I don't think easier Linear-sRGB ↔️ sRGB conversion is the right application for it.

Another thing that I don't think we've really thought about, though I suppose it's longer term, is how vertex colors are handled...

I think it's fair to say that any wide-gamut working color space will require considerable care; nothing in any 3D model format will work 'out of the box'. 😢 But ultimately we'd expect vertex colors to end up in the working color space, and it's straightforward to pre-process glTF files for that purpose.

[mrdoob]

Thanks!

[gkjohnson]

A few quick thoughts. If we added such a property, its default should certainly be the working color space, Linear-sRGB (not sRGB). Automatic conversions should be generally avoided — if an operation is valid on the given color space it should run, otherwise the operation should probably warn or fail — to avoid a whole mess of complexity.

I guess to that end I'm not seeing what the real benefit is to the flag. It seems to make things more confusing. I just imagine users wondering why they can't interpolate between two colors. And if two colors must be in the same color space to be operated on together then we're not adhering very strictly to our concept of a "working color space" across three.js, then.

I think it's fair to say that any wide-gamut working color space will require considerable care; nothing in any 3D model format will work 'out of the box'. 😢 But ultimately we'd expect vertex colors to end up in the working color space, and it's straightforward to pre-process glTF files for that purpose.

I want to believe there's a better solution here but I acknowledge that it's a future problem. Something like a ColorBufferAttribute could be added that implicitly converts to the working color space, for example, or a color space could be included alongside the BufferAttribute similar to textures so vertex colors can be converted in shaders since byte arrays are not well suited to linear color data.

[gkjohnson]

Perhaps a line item should be added to #23614 to think about handling of Color BufferAttributes?

[donmccurdy]

I would say uint8 is not ideal for Linear-sRGB vertex color data, but uint16 and float32 are fine. glTF is (as far as I know?) the only format we have that supports anything more compact than float32, and it specifies Linear-sRGB vertex colors. So I feel like sRGB vertex colors in uint8 buffers don't seem to justify a new BufferAttribute API and shader permutations here. To the extend that we can avoid conversions and store data in working color space, that is a very valuable thing. Perhaps a new issue if there's more to consider on that side? I'm not ready to think about vertex colors in P3 or ACEScg quite yet I'm afraid. 😓

[gkjohnson]

I would say uint8 is not ideal for Linear-sRGB vertex color data, but uint16 and float32 are fine. glTF is (as far as I know?) the only format we have that supports anything more compact than float32, and it specifies Linear-sRGB vertex colors.

I agree -- this isn't just about file formats, though. It's about enabling users to consistently use, operate on, and render color in three.js with or without existing model formats.

To the extend that we can avoid conversions and store data in working color space, that is a very valuable thing. Perhaps a new issue if there's more to consider on that side? I'm not ready to think about vertex colors in P3 or ACEScg quite yet I'm afraid. 😓

I just suggest adding it to #23614 in order to keep everything consolidated understanding that it's not an immediate priority. Perhaps as the last line item or as a (2.0+) category bullet.

[donmccurdy]

Sounds good! We'll have to explore more, but added this under 2.x in #23614 for now.