Material: Added onBeforeCompile()

[mrdoob]

Over the years, a common feature request has been to be able to modify the built-in materials. Today I realised that it could be implemented in a similar way to Object3D's onBeforeRender().

e55898c

WebGLPrograms adds onBeforeCompile.toString() to the program hash so this shouldn't affect other built-in materials used in the scene.

Here's an example of the feature in action:

http://rawgit.com/mrdoob/three.js/dev/examples/webgl_materials_modified.html

material.onBeforeCompile = function ( shader ) {

	// console.log( shader )

	shader.uniforms.time = { value: 0 };

	shader.vertexShader = 'uniform float time;\n' + shader.vertexShader;
	shader.vertexShader = shader.vertexShader.replace(
		'#include <begin_vertex>',
		'vec3 transformed = vec3( position.x + sin( time + position.y ) / 2.0, position.y, position.z );'
	);

	materialShader = shader;

};

Not only we can mess with the shader code, but we can add custom uniforms.

if ( materialShader ) {

	materialShader.uniforms.time.value = performance.now() / 1000;

}

Is it too hacky?

/cc @WestLangley @bhouston @tschw

[unconed]

It does seem a bit hacky to require custom string manipulation when WebGLProgram.js already does a lot of processing. There is a related use case, which is to add new #includes rather than overriding existing ones. You can do this today by modifying THREE.ShaderLib at runtime, but it feels equally dirty.

Perhaps a nicer solution for both is to allow you to provide a custom ShaderLib for a material, that would override/augment the built-in chunks without manual string manipulation and without permanently clobbering them. That doesn't preclude an onBeforeCompile hook for other uses, but it seems more in line with what your example is trying to do.

That said, shader composition based on raw includes is always going to be very fragile from one version to the next. If you don't mind including all the skeleton code (like meshphong_vert.glsl), you can already extend the built-in materials with something like this:

export class MyPhongMaterial extends ShaderMaterial {
  constructor({ color, radius, map, normalMap, emissiveMap }) {
    super();
    this.vertexShader = "...."
    this.fragmentShader = "....";
    this.uniforms = UniformsUtils.clone(ShaderLib.phong.uniforms);

    this.isMeshPhongMaterial = true;
    this.lights = true;
   
    this.uniforms.time = { value: 1 };
    //...
  }
}

In all cases, you have to rely on the internal organization of built-in shaders not to change too much from one release to the next. Each include is already a function in disguise, with an ill-specified signature. I suspect it is going to be much cleaner and maintainable on both sides to provide function-level rather than include-level overrides.

[pailhead]

Today I thought..

Nice! I thought so too on february 10th when i made this pull request:

#10791

It seems the only difference is that you're doing it through a callback? I tapped into the unused argument in the WebGLRenderer.

I like using this for example to get threejs to work with normal maps, but I've linked some issues that seem like they would be trivial to solve with this.

You can look at my branch and take this for a spin or check out this super simple example (just tests a custom deformation with lighting/shadows etc, and adds a few uniforms).

http://dusanbosnjak.com/test/webGL/three-material-shader-override/webgl_materials_shader_override.html

Perhaps a nicer solution for both is to allow you to provide a custom ShaderLib for a material, that would override/augment the built-in chunks without manual string manipulation and without permanently clobbering them.

Exactly sums up what happens in: #10791 I wish i could write better descriptions :)

---

I must admit that i don't know wth is going on here. I think i know someone who gave up contributing to three, didn't understand it then, but now i find it kinda frustrating.

Frustrating mostly because i'm having a deja vu when i read the first paragraph:

...similar way to Object3D's onBeforeRender().

This exact same thing happened with the onBeforeRender() #9738, when it took a year of convincing to get the feature in.

Jeez, twice, with the same dev? I'm obviously doing something wrong here, but what? The first time around was the build files being checked in, i didn't know what to do, and it just got forgotten. But this time around i was on top of this pull request, there's a conflict right now but it's easily solved, were no conflicts for months.

Don't get me wrong, i don't think it's vanity in the works here, i think i just want to run ideas by people and get feedback. @unconed it's obvious that you're familiar with the problem and you've probably tried different things. What may have caused an user like you to miss the other PR?

I felt pretty good about the per material shaderlib, but i did find one function in the renderer that looked/compared entire shaders as strings to figure out caching, wasn't feeling all that great about that. Wish there was some feedback given...

Was the example flawed? The head may make it much more obvious what is going on than my abstract spike ball, but the whole example works with shadows which covers more ground. It's heavy for some reason, but i think it's because sin/cos on many verts.

[pailhead]

I guess if you're starting over... Even though scene kit is absolutely horrid, this api is really nice:

https://developer.apple.com/documentation/scenekit/scnshadable

Albeit horribly documented, i can't find the part that's of more interest, but basically they have abstracted hooks at many different stages of the shader.

[mrdoob]

Earlier this year i made a pull request that seems to do something similar if not the exact same thing:

#10791

It seems the only difference is that you're doing it through a callback? I tapped into the unused argument in the WebGLRenderer.

Sorry for not being able to take care of that PR yet @pailhead. I guess the main advantage of my approach is that it only required 3 new lines.

Having said that, I'll now spend some time studying yours and @unconed suggestions.

[pailhead]

This one definitely feels more elegant, only three lines. I was following a different pattern in the other one. Was going to say that it may be a bit more verbose, but not so sure. I guess the only advantage of the other one is that it was good to go a few months ago :)

[mrdoob]

Jeez, twice, with the same dev? I'm obviously doing something wrong here, but what?

Sorry about that. The only thing I can think of is that probably I got overwhelmed. Maybe a long discussion or a complicated PR that would consume too much of my energies, so I decide to leave it for later and move to simpler PRs.

It's not only you @pailhead. @bhouston has had a lot of PRs like this, even @WestLangley and @Mugen87.

Again, is not that I don't like the PR, is that the PR requires some attention I can't offer at the time. A good example is the Instancing PR. I managed to find time to read through it, did my own experimentation and suggested simplifications. Hopefully I'll be able to revisit it soon.

It's difficult to manage all this and give every PR the attention they deserve.

Was the example flawed? The head may make it much more obvious what is going on than my abstract spike ball, but the whole example works with shadows which covers more ground. It's heavy for some reason, but i think it's because sin/cos on many verts.

Now you mention it... Yeah, runs at 10fps on a new MacBook Pro when zooming in 😮

[pailhead]

I think it's the shadows and the rand, sin cos and stuff, but yeah it looks like it takes too much of a hit.

[pailhead]

Either way, i'm surprised you pulled this off in three lines, i was focused too much on how material params get turned into uniforms and followed that pattern. The difference is that I provide an alternative ShaderLib like dictionary, so #include <> brings in different code. You remove the include before this happens, and replace it with glsl. I think if you return some kind of a wrapper around the shader, perhaps this syntax could be cleaner (just provide the chunk name + glsl, instead of replace(). But it may end up looking much more like the other one.

It would be really nice to have this, i haven't worked with that many 3d engines, but unity and scene kit seem to have something like this.

[mrdoob]

@unconed

I guess one benefit from the onBeforeCompile() is that the material properties remain unchanged. So it feels more like extending the built-in materials.

export class MyMeshPhongMaterial extends MeshPhongMaterial {
  constructor( parameters ) {
    super( parameters );
    this.onBeforeCompile = function ( shader ) {
      shader.vertexShader = shader.vertexShader.replace(
        '#include <begin_vertex>',
        'vec3 transformed = vec3( position.x + sin( position.y ) / 2.0, position.y, position.z );'
      );
    };
  }
}

var material = new MyMeshPhongMaterial();
material.color.setRGB( 1, 0, 0 ); // this still works

Messing with ShaderLib is tricky indeed. Adding forever-unchanged hooks should be doable though:

	#include <begin_vertex>
	% vertex %
	#include <morphtarget_vertex>
	#include <skinning_vertex>
	% transformed_vertex %
	#include <project_vertex>
	% projected_vertex %

The replace code will become this:

this.onBeforeCompile = function ( shader ) {
  shader.vertexShader = shader.vertexShader.replace(
    '% vertex %',
    'transformed.x += sin( position.y ) / 2.0;'
  );
);

We'll then remove any hook that hasn't been used before compiling, of course.

[pailhead]

here's what it looks like compared to the per instance ShaderChunks

//given some material
var material = new THREE.MeshNormalMaterial();

//and some shader snippet
var myShader = [
	'float theta = sin( time + position.y ) / 2.0;',
	'float c = cos( theta );',
	'float s = sin( theta );',
	'mat3 m = mat3( c, 0, s, 0, 1, 0, -s, 0, c );',
	'vec3 transformed = vec3( position ) * m;', //and making assumptions about THREE's shader framework
	'vNormal = vNormal * m;'
].join( '\n' );

#10791 same as Material.defines approach:

material.shaderIncludes = {
	
        begin_vertex: myShader,

	//uv_pars_vertex: [
	//	THREE.ShaderChunk['uv_pars_vertex'], //this doesn't have to be
	//	"uniform float time;",
	//].join('\n')
};

material.shaderUniforms = { time: { value: 0, type: 'f' || 'float' } }; //because this could just inject it in the right place (but needs type)

It's only a dictionary of chunk names, and the uniforms object. The string manipulation here is more along the lines of "i want to reuse this chunk, and do something on top of it"

this PR:

material.onBeforeCompile = function ( shader ) {

	shader.uniforms.time = { value: 0 };

	shader.vertexShader = 'uniform float time;\n' + shader.vertexShader; //this feels hacky
	
	shader.vertexShader = shader.vertexShader.replace( //this is more verbose
		'#include <begin_vertex>',
		myShader
	);

};

I guess one benefit from the onBeforeCompile() is that the material properties remain unchanged.

It works the same in the other PR, so I think it doesn't matter how it's done.

---

One probably irrelevant benefit that i can think of is that you can keep the logic for manipulating the material in the same block and on the same level. Even if the code doesn't look much different, conceptually, you're manipulating .color on Material while manipulating a .uniforms object on a "shader". If you had a piece of code somewhere changing color, and you later add custom GLSL to that material, you can add the code to change the uniform with the color. (skip wall of text)

---

If the uv_pars_vertex is confusing, i think a good question to ask is:

"where do I inject my custom GLSL function, like float rand( vec2)?"

Then this one might make sense #11050

---

Another argument could be - assuming you know GLSL, assuming you know THREE's shader framework, you still have to be creative and think about where you inject what. I had to think a bit and look through most of the shaders to figure out that uv_pars_vertex is a convenient place to do my material extension.

It would be better to go from something like this towards an abstraction, have a lightingPhase, objectTransformation , perspectiveTransformation etc hooks. Or at least a guaranteedToBeAboveMainButBelowCommonsAndExtensionCalls dummy chunk that's included in every shader program. It looks like you're going for that with % vertex %? But i'm unfamiliar with that syntax.

This PR, as is, seems like it's going in the opposite direction. On top of knowing where you need to tap in, you also need to be explicit and manipulate strings, repeating some of the work that the renderer already does for you. Also, if you ask another question in addition to the first one

How do i use a function, declared in the commons chunk, inside the function that i'm injecting?

you might find yourself doing more string manipulation than just prepending the uniforms to the entire shader.

[mrdoob]

I had to think a bit and look through most of the shaders to figure out that uv_pars_vertex is a convenient place to do my material extension.

I don't think there is a way around that unless we over-engineer. The idea with % vertex % is to try helping with that a bit by naming the hooks and more or less tell you at what state you're injecting code, but it would be hard to document what variables are available at what point.

We're opening a door for allowing built-in materials hacking, but I don't think we can provide proper "support". The user should be aware that chances are things may break.

[pailhead]

I tried to tackle it in the other PR. I added a dummy hook at least for the functions, varyings, attributes and uniforms. Lots of the GLSL logic already happens on structs, scenekit documented each variable (cant find the same documentation anymore though).

It would probably need to be refactored as we go along but something along these lines:

#ifdef PHASE_FOO

#include <shader_foo> 
//someGlobalStruct.mvPosition = modelViewMatrix * myLogic( transformed );
//if there is glsl provided for phase "foo" document that it should operate on "transformed" 
//and that it should return "mvPosition" 
#end including <shader_foo>

#else 

  //default
  #ifdef USE_SKINNING

	vec4 mvPosition = modelViewMatrix * skinned;

  #else

	vec4 mvPosition = modelViewMatrix * vec4( transformed, 1.0 );

  #endif

#ifdef PHASE_BAR

#include <something_like_this>
//mvPosition = myPostDefaultTransformationLogic( mvPosition );

#endif

gl_Position = projectionMatrix * mvPosition;

#endif

Of course ifdefs and includes would proably not work like this. But i can see it being relatively straight forward with the other approach. We could add #include <some_phase> always, with empty strings. Something could help manage what gets triggered by using #ifdefs. So for example, if you provide logic for "vertex_transformation" phase, the shader will define whatever you provide. Documenting what each chunk does with GLSL would be overall helpful. Without going further into abstraction, i think it would be great to have this map to how the chunks are structured now.

[pailhead]

We're opening a door for allowing built-in materials hacking, but I don't think we can provide proper "support". The user should be aware that chances are things may break.

We already have the defines stuff available on every material. #10764 suggested to modify THREE.ShaderLib and then define a .defines property on an instance of a Material. Granted, the property was undocumented until i wrote the entry (but you approved it :)), and is still undefined.

But the way this currently works, and is inherited by every material, it is supported - if you put stuff in it, it will affect the shader for that material. Furthermore, if you happen to define something that is already part of the shader library, it may break things.

Not trying to be a smart-ass but trying to come up with an example. I mostly used defines with ShaderMaterial but it does affect all materials, just because how WebGLRenderer works. There is no logic that says:

if you're a much smarter, surface like abstraction of a material like Phong then don't take defines into account. Because three is not designed to have it's shader chunks overriden, and the Phong template is the final authority on what GLSL needs to do, and serves to hide this from the use, it makes sense to not have GLSL defines interfere with this.

I'm not entirely sure if it can be broken though, i'd have to try.

With that being said, i'd like the option to break things, if it provides a lot of gain. With a pretty encapsulated and tiny GLSL snippet, i could easily change my version of three to render normal maps differently. Taking a normal, and transforming it to get another normal is super straight forward, unless we start doing computer graphics in a completely different way, i can't see this ever breaking :)

Instancing was another example:

oh i'd like to use ANGLE_INSTANCED_ARRAYS, three doesn't seem to support it with it's shadows or PBR materials, let me just add these two lines, and make it work for me"

[fernandojsg]

I'm happy to see that this features is finally coming to three.js :). Yes I also created a yet another material modifier PR (#7581) It was so old that the code is not relevant anymore, but is basically injecting hooks like @mrdoob proposes and then just replace them by your own code.
I like the idea of predefined hooks as it's easy to understand what you're modifying as I believe most of the people that want to use this feature want to modify "slightly" the default material instead of completely rewrite it.

I like the simplicity of this PR but if we're going for a more structured/clean way to do things I'd prefer to have already a dictionary of hooks to replace with your code and a simpler way to add uniforms or custom code than just replacing strings.

[pailhead]

@fernandojsg any chance you could check out #10791, give feedback, it seems super similar to what you did except i've put the additional hook just outside of main (something like your "pre_vertex") and there's slightly more management.

[mrdoob]

@fernandojsg I had forgotten about your PR 😚. I think your PR looks quite a lot of how I was starting to see this working. Subconscious?

[mrdoob]

Not trying to be a smart-ass but trying to come up with an example. I mostly used defines with ShaderMaterial but it does affect all materials, just because how WebGLRenderer works.

I'm glad the library is hackeable in that way, but we shouldn't use abuse features inernally for things that were not intended for. It's easy to end up with fragile code that way.

[mrdoob]

Reading #7581 again... On top of e55898c we can add the % HOOKS % and then create a class like this:

THREE.ExtendedMaterial = function ( material, hooks ) {

    material.onBeforeCompile = function ( shader ) {
        var vertexShader = shader.vertexShader;
        var fragmentShader = parameters.fragmentShader;
        for ( var name in hooks ) {
           vertexShader = vertexShader.replace( '%' + name + '%', hooks[ name ] );
           fragmentShader = fragmentShader.replace( '%' + name + '%', hooks[ name ] );
        }
        shader.vertexShader = vertexShader;
        shader.fragmentShader = fragmentShader;
    };

    return material;

};

Then we can do this:

var material = new THREE.ExtendedMaterial(
    new THREE.MeshBasicMaterial(),
    { vertex: 'transformed.x += sin( position.y ) / 2.0;' }
);

[mrdoob]

So the question is, what hooks should we have?

VERTEX_UNIFORMS
VERTEX
TRANSFORMED_VERTEX
PROJECTED_VERTEX

NORMAL
TRANSFORMED_NORMAL

FRAGMENT_UNIFORMS
INPUT_FRAGMENT
OUTPUT_FRAGMENT

[WestLangley]

@mrdoob As an example, you can see in this gist where code needs to be injected into the MeshBasicMaterial vertex shader to support instancing.

[pailhead]

And here for the rest of the materials :)

#10750

[pailhead]

VERTEX_UNIFORMS sounds too opinionated, where should you add a function, an attribute or a varying? PRE_VERTEX or something?

NORMAL_MAP is definitely needed

vert:

PRE_VERTEX
VERTEX
TRANSFORMED_VERTEX
PROJECTED_VERTEX

NORMAL
TRANSFORMED_NORMAL

UV

fragment:

PRE_FRAGMENT
INPUT_FRAGMENT
LIGHT
NORMAL

OUTPUT_FRAGMENT

[mrdoob]

VERTEX_UNIFORMS sounds too opinionated, where should you add a function, an attribute or a varying? PRE_VERTEX or something?

Hmm, opinionated? How so?

[pailhead]

attribute vec4 aMyAttribute; is not a uniform :)

float myRand( vec4 foo ) { /*logic*/ return vec4(bar,baz)} is not a uniform

varying vec3 vMyVarying; is also not a uniform

either not plural, or not uniforms at all

[mrdoob]

What's the use case for "injecting" attributes?
Shouldn't you be using geometry.addAttribute( 'aMyAttribute', ... ) instead?

[pailhead]

I didn't know you don't have to declare it. Still leaves varyings and functions?

[mrdoob]

Still leaves varyings and functions?

Good point.

[pailhead]

#13192

[cdhanna]

Has this been included in a recent release, or is it all still in development?

[mrdoob]

@cdhanna http://threejs.org/examples/#webgl_materials_modified

[takahirox]

@mrdoob I think it's difficult (or impossible) to serialize Materials hacked with .onBeforeCompile(). Do you think users should use ShaderMaterial if they want full functional(render, copy, clone, serialize, and so on) modified built-in materials?

[pailhead]

Please read the following in the nicest, constructive, non-confrontational way possible :)

@takahirox why does this keep getting claimed?

I think it's difficult (or impossible) to serialize Materials hacked

When you "hack" materials with onBeforeCompile you just add uniforms which are the same as with any other material. There is no difference. If you can serialize material.color why can't you serialize material.userData.myColor?

I'm not trying to start a fight, walls of text or anything. In the simplest shortest possible terms, can you please explain, or at least point me in the right direction as to why it's impossible or difficult? I am open to the possibility that i'm missing something obvious, but if i am i'd like to understand what it is :)

From a very small sample, generated by articles, this experiment it does not seem like people want to use the ShaderMaterial approach.

---

This just came to mind? Does there exist some kind of a test that proves that this is hard or impossible? Like:

runSerializationTest( someMaterialWithOnBeforeCompile ).expect( something )

[donmccurdy]

As onBeforeCompile method could contain very arbitrary code, it's certainly impossible to robustly serialize that callback. For example, the method could make a synchronous XMLHttpRequest and do something with the result. 😅

However... for the way it's used practically (to patch the shaders) I agree it's not impossible. But the easy APIs (e.g. fn.toString()) are hacky, and the robust APIs are more complex.

[pailhead]

I still don't understand the problem. How about this point of view:

Take rN that had MeshPhongMaterial and didn't have MeshStandardMaterial.

You can serialize MeshPhongMaterial ie. write a JSON like this:

{
  color: 'red',
  specular: 'very',
  glossy: 'not much'
}

Take rN+1 that has both materials:

You can still serialize both the same way:

//phong
{
  color: 'red',
  specular: 'very',
  glossy: 'not much'
}

//standard
{
  color:'red',
  shininess: 'shiny',
  metalness: 'not much'
}

Nowhere did you serialize the GLSL from MeshStandardMaterial.

The same way, serializing any extended material.

//extended PhongMaterial
{
   color: 'red',
   specular: 'very',
   glossy: 'not much',
   hamburger: 'medium rare'
}

Deserializing:

if ( data.userData.hamburger && HamburgerPhongMaterial )
   mat = new HamburgerPhongMaterial( data ) 
else{ 
   console.warn(`I'm terribly sorry, but you don't have the HamburgerPhongMaterial extension, using default fallback`)
   mat = new PhongMaterial( data ) 
}

What am i missing here?

---

As onBeforeCompile ... serialize that callback.

Why, why would you even consider it :) I think this is the crux of my confusion, why is any thought given to this hook, it's not part of the material definition, description, anything. It has something to do with the engine, plugins, apps etc, not the data.

[donmccurdy]

The example you gave above looks fine to me, it just requires that the user code deserializing the material knows about the existence of HamburgerPhongMaterial? I have no problem with that, it doesn't necessarily even require API changes if you take an approach similar to #14099 and override toJSON and fromJSON.

Maybe we've interpreted @takahirox's question differently here?

...it's difficult (or impossible) to serialize Materials hacked with .onBeforeCompile(). Do you think users should use ShaderMaterial if they want full functional(render, copy, clone, serialize, and so on) modified built-in materials?

I'll try to split that apart:

• if the user calls toJSON() on a plain MeshStandardMaterial, where the GLSL that has been modified by its onBeforeCompile, the changes to the GLSL will not automatically be serialized.
• if the user serializes a MeshStandardMaterial with some extra properties indicating a change to the GLSL, and loads it with custom code that knows what to do with those extra properties, then certainly that's fine.
• various other options (ShaderMaterial, NodeMaterial, glTF's KHR_techniques_webgl, ...) are available for serializing arbitrary custom materials.

If the case we're talking about isn't one of those, then I guess I'm misunderstanding here. If this is about #14099, I'm (still) in favor of merging that.

[pailhead]

Hmmm, i wasn't actually distinguishing between the first two. I'm trying to think of something that wouldn't require uniforms, it could be a gl_FragColor.xyz /= gl_FragColor.a; for example. But i can't imagine a scenario where you would want to serialize something like this (along with the material?). It seems like it would fall in the scope of some effect renderer or something, not material data.

materials = loadSerializedMaterials()

effect = new Effect()

effect.load( 'alphaDivide').then(()=>materials.forEach(mat.onBeforeCompile = effect.getOnBeforeCompile('alphaDivide')))

The second example is what i had in mind all along. Along with various inputs, serialize a hint as to what to do with them userData.extension === 'hamburger'.

[takahirox]

I think I wrote poorly. What I wanted to mention is

• Built-in Material hacked with .onBeforeCompile() isn't copied, cloned, serialized correctly in the same API as built-in materials'. It's handled as non-hacked material.
• If users want the correct result, they need some extra work in user side.
• If users want the hacked built-in material with Three.js core API without any user-side extra work, they should think of other options, like ShaderMaterial.

My question is just for the policy of .onBeforeCompile() rather than for the specific case. I wanted to clarify the limitation and add a note in the doc.

[pailhead]

If i forget about three.js and the whole rendering system, i think i see the point. As a generic object, if it has .clone() you want to be able to clone and have consistent expectations. The problem is both solutions (clone this not clone it) seem to have a valid argument. You don't clone listeners, but you also somewhat expect something that influences drawing to be consistent.

I still think that my proposal from above is a valid way to do this, as i've encountered the same problem in other areas (like sharing depth buffers between targets can also be solved with ownThing || outsideThing.

Instead of caching the function, which doesn't work, you would basically cache userData but a subset of it. Makes no sense to cache userData.hamburger: 'rare' but it does userData.effectOn: true". The ownInput | ownChunks | ownWhatever` would solve this problem.

It also solves the problem of:

As onBeforeCompile method could contain very arbitrary code, it's certainly impossible to robustly serialize that callback.

It has been a long time, people have used onBeforeCompile we should know by now what kind of arbitrary code can be found in there. I think it only operates on shader object that is passed in. You mutate that object, but only mutations that make sense with ShaderMaterial would have an effect anyway. Setting some gl stuff yourself would probably be overridden, and that's about the only thing that pops to my mind.

It's essentially a pre three parse step, where you get a chance to parse the shader yourself along with the mandatory uniforms:{} interface, since you don't have it available on Material (defines:{} on the other hand you do, and both are siblings in ShaderMaterial).

How you do it, i.e whatever the arbitrary code is, doesn't matter. It does not return anything, but it mutates shader:{} and renderer then uses it.

Approaches that i suggest:

1. Add ownThing to various classes. THREE.WebGLRenderTarget could have .ownStencilDepthBuffer. Here, as i've suggested above, it would be some some version of ownGLSL:{} ownInput | ownUniforms . You remove the need for arbitrary code to mutate the shader:{} object:

Given input like this that originates from inside the WebGLRenderer (super private):

const shader = {
  uniforms: buildFromMaterial(mat),
  vs: getVSTemplate(key),
  fs: getFSTemplate(key)
}

nuke this:

shader = onBeforeCompile( shader )

shader.vs //invalid GLSL
shader.uniforms //you got some extra ones that onBeforeCompile may have mutated, maybe removed some others

shader = ___parse(shader) //private step

shader.vs //valid glsl, (not a template)

compile(shader)

Use this:

function parseShader(shader){
   return {
     uniforms: shader.uniforms,
     vs: parseChunks(shader.vs)
     fs: parseChunks(shader.fs)
   }
}
//FIX FOR HASHING BUG
function parseChunk( material, chunkName ){
  return material.ownChunks[chunkName] || THREE.ShaderChunks[chunkName]
}

//do whatever you want with the templates, maybe remove an `#include <>`
shader = onBeforeParse(shader)

shader.vs //template || valid glsl

shader = parseShader(shader) //sample OWN || PRIVATE 

shader.vs //valid GLSL 

//if you want to apply a regex on the entire valid glsl, or compile node material or something else
shader = onBeforeCompile( shader )

compile(shader)

2. Make WebGLRenderer not know about anything other than ShaderMaterial.

Wherever you would have

const mat = new THREE.MeshBasicMaterial()
const sm = new THREE.ShaderMaterial()

Have

const mat = nodeMaterial.compile() //returns ShaderMaterial, with a friendly interface (same as StandardMaterial for example)

const mat = chunkMaterial.compile()

[pailhead]

The tl:dr of the stuff above is user does not care about the particular point in time when "compile" (parse) occurs. I think they care more about what happens, and what happens can be isolated to a handful if not a single use case in: shaderString, out: shaderString. The reason why it has to be done at that particular point in the pipeline i think is worth investigating and understanding. It seems to be getting in the way of things more than its helping.

[mrdoob]

@mrdoob I think it's difficult (or impossible) to serialize Materials hacked with .onBeforeCompile(). Do you think users should use ShaderMaterial if they want full functional(render, copy, clone, serialize, and so on) modified built-in materials?

I didn't expect (considered) the the use of onBeforeCompile() to be serialised...

[donaldr]

I just got bitten hard by using onBeforeCompile. I have a shader helper class that allows me to make modifications to built-in shaders in a defined way. I use the onBeforeCompile method to do this. Apparently, like stated in the first comment, WebGLProgram uses onBeforeCompile.toString() as a hash. But since my function is generic (it substitutes parts of the vertex and fragment shaders with variables) onBeforeCompile.toString() doesn't look different for different shaders. This meant that all my different shaders got cached as the same program. An eval call and a uuid and now my functions all look different. Took forever to figure this out.

[pailhead]

@donaldr See #13192. I think it would be good to document these limitations.

could you share how you're using the eval? I've been thinking about that but it just seemed too dirty. You could basically add const fdscxnmdrek435rkjl to the top of the body and then eval?

[donmccurdy]

This meant that all my different shaders got cached as the same program. An eval call and a uuid and now my functions all look different. Took forever to figure this out.

That sounds very frustrating, sorry. 😞 If the modifications you needed to make seem like they'd make sense in the library itself please feel welcome to open issues, too.

[pailhead]

please feel welcome to open issues, too.

Would you consider re-opening existing ones? #13192 seems to be the same but it was closed 😿 It would be really nice if a work around was posted in there if it was deemed a feature.

[pailhead]

Hmmm...

const _obc = shader => {...}
const obc = `
function (shader){
  const _${ hash() }
  ${ _obc.toString() }
  _obc(shader)
}
`
material.onBeforeCompile = eval(obc)

^Would this work? I never used eval.

[Mugen87]

The introduction of Material.customProgramCacheKey() via #17567 ensures that developers now have the possibility that shader programs are not shared for materials modified via onBeforeCompile() with conditional statements.

Please discuss other existing issues or enhancements in context of onBeforeCompile() in new threads (like #13446).