Automatically fade to Z far in environment depth fog #55388
Conversation
This is useful for open world games to smoothly fade out distant scenery in the fog. Exponential fog cannot do this on its own, so quadratic fog is combined with exponential fog using a `max()` function. The quadratic fog start distance is automatically set at 50% of the camera's Z far distance.
Calinou
force-pushed
the
environment-fog-add-z-far-fading
branch
from
e79b139
to
bd202f2
Compare
|
See also #54248 I don't think we should be adding hacks into the engine that are targeted for a specific hypothetical game's needs. If a particular game needs to use fog as a fade in mechanism, then they should override Fog as an LOD tool is, to my knowledge, not widely used by open world games anymore. Instead alpha or dithered fade in is used. Forcing every user who has fog to use it as a fade in technique seems like it will cause more issues than it will solve. |
|
is fog result radial, or just standard linear fade? |
In this case, we'll most likely need a better mechanism to globally extend/override StandardMaterial3D. This would likely remove the need for a lot of built-in features without introducing the usability issues around specifying custom ShaderMaterials manually everywhere.
It's radial, like exponential fog itself. |
|
Personally I can totally see this as being useful, to games large and small, to hide the far distance clipping, which the current Fog system simply couldn't do in 4.0 Here are some examples of what I mean: |
|
I have to agree with clayjohn that forcing this feature for everybody is a less than ideal solution. Imo it would be better if users were provided with the api and documentation they need to implement this (or another solution) themselves. Making the shader and post processing pipeline more modular and customizable (without having to recompile the entire engine) is probably the best solution. Though that would most likely require reduz' input. The last time a new post processing pipeline was proposed (the current viewport based solution is terrible ux) he raised some valid performance and paralellisation concerns and hinted that he would do it himself for 4.x. Is the depth buffer currently exposed to viewport shaders? Because if it is it should be possible to offer this effect as a viewport based addon instead. |
|
I disagree with @clayjohn @Ansraer since most mobile games I worked on (and seen) in the past several years uses quadratic fog as LOD fade, and is by far the most common usecase when using fog in games in general (and I've had an artist complain about the exponential fog in godot). If one has to go for performance reasons, I would vote for exponential fog to be removed instead, since it's more atmospheric and only used for a specific type of ambience/art style (horror/suspence), whereas quadratic fog is more general purpose and dozens of times more likely to be used. Who actually uses purely exponential fog without a distance clamp? I would like examples presented from published games. |
Here is why The Fog in this PR is a must - 10km view distance on a Massive Dummy Terrain It looks even worse when rotating the camera, with seriously multiple Square Kilometers of ground popping in and out of the view, something that wouldn't be a problem with this fog change, as Z-Far would be hidden, and as it is radial too, you will always see the same stuff from the same place, no matter your camera's rotation (if it is in your view ofc) |
|
@mrjustaguy have you tried using instance fade or something similar to fade out objects as they approach zfar? It was implemented in #54222 |
|
4 Problems with that (Material):
Distance fade on Mesh Instance with HLOD seems to result in Bool like behavior, for the entire mesh. so it's Visible or it is not. |
|
BaseMaterial3D distance fade isn't a good idea for fading terrain elements, as it requires making the materials transparent (or using dithering). It also requires you to apply this distance fade on every material in your scene, which can be tedious, especially when using imported 3D materials. Distance fade is good for fading smaller props at a distance, but I think we still need generalized open world fog fading. |
Fair enough, instance fading won't work for large terrains, its more suited for props. And the primary issue you are having is the terain hitting the far plane. Re-reading the full comment thread, it sounds like the use-case for this feature is purely about making terrain (or other very large objects) blend into the background as they approach the far plane. In that case, wouldn't it make more sense to just have a custom shader for your terrain that includes this? Terrain needs a custom shader anyway. For small props, they can fade out using the instance fade (which is way more efficient). I am very concerned about making every user pay for a feature that is only needed by some, also this feature drastically changes the appearance of fog for half of the view distance, so it may even create a visual problem for some games. |
The issue with a custom material shader is that you'd also need to use a custom shader for other large props (such as houses, large rock formations that can't be represented by a heightmap terrain, etc). Ensuring this is applied properly to every material where it makes sense can take a lot of time. My point is that fog should be the responsibility of the environment, not the material. (You shouldn't have to change your materials when the user changes their draw distance setting.)
The original plan was to make this an option (like choosing between curve-based fog and exponential fog), but reduz was against it when he reworked fog for 4.0. This is what other game engines generally provide. |
|
The use case for this is two fold, to Hide Z-Far, and to Make the view distance independent of camera angle. So far there have been a dozen people for this, and only a couple expressing any doubts regarding this, of which, none actually stated an actual problem it creates for them. I honestly doubt there are many people who would be against this that are actually using the current fog, as one of the most common use cases for fogs are to hide Z-far, and LOD, or a Subtle effect over long distances which while it is stronger with this, it's only really stronger at values close to Z-far, which you want to hide anyhow. Also, if someone doesn't like this solution, the current one could be made as a custom shader, like it was suggested for this, and if we have to pick this PR or stick with current fog, The current fog would be suited for a much, much more niche set of requirements and thus should be the one to require doing such a "workaround" if you desire it specifically over this PR. |
I have been expressing doubts because this will hurt performance of the core scene renderer. It adds extra calculations and (more importantly) extra VGPR usage to a core part of the scene renderer. This will decrease performance for every object in the scene regardless of how close they are to the far plane. That may be an acceptable trade off for your game. But I am unsure that it is a tradeoff that the engine should be making for users. That impacts every users that uses 3D. Its also not fair to expect people to to be aware of this PR. People who aren't having issues with the current implementation would have no reason to search for this PR and test it out.
This PR starts the gradient at Z-far * 0.5, so the effect can be visible starting from half the view distance. I wouldn't call that close to the Z-far.
Ideally we would have a way to inject custom code into all shaders so that we could allow people to (for example) write custom fog logic that would automatically apply to all of their shaders.
I'm not convinced that anything other than terrain needs this. A house that is 10km away will appear small enough that you can just use instance fade. This PR only benefits objects that span a large proportion of the view distance.
Object occlusion should not be the responsibility of fog or the environment. Users shouldn't have to pay for hacks that aren't useful in their game. I'd like to summarize the issue and my objects a little bit below. The Issue Instance fade is not an appropriate solution becuase it fades the whole object uniformly rather than fading on a pixel by pixel (or vertex-by-vertex) basis. Users desire a solution that doesn't have to be applied on a case-by-case basis, e.g. applied to some materials and not others. Possible solutions
Issues with 1:
Issues with 2
My thoughts I still remain unconvinced that having an object that spans all the way to the far plane is a common use-case (I'm happy to be proven wrong here). As far as I understand it, all modern terrain systems use either an HLOD-based chunking system, or at the very least a quad-tree approach to terrain. In either case, you never have one singular object that spans from the near plane to the far plane. |
|
In the meantime, it's possible to use an unshaded sphere with its cull mode set to Front and Proximity Fade enabled to fake fog. This sphere can then be parented to the camera. This approach gives you a lot of control (including using an arbitrary gradient texture to modulate fog colors), but it's also fairly expensive and may have sorting issues with other transparent materials: Testing project: test_fog_sphere.zip The fog sphere's size determines where the fog will start, while the proximity fade distance value determines the distance over which the fog will fade. The Camera's Far value should be set at least to the sum of those two values (and preferably higher for smoother fading, in my experience). Fog sphere disabled
Fog sphere enabled
This is a good approach if you want fog around the player with an unusual shape, such as a plane, box or cylinder. |
|
this thing's only downside is not really being able to easily keep the sky intact |
If you add a panorama sky texture to the SphereMesh's albedo, it will effectively display the sky instead of fog. This can't be done for procedural skies though (unless you render them to a texture somehow). Testing project: test_fog_sphere_1.zip Fog sphere disabled
Fog sphere enabled
|
|
Physical Sky, so yea, big issue with that workaround there for me.. |
|
I'm not sure that agumenting the far plane to have a better fog smoothing is a good idea in terms of perf, since more object will end up in the frustum. |
|
We've discussed this and decided to wait for more feedback regarding the use-cases. Can probably be adjusted in later versions, if there is a good use-case and demand. |
|
Use case: I use standard (non-volumetric) fog to hide scenery pop-in in an open world game (my draw distance is, erm, not too large for performance reasons). I got the trick off some gamedev blog I can't find now, but I have trouble adjusting the distances so that it works ;) |
|
Use case 2: always hiding everything close to being the distance of Camera Far which afaik is ALWAYS desired, though at large enough far distances the effect gets less and less useful as you're unlikely to actually see anything close to Camera Far getting cut anyway |
|
I am not familiar enough with the new renderer architecture or with lower level GPU performance specifics, but would introducing a branch here to optionally use quadratic fog in addition to exponential fog still be a problem for VGPR pressure? I'm wary on it being a default behavior, but introducing a branch here (or maybe a conditional compilation flag) and then reintroducing the Godot 3 Environment properties that let the user control the quadratic fog curve directly may accomplish godotengine/godot-proposals#4619 as well. That'd make using both fog modes an opt-in option rather than something done implicitly when using depth fog. There are aesthetic choices in games trying to mimic legacy fixed-function pipeline fog from the mid 90s and early 2000s that can't currently be expressed in the Godot 4 renderer due to the exponential fog, as I mentioned in that proposal. Godot's relative light-weightedness makes it a good candidate for games trying to mimic PS1 or N64 graphics today, so it seems a bit remiss to not have quadratic fog at all. |
VGPRs are allocated based on the worst case scenario. For a given pixel the worst case scenario would be that at least one pixel in a bunch goes down one branch and another pixel goes down the other branch. Adding a branch thus has the potential of increasing the VGPR usage at that point in the shader. Ultimately, whether that reduces occupancy is not certain because occupancy is based on your highest VGPR usage in the shader. To me what this comes down to is whether or not we want to implement, maintain, and support specific art styles in the core rendering code. Similarly, we have to ask if this is something worthwhile enough for all users to pay a (very small) performance cost so we can support this feature. That is a very hard question to answer and it is one we are bound to get wrong from time to time. My opinion right now is that this specific effect is not worth the performance cost. Especially now that shader includes have been merged into the engine, I don't see a good reason why users can't implement specific effects like this on their own. You can override the built in fog by writing to the |
Would this problem be alleviated if the engine could specialize the standard shader (or, any shader at all) based on environment features being used? |
However, extending or overriding the default material shader is still not possible, which is bad from an editor and asset pipeline usability perspective. Custom shaders don't have all the inspector hints that the default shaders have. Also, while this is now easier to achieve via custom shaders, my point is that this is a very common thing to do among people currently making games with Godot. I don't see the retro trend going anytime soon – if anything, many developers will move on to mimicking the 6th-generation console era, which still used distance fog liberally. I don't think we should make this large portion of Godot's 3D developer userbase jump through too many hoops. So far, I've seen a dozen posts or so about this subject among 4.0.alpha testers on various community platforms. I can only bet there's going to be significant outcry after 4.0's release if this is not addressed in a built-in way 🙁 In fact, if we have to choose, I'd argue that making non-volumetric fog support only distance fog would be more versatile for most users. If you need exponential non-volumetric fog, you could use a custom shader. Alternatively, you could use volumetric fog instead, as realistic-looking games can generally afford having higher performance requirements. |
It's not something I'd even thought about until someone asked me about it. I've been advocating for using the beta release as a starting point to start porting projects over, but it's a tough sell when even the forward mobile renderer doesn't have it built in yet. I do like the way exponential fog behaves, so I wouldn't want that to disappear in favor of an option that, while flexible, is noticeably harder to create realistic scenes with. Having both options in the Environment and being able to configure them independently would be the ideal, solving this MR's goals and answering the configurable quadratic fog question for everything else. There's still plenty of room in the specialization constant bitmap in both renderers to add a few more, if that would reduce the register occupancy pressure. |
|
Tossing my comment in here just to hopefully add proof to @Calinou's statement that many people are going to miss distance fog. Just ran into this problem in a project I've been toying around in using the 4.0 Betas. Please add distance fog back. This will also break one of my other pre-existing projects that I intend to port when 4.x is fully stable.
I have played around with the 4.0 non-volumetric fog system, and I genuinely cannot even imagine a single way to make use of it, it seems totally useless... And volumetric fog is extreme overkill for the level of graphics I usually work in, even if I could slightly replicate distance fog with it. Not saying it should be removed, just that it's not in any way a solution for me. Shaders are hard. I even still have a lot of trouble with them. I don't think "you could make a distance fog shader" is a valid response to this issue, especially since this is functionality that 3.x already had. |
|
This was brought to my attention tonight as an issue in 4.0, and as @Calinou mentioned it IS solvable with a volumetric fog shader, while also allowing sky penetration. I've written a really quick volumetric shader that does something similar to what Calinou mentioned, implementing a custom color, two cutoff scalar values between 0.0-1.0 (based on the size of the fog volume) with a minimum and maximum gradient distance from the fog center, which would be parented by the player object. shader_type fog;
uniform bool enable_y = true;
uniform vec3 fog_color : source_color;
uniform float fog_near_cutoff : hint_range(0.0, 1.0) = 0.1;
uniform float fog_far_cutoff : hint_range(0.0, 1.0) = 1.0;
void fog() {
ALBEDO = fog_color;
float dist = distance(UVW.xz, vec2(0.5));
if(enable_y) {
dist = distance(UVW, vec3(0.5));
}
DENSITY = smoothstep(fog_near_cutoff, fog_far_cutoff, dist);
}If anyone needs it for anything, this might provide a stop-gap solution but I was able to make it work for all of the use-cases that were brought to my attention as issues with 4.0 for retro-style games, including full distance occlusion, and allowing sky penetration by setting the WorldEnvironment "Sky Affect" value to 0.0, or as low as desired to allow sky penetration...
|
|
The volumetric fog solution only works on the clustered Vulkan renderer where volumetric fog is available. This effectively means that it only works on native desktop platforms and not on Web or Mobile which are arguably the two platforms that would benefit the most from this type of fog. I would also like to voice my support for a built-in solution for this. My project is a relatively simple 3D open world game that loads in chunks of the world as you move around. I am targeting Web and Mobile platforms first. I want to hide the chunk pop-in and I simply do not have the time for crazy LOD solutions or anything like that. Fog cutoff would be a very quick and easy way to achieve this. Most Godot users create simple games and this option would be a boon to them. |
|
Closing in favor of #66030, which seems to be the favored solution nowadays (while also being more flexible). |
This is useful for open world games to smoothly fade out distant scenery in the fog. Exponential fog cannot do this on its own, so quadratic fog is combined with exponential fog using a
max()function. The quadratic fog start distance is automatically set at 50% of the camera's Z far distance, which I've found to be the best compromise between visibility and smoothness.If you don't want quadratic fog, you can increase the Camera's Z far property enough so that the linear fog starts outside the play area.
This closes godotengine/godot-proposals#3429.
Testing project: test_fog_fade.zip
Set the editor camera's Z far distance to 50 in the View menu at the top of the 3D editor.
Preview
No fog
Exponential fog (current)
Exponential fog + quadratic fog (new)
Quadratic fog only (exponential density = 0)
This can be used for pure quadratic fog when having fog up close is undesired.