Slow Render Time on Scenes with 24k+ Objects

[Microno95]

On large scenes, around 24000 objects, the recording of the computation graph dominates the render time (~50s to record the computation and ~8 seconds to render) and when running any kind of optimisation leads to a relatively slow iteration time.

In my case, I'm working with the optimisation of a texture used to derive the volumetric emission, and the only change between iterations is the values inside this bitmap texture.

I was wondering if there's any way to speed this up as the render kernel should not change between iterations (which I've checked by looking at the cache hit/miss rate in drjits log output) - or perhaps is this a limitation of how kernels are cached?

[wjakob]

The cost here is going through the entire scene to discover that the computation is still the same. So the cache is likely working, but it's the computation of the cache key that is costly.
Mitsuba has a merge shape that automatically groups all compatible geometry to reduce such costs, can you give this a try? Simply specify it at the top level and nest all your scene objects within.

[Microno95]

Right, that's roughly what I thought and it makes sense. The merge shape has sped up things dramatically as it's now ~400ms to generate the kernel rather than ~40s which has helped greatly, thank you so much! Didn't realise it existed since it seems to be undocumented 😅