CMAKE using -D CMAKE_INSTALL_PREFIX=install will not copy drjit to Install folder.

[ssh4net]

Summary

System configuration

System information:

OS: Windows-10
CPU: Intel64 Family 6 Model 85 Stepping 4, GenuineIntel
GPU: NVIDIA GeForce GTX 1080 Ti
NVIDIA GeForce GTX 1080 Ti
NVIDIA GeForce GTX 1080 Ti
NVIDIA GeForce GTX 1080 Ti
Python: 3.11.9 | packaged by conda-forge | (main, Apr 19 2024, 18:27:10) [MSC v.1938 64 bit (AMD64)]
NVidia driver: 560.70
CUDA: 11.7.64
LLVM: 14.-1.-1

Dr.Jit: 0.4.6
Mitsuba: 3.5.2
Is custom build? False
Compiled with: MSVC 19.40.33811.0
Variants:
scalar_mono
scalar_mono_polarized
scalar_rgb
scalar_rgb_polarized
scalar_spectral
scalar_spectral_polarized
llvm_mono
llvm_mono_polarized
llvm_rgb
llvm_rgb_polarized
llvm_spectral
llvm_spectral_polarized
llvm_ad_mono
llvm_ad_mono_polarized
llvm_ad_rgb
llvm_ad_rgb_polarized
llvm_ad_spectral
llvm_ad_spectral_polarized
cuda_mono
cuda_mono_polarized
cuda_rgb
cuda_rgb_polarized
cuda_spectral
cuda_spectral_polarized
cuda_ad_mono
cuda_ad_mono_polarized
cuda_ad_rgb
cuda_ad_rgb_polarized
cuda_ad_spectral
cuda_ad_spectral_polarized

System information:

OS: Windows 10 Pro
CPU: AMD
GPU: 3090ti
Python version: 3.11

Dr.Jit version: 0.4.6
Mitsuba version: 3.5.2
Compiled with: VS 2022
Variants compiled: scalar, llvm, cuda all (except mono and double)

Description

If you define installation folder in Cmake script will not copy drjit (and maybe other external packages) to this folder

Steps to reproduce

cmake -DCMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=install ..
cmake --build . --config Release --target install -j8

[KBentley57]

I can confirm. I was having trouble importing mitsuba in python, as it said "no dr jit found" basically.

In the install prefix, there is a mitsuba folder, which is installed correctly. The corresponding dr jit folder is missing. There is a workaround, however. Copy the folder build/python/drjit to the installation root, alongside the mitsuba folder, or put it with the rest of the python site packages.

I am working on fixing this myself as we speak in the drjit cmake. That whole section looks like it could use a little attention.

[njroussel]

Hi

This is not something we support, or intend to support. As per the docs, we recommend building locally and the altering the PYTHONPATH through the scripts we provide.

Is there a specific reason why the procedure above doesn't fit your needs? The build system is already ridiculously complex, I'd prefer not to add any new paths.

[KBentley57]

@njroussel

It's the mashup of the two that is problematic, and it currently doesnt seem to work "as is". On install, the mitsuba folder is installed to CMAKE_INSTALL_PREFIX, and yet the mitsuba module can't be imported in python because there's no default visibility into the drjit folder. That may be in the documentation, but a seasoned linux user would never expect to have to do that. Furthermore, a user should be able to remove the build directory after making an install target, and the program should still work - if we point PYTHONPATH into the build directory this promise is broken, and that's no good.

In a typical install, i would reasonably expect $make install to place things like this

bin/mitsuba (and related binaries)
lib/mitsuba/plugins (instead of the bin/plugins)
lib/python$PYTHONVERSION/{mitsuba,drjit}
lib/cmake/mitsuba
include/mitsuba
share/doc/mitsuba

That's the standard "LFSH" way, and right now it just feels a little out of whack is all. I appreciate the complexity of the situation though, and I'm sure it's difficult to manage the cross platform components with all of the different variants that can be produced.

[njroussel]

Sorry if I wasn't clear, what I meant was that installation is not supported "locally" through CMake. The existing CMake install directives that are present are used by skbuild which is only really used to build wheels that will be found on PyPI. Having extra/different directives for local installation is what is missing. In addition, this is why the install commands only install mitsuba and not drjit, because it is expected to be downloaded by pip install mitsuba as a dependency.

However this also means that installing through that Python toolchain is supported. You can run python setup.py install or pip install . which will trigger the skbuild path. This has some limitations (see #473 and other relevant discussions).

[KBentley57]

@njroussel
I see now, that makes sense, from the perspective of how it's built with skbuild. However, it's so close to working with the 'local cmake install' that I think the group should consider supporting it. I've only had to make a very small number of changes in the CMake, and a handful in the mitsuba.cpp src to look for the installation paths (fileregistry). Moving the python folders to the appropriate places, etc. It's maybe 100 lines or less of code combined, and it would certainly make it more flexible. I didn't read the docs, admittedly, so i missed the part about skbuild. I mostly use the c++ side with mitsuba directly, so it never crossed my mind to actually have python be a part of the build, though i know the python bindings were being generated.