Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libvmaf with CUDA -- how to build and use #1227

Open
DonTequila opened this issue Aug 18, 2023 · 11 comments
Open

libvmaf with CUDA -- how to build and use #1227

DonTequila opened this issue Aug 18, 2023 · 11 comments
Labels

Comments

@DonTequila
Copy link

Is there a documentation explaining how to build libvmaf with support for CUDA and how it's been used?

@DonTequila
Copy link
Author

Hi all, can somebody please point me to some documentation about building libvmaf with CUDA support if it exists? Thanks!
Also, do I need a NVIDIA card for building it?

@gedoensmax
Copy link
Contributor

You don't need one for building, but you will need a card for executing. Also you need driver stub libraries, which are included in a CUDA docker container.

@gedoensmax
Copy link
Contributor

Could you ldd ffmpeg to see if it really loads the correct libraries ? Whenever that happened it just loaded the wrong .so for me.

@gedoensmax
Copy link
Contributor

gedoensmax commented Sep 17, 2023

Wait did you apply https://ffmpeg.org/pipermail/ffmpeg-devel/attachments/20230807/29cf249a/attachment.obj to ffmpeg ? See #1232.

@zwayn
Copy link

zwayn commented Sep 26, 2023

I successfully built it on Linux with meson setup -Denable_cuda=true libvmaf/build libvmaf ninja -C libvmaf/build install

sudo ln -s /opt/cuda/ /usr/local/cuda was also required on my operating system before those two commands, but that's distribution specific. I use Arch Linux.

Standalone vmaf now successfully uses CUDA, by default.

However, vmaf in ffmpeg and aomenc don't... Both are freshly compiled.

I tried that however I still got a compilation error:

`[183/206] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o 
nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Xcompiler=-fPIC -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
[187/206] Compiling C object test/test_cambi.p/test_cambi.c.o
../test/test_cambi.c: In function ‘get_sample_image’:
../test/test_cambi.c:59:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
   59 |     int err, count = 0;
      |         ^~~
../test/test_cambi.c: In function ‘get_sample_image_8b’:
../test/test_cambi.c:82:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
   82 |     int err, count = 0;
      |         ^~~
../test/test_cambi.c: In function ‘get_sample_image_8x8’:
../test/test_cambi.c:94:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
   94 |     int err, count = 0;
      |         ^~~
[189/206] Compiling C++ object test/test_predict.p/.._src_svm.cpp.o
ninja: build stopped: subcommand failed.`

Any guess ?

@HunterAP23
Copy link

HunterAP23 commented Oct 30, 2023

I've been trying to build libvmaf with CUDA support into a Windows build that's being cross-compiled on an Ubuntu WSL / Docker container. I was doing this through ffmpeg-windows-build-helpers.

After installing CUDA (I tried 10.2, 11.6, 11.8, and 12.2) and NVCC, along with all the requirements for both libvmaf and the ffmpeg builder tool, I modified the ffmpeg build script to do the following three things:

  1. Added the -Denable_cuda=true flag to the libvmaf compile options
  2. Added --enable-nonfree --enable-ffnvcodec to the ffmpeg configuration options
  3. Changed the script to pull the latest commit of the main branch rather than a specific release

After doing that, I get this issue:

The Meson build system
Version: 1.2.3
Source dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf
Build dir: /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build
Build type: cross build
Project name: libvmaf
Project version: 2.3.1
C compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc (gcc 10.2.0 "x86_64-w64-mingw32-gcc (GCC) 10.2.0")
C linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-gcc ld.bfd 2.36.1
C++ compiler for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ (gcc 10.2.0 "x86_64-w64-mingw32-g++ (GCC) 10.2.0")
C++ linker for the host machine: /ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++ ld.bfd 2.36.1
C compiler for the build machine: cc (gcc 7.5.0 "cc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C linker for the build machine: cc ld.bfd 2.30
C++ compiler for the build machine: c++ (gcc 7.5.0 "c++ (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0")
C++ linker for the build machine: c++ ld.bfd 2.30
Build machine cpu family: x86_64
Build machine cpu: x86_64
Host machine cpu family: x86_64
Host machine cpu: x86_64
Target machine cpu family: x86_64
Target machine cpu: x86_64
Check usable header "stdatomic.h" : YES
Configuring version.h using configuration
Program nasm found: YES (/usr/bin/nasm)
Configuring config.asm using configuration
Program xxd found: YES (/usr/bin/xxd)
Run-time dependency CUDA found: NO

src/meson.build:162:22: ERROR: Dependency lookup for cuda with method 'system' failed: Couldn't find requested CUDA module 'cudart_static'

A full log can be found at /ffmpeg-windows-build-helpers/sandbox/win64/vmaf_git/libvmaf/build/meson-logs/meson-log.txt

I fixed the above issue by copying the libcudart_static.a file that gets added to my system when installing Cuda 11.6 through the cuda-toolkit-11-6 package. For me this file is in both /usr/local/cuda-11.6/targets/x86_64-linux/lib and /usr/lib/x86_64-linux-gnu, and I copied this file to the ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/x86_64-w64-mingw32/lib directory.

After doing this, the cross_compile_ffmpeg.sh script to change what normally gets written to meson-cross.mingw.txt by changing the following section:

[binaries]
c = '${cross_prefix}gcc'
cpp = '${cross_prefix}g++'
ld = '${cross_prefix}ld'
ar = '${cross_prefix}ar'
strip = '${cross_prefix}strip'
pkgconfig = '${cross_prefix}pkg-config'
nm = '${cross_prefix}nm'
windres = '${cross_prefix}windres'

and adding cuda = '/usr/bin/nvcc' to the bottom.

Afterwards attempting to run the compile script again, I'm met with a different error, this one from libvmaf itself:

[2/19] Linking target test/test_picture.exe
FAILED: test/test_picture.exe
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/bin/x86_64-w64-mingw32-g++  -o test/test_picture.exe test/test_picture.exe.p/test.c.obj test/test_picture.exe.p/test_picture.c.obj test/test_picture.exe.p/.._src_picture.c.obj test/test_picture.exe.p/.._src_mem.c.obj test/test_picture.exe.p/.._src_ref.c.obj test/test_picture.exe.p/.._src_thread_pool.c.obj -Wl,--allow-shlib-undefined -Wl,-O1 -pthread -Wl,--start-group -lcuda /home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/lib/../lib/libcudart_static.a -L/usr/local/cuda/lib/x64 -Wl,--subsystem,console -lkernel32 -luser32 -lgdi32 -lwinspool -lshell32 -lole32 -loleaut32 -luuid -lcomdlg32 -ladvapi32 -Wl,--end-group
/home/hunterap/ffmpeg-windows-build-helpers/sandbox/cross_compilers/mingw-w64-x86_64/lib/gcc/x86_64-w64-mingw32/10.2.0/../../../../x86_64-w64-mingw32/bin/ld: cannot find -lcuda
collect2: error: ld returned 1 exit status
[3/19] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj
/usr/bin/nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.obj -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
ninja: build stopped: subcommand failed.

Not sure what the -lcuda option does specifically, but it seems to look for CUDA even though it was able to find it earlier.

I've relayed this info to both the ffmpeg-windows-build-helpers and the media-autobuild_suite repos. The former can't do much since they can't release builds with nonfree features. I'll check later today if the changes I listed above work with media-autobuild_suite

@Xosrov
Copy link

Xosrov commented Jan 7, 2024

I successfully built it on Linux with meson setup -Denable_cuda=true libvmaf/build libvmaf ninja -C libvmaf/build install
sudo ln -s /opt/cuda/ /usr/local/cuda was also required on my operating system before those two commands, but that's distribution specific. I use Arch Linux.
Standalone vmaf now successfully uses CUDA, by default.
However, vmaf in ffmpeg and aomenc don't... Both are freshly compiled.

I tried that however I still got a compilation error:

`[183/206] Compiling Cuda object src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o
FAILED: src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o 
nvcc -Isrc/libcuda_common_vmaf_lib.a.p -Xcompiler=-Wall,-Winvalid-pch,-Wextra -O3 -Xcompiler=-fPIC -Isrc -I/usr/local/cuda/include -I/usr/local/cuda/include -I../src/cuda -I../src/feature -I../src/cuda -I../src/feature/common -I../src -Isrc -I../src/feature/common -I../src/feature -I../src -Isrc -I../include -Iinclude -I../src -Isrc -I../src -Isrc -Isrc/libcuda_common_vmaf_lib.a.p -o src/libcuda_common_vmaf_lib.a.p/feature_cuda_integer_adm_adm_cm.cu.o -c ../src/feature/cuda/integer_adm/adm_cm.cu
../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

../src/feature/cuda/integer_adm/adm_cm.cu(52): warning #68-D: integer conversion resulted in a change of sign

/usr/include/c++/11/bits/std_function.h:435:145: error: parameter packs not expanded with ‘...’:
  435 |         function(_Functor&& __f)
      |                                                                                                                                                 ^ 
/usr/include/c++/11/bits/std_function.h:435:145: note:         ‘_ArgTypes’
/usr/include/c++/11/bits/std_function.h:530:146: error: parameter packs not expanded with ‘...’:
  530 |         operator=(_Functor&& __f)
      |                                                                                                                                                  ^ 
/usr/include/c++/11/bits/std_function.h:530:146: note:         ‘_ArgTypes’
[187/206] Compiling C object test/test_cambi.p/test_cambi.c.o
../test/test_cambi.c: In function ‘get_sample_image’:
../test/test_cambi.c:59:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
   59 |     int err, count = 0;
      |         ^~~
../test/test_cambi.c: In function ‘get_sample_image_8b’:
../test/test_cambi.c:82:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
   82 |     int err, count = 0;
      |         ^~~
../test/test_cambi.c: In function ‘get_sample_image_8x8’:
../test/test_cambi.c:94:9: warning: variable ‘err’ set but not used [-Wunused-but-set-variable]
   94 |     int err, count = 0;
      |         ^~~
[189/206] Compiling C++ object test/test_predict.p/.._src_svm.cpp.o
ninja: build stopped: subcommand failed.`

Any guess ?

I have the same problem, Ubuntu 22.04, CUDA 12.3. It seems this is a problem with gcc and nvcc compaibility (see here)

@gedoensmax
Copy link
Contributor

@Xosrov the Dockerfile in #1330 builds on 22.04 and 12.3 without any problems on my side. Could you try this ? I know that it might not be the best solution for your case, but at least a sample to go by ?

@alessandroberna
Copy link

@damian101 I think i had your same issue. I compiled both vmaf and FFmpeg on Arch from the latest git version, however the filter libvmaf_cuda was missing from the freshly built ffmpeg.

It turns out that ninja -C libvmaf/build install installs to /usr/local/ (so /usr/local/include, /usr/local/lib and /usr/local/bin), ffmpeg on the other hand seems to look for libraries inside /usr/include and /usr/lib first.
It's thus very likely that you were compiling ffmpeg with the version of libvmaf that's included on the arch repos, which is compiled without -Denable_cuda=true.

Before compiling FFmpeg, take a look at the output of the configure script, you should see libvmaf_cuda listed under Enabled filters:, if that's not the case you either forgot to enable the necessary configurations or it's not using the right library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

8 participants
@HunterAP23 @zwayn @DonTequila @nilfm @gedoensmax @Xosrov @alessandroberna and others