Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

On windows 10, bug on _check_executable and on formatting a ffmpeg cmd to execute #2470

Open
dtraparic opened this issue Dec 5, 2022 · 1 comment
Assignees

Comments

@dtraparic
Copy link

dtraparic commented Dec 5, 2022

Hello, hope this will be useful for someone.

Checklist

  1. I have searched related issues but cannot get the expected help: Yes
  2. I have read the FAQ documentation but cannot get the expected help: Yes
  3. The unexpected results still exist in the latest version.: Yes

Describe the Issue
Two bugs here can be triggered by the same code snippet on Windows.

  • First bug : When a mmcv function needs an external command (ffmpeg in my case), it triggers mmcv.utils.py : check_prerequisites() function, which calls _check_executable() for the existence of ffmpeg, by testing "which ffmpeg" as a shell command.
    As which is a linux only command, this triggers a "RuntimeError: Prerequisites not meet."

  • Second bug : When a mmcv function is formatting a shell command for ffmpeg, it seems to lack robustness on spaces in Windows paths.

Reproduction

  1. What command, code, or script did you run?
    The two bugs can be triggered with :
mmcv.video.resize_video(in_file='C:/Users/Name Surname/Documents/prog/test_windows_linux_mmcv/input/in.mp4',
                        out_file='C:/Users/Name Surname/Documents/prog/test_windows_linux_mmcv/output/out.mp4',
                        size=(512, 214),
                        log_level='warning')
  1. Did you make any modifications on the code? Did you understand what you have modified?
    No modification have been made to trigger the errors.

Environment

  1. Please run python -c "from mmcv.utils import collect_env; print(collect_env())" to collect necessary environment information and paste it here.

I'm using Windows 10, and this is the result of collect_env():

'tail' n'est pas reconnu en tant que commande interne
ou externe, un programme ex�cutable ou un fichier de commandes.
'gcc' n'est pas reconnu en tant que commande interne
ou externe, un programme ex�cutable ou un fichier de commandes.
{'sys.platform': 'win32', 'Python': '3.9.12 (main, Apr  4 2022, 05:22:27) [MSC v.1916 64 bit (AMD64)]', 'CUDA available': True, 'GPU 0': 'NVIDIA GeForce GTX 1660 Ti', 'CUDA_HOME': 'C:\\Program Files\\NVIDIA GPU Computing Toolkit\\CUDA\\v11.3', 'NVCC': 'Not Available', 'GCC': 'n/a', 'PyTorch': '1.11.0', 'PyTorch compiling details': 'PyTorch built with:\n  - C++ Version: 199711\n  - MSVC 192829337\n  - Intel(R) Math Kernel Library Version 2020.0.2 Product Build 20200624 for Intel(R) 64 architecture applications\n  - Intel(R) MKL-DNN v2.5.2 (Git Hash a9302535553c73243c632ad3c4c80beec3d19a1e)\n  - OpenMP 2019\n  - LAPACK is enabled (usually provided by MKL)\n  - CPU capability usage: AVX2\n  - CUDA Runtime 11.3\n  - NVCC architecture flags: -gencode;arch=compute_37,code=sm_37;-gencode;arch=compute_50,code=sm_50;-gencode;arch=compute_60,code=sm_60;-gencode;arch=compute_61,code=sm_61;-gencode;arch=compute_70,code=sm_70;-gencode;arch=compute_75,code=sm_75;-gencode;arch=compute_80,code=sm_80;-gencode;arch=compute_86,code=sm_86;-gencode;arch=compute_37,code=compute_37\n  - CuDNN 8.2\n  - Magma 2.5.4\n  - Build settings: BLAS_INFO=mkl, BUILD_TYPE=Release, CUDA_VERSION=11.3, CUDNN_VERSION=8.2.0, CXX_COMPILER=C:/cb/pytorch_1000000000000/work/tmp_bin/sccache-cl.exe, CXX_FLAGS=/DWIN32 /D_WINDOWS /GR /EHsc /w /bigobj -DUSE_PTHREADPOOL -openmp:experimental -IC:/cb/pytorch_1000000000000/work/mkl/include -DNDEBUG -DUSE_KINETO -DLIBKINETO_NOCUPTI -DUSE_FBGEMM -DUSE_XNNPACK -DSYMBOLICATE_MOBILE_DEBUG_HANDLE -DEDGE_PROFILER_USE_KINETO, LAPACK_INFO=mkl, PERF_WITH_AVX=1, PERF_WITH_AVX2=1, PERF_WITH_AVX512=1, TORCH_VERSION=1.11.0, USE_CUDA=ON, USE_CUDNN=ON, USE_EXCEPTION_PTR=1, USE_GFLAGS=OFF, USE_GLOG=OFF, USE_MKL=ON, USE_MKLDNN=OFF, USE_MPI=OFF, USE_NCCL=OFF, USE_NNPACK=OFF, USE_OPENMP=ON, USE_ROCM=OFF, \n', 'TorchVision': '0.12.0', 'OpenCV': '4.5.4', 'MMCV': '1.4.8', 'MMCV Compiler': 'MSVC 192930143', 'MMCV CUDA Compiler': '11.3'}
  1. You may add addition that may be helpful for locating the problem, such as
    I installed in a conda environment, with
conda install pip
pip install mmcv-full==1.4.8

Error traceback
First bug traceback :

'which' n'est pas reconnu en tant que commande interne
ou externe, un programme ex�cutable ou un fichier de commandes.
Traceback (most recent call last):
  File "C:\Users\David Traparic\Documents\prog\test_windows_linux_mmcv\trigger_bug_2.py", line 27, in <module>
    mmcv.video.resize_video(in_file='C:/Users/David Traparic/Documents/prog/test_windows_linux_mmcv/input/in.mp4',
  File "C:\Users\David Traparic\miniconda3\envs\testmmcv\lib\site-packages\mmcv\utils\misc.py", line 235, in wrapped_func
Prerequisites "ffmpeg" are required in method "resize_video" but not found, please install them first.
    raise RuntimeError('Prerequisites not meet.')
RuntimeError: Prerequisites not meet.

Second bug traceback (when first is fixed):

C:/Users/David: No such file or directory

Bug fix

  • First bug fix :
    In mmcv.utils.misc.py,
def _check_executable(cmd):
    if subprocess.call(f'which {cmd}', shell=True) != 0:
        return False
    else:
        return True 

can be replaced by this to fix it:

import platform
def _check_executable(cmd):
    if platform.system() == 'Linux':
        if subprocess.call(f'which {cmd}', shell=True) != 0:
            return False
        else:
            return True
    if platform.system() == 'Windows':
        if subprocess.call(f'where {cmd}', shell=True) != 0:
            return False
        else:
            return True
  • Second bug :
    In mmcv.video.processing.py, inside convert_video() function,
cmd = f'ffmpeg -y {pre_options} -i {in_file} {" ".join(options)} ' \
          f'{out_file}'

can be replaced by this to fix it:

cmd = f'ffmpeg -y {pre_options} -i "{in_file}" {" ".join(options)} ' \
          f'"{out_file}"'

I have not tested the fixes on Linux, so I can't be sure it does not break anything.

@HAOCHENYE
Copy link
Collaborator

Soooo sorry for my late reply and thanks for your feedback! Would you mind posting a PR to fix this, this solution looks good to me 😆 !

dtraparic pushed a commit to dtraparic/mmcv that referenced this issue Sep 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants