fix: runpy.run_path(path) crashes with pathlib.Path

[askhl]

I ran into the following problem where coverage replaces runpy.run_path() in a way that does not accept Path objects:

import runpy

script = "print('hello, world!')\n"

def test_path(tmp_path):
    pyfile = tmp_path / 'script.py'
    pyfile.write_text(script)
    runpy.run_path(pyfile)

This passes with ordinary pytest but crashes with pytest --cov with the following traceback:

Traceback (most recent call last):
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/_pytest/runner.py", line 341, in from_call
    result: Optional[TResult] = func()
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/flaky/flaky_pytest_plugin.py", line 146, in <lambda>
    lambda: ihook(item=item, **kwds), when=when, reraise=reraise
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_callers.py", line 182, in _multicall
    return outcome.get_result()
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_result.py", line 100, in get_result
    raise exc.with_traceback(exc.__traceback__)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/_pytest/runner.py", line 177, in pytest_runtest_call
    raise e
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/_pytest/runner.py", line 169, in pytest_runtest_call
    item.runtest()
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/_pytest/python.py", line 1792, in runtest
    self.ihook.pytest_pyfunc_call(pyfuncitem=self)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_hooks.py", line 513, in __call__
    return self._hookexec(self.name, self._hookimpls.copy(), kwargs, firstresult)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_manager.py", line 120, in _hookexec
    return self._inner_hookexec(hook_name, methods, kwargs, firstresult)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_callers.py", line 139, in _multicall
    raise exception.with_traceback(exception.__traceback__)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/pluggy/_callers.py", line 103, in _multicall
    res = hook_impl.function(*args)
  File "/home/askhl/install/pyenv/lib/python3.10/site-packages/_pytest/python.py", line 194, in pytest_pyfunc_call
    result = testfunction(**testargs)
  File "/home/askhl/coverage-runpy/test.py", line 11, in test_path
    runpy.run_path(pyfile)
  File "/usr/lib/python3.10/runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/askhl/src/coveragepy/coverage/control.py", line 395, in _should_trace
    disp = self._inorout.should_trace(filename, frame)
  File "/home/askhl/src/coveragepy/coverage/inorout.py", line 324, in should_trace
    filename = source_for_file(dunder_file)
  File "/home/askhl/src/coveragepy/coverage/python.py", line 107, in source_for_file
    if filename.endswith(".py"):
AttributeError: 'PosixPath' object has no attribute 'endswith'

This PR contains a minimal fix which sends the input through str(Path(...)). However a proper fix might require some awareness of the surrounding code as well as a test, so I'd expect the PR in its current form to be insufficient. Feedback would be appreciated.

[nedbat]

Thanks, I see there is a problem. I'd want to understand the full call stack to know where I should be accepting Path|str instead of just str.

[askhl]

Thanks @nedbat!

It crossed my mind that it could originate with pytest or pytest-cov, since there's something finicky going on between runpy and the code object that could originate from some out-of-sight hook. I reproduced it now with only coverage run:

import runpy
from pathlib import Path

script = "print('hello, world!')\n"
pyfile = Path('script.py')
pyfile.write_text(script)
runpy.run_path(pyfile)

This yields the considerably shorter stacktrace

(pyenv) askhl@alberich:~/coverage-runpy$ coverage run test.py 
Traceback (most recent call last):
  File "/home/askhl/coverage-runpy/test.py", line 8, in <module>
    runpy.run_path(pyfile)
  File "/usr/lib/python3.10/runpy.py", line 289, in run_path
    return _run_module_code(code, init_globals, run_name,
  File "/usr/lib/python3.10/runpy.py", line 96, in _run_module_code
    _run_code(code, mod_globals, init_globals,
  File "/usr/lib/python3.10/runpy.py", line 86, in _run_code
    exec(code, run_globals)
  File "/home/askhl/src/coveragepy/coverage/control.py", line 395, in _should_trace
    disp = self._inorout.should_trace(filename, frame)
  File "/home/askhl/src/coveragepy/coverage/inorout.py", line 324, in should_trace
    filename = source_for_file(dunder_file)
  File "/home/askhl/src/coveragepy/coverage/python.py", line 107, in source_for_file
    if filename.endswith(".py"):
AttributeError: 'PosixPath' object has no attribute 'endswith'

After some messing around I found that usually the function receives a str, but it receives a Path('script.py') when it does dunder_file = frame.f_globals and frame.f_globals.get("__file__").

I pushed a change inserting a conversion there instead of in the previous place.

So that fix will equally well solve the problem without the need to think about Path|str in multiple places, and hence is a more credible fix.

But I don't know if there's any documentation that could reveal what types frame.f_globals.get('__file__') should have.

[nedbat]

It crossed my mind that it could originate with pytest or pytest-cov

No worries, when a bug report mentions pytest-cov, the first thing I do is try to reproduce it without it. I also saw it with just coverage run.

I like the latest change: it feels like the right place.

Do you want to add yourself to CONTRIBUTORS and add a changelog entry? I can take care of it if you like.

[askhl]

I have added a test in what I perceive to be a reasonably correct place.

Do you want to add yourself to CONTRIBUTORS and add a changelog entry? I can take care of it if you like.

Thanks. Since the list of unreleased changes is empty, I was not sure whether to add a note before or after .. scriv-start-here, or whether that's necessarily the correct place. Please make any changes you think appropriate.

[askhl]

I just pushed a lint-fixing change but you have apparently already done something about it. Feel free to ignore/adapt/close this.

I am not sure whether this should actually be considered an issue with runpy. The docs (https://docs.python.org/3/reference/import.html#file__) say rather clearly that __file__ should be str if it exists.

Thanks a lot!

[nedbat]

I also beefed up the test a bit in askhl#1. You can merge that pull request to get my changes.

[askhl]

I also beefed up the test a bit in askhl#1. You can merge that pull request to get my changes.

Nice. It's merged.

[nedbat]

Thanks!

[nedbat]

This is now released as part of coverage 7.6.1.