Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

merge_pyi crashes with keyword-only args in function def. #1153

Closed
mrolle45 opened this issue Mar 20, 2022 · 1 comment
Closed

merge_pyi crashes with keyword-only args in function def. #1153

mrolle45 opened this issue Mar 20, 2022 · 1 comment
Labels
bug cat: tools pytype-based tools (merge-pyi, annotate-ast, etc.)

Comments

@mrolle45
Copy link

Note: I have included a patch to the code that fixes this bug, but as I am not familiar with pull requests and tests, I would like some reader to apply this patch. Thanks.

The source of the function def is

def foo(self, *, kwd: bool = False) -> int:
	return 42

The exception traceback:

Traceback (most recent call last):
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\main.py", line 101, in <module>
    main()
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\main.py", line 82, in main
    annotated_src = merge_pyi.annotate_string(args, py_src, pyi_src)
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\merge_pyi.py", line 928, in annotate_string
    tree = tool.refactor_string(py_src + '\n', '<inline>')
  File "D:\Python37\lib\lib2to3\refactor.py", line 367, in refactor_string
    self.refactor_tree(tree, name)
  File "D:\Python37\lib\lib2to3\refactor.py", line 441, in refactor_tree
    new = fixer.transform(node, results)
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\merge_pyi.py", line 518, in transform
    src_sig = FuncSignature(node, results)
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\merge_pyi.py", line 306, in __init__
    self._arg_sigs = tuple(map(ArgSignature, args))
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\merge_pyi.py", line 123, in __init__
    sig = ArgSignature._split_arg(arg_nodes)
  File "D:\Solutions\trunk\Python\Pytype\pytype\tools\merge_pyi\merge_pyi.py", line 240, in _split_arg
    raise KnownError()  # expected/parse_error.py
pytype.tools.merge_pyi.merge_pyi.KnownError

The bug
_split_arg is not prepared for a Texan input (i.e. lone star :) ). It receives an arg list with only the "*" leaf.
After stripping off this leaf, the arg list is now empty.

Proposed fix
In this case, make a return with the '*' as stars, and an empty arg. Further processing is not harmed by this.
Here is the change I made to _split_arg lines 239-240:

    if len(arg) != 1:
      if not arg and stars == '*':                                  # added
        return is_tuple, stars, arg_type, arg, default              # added
      raise KnownError()  # expected/parse_error.py

Here is the resulting output, using unannotated function def in the .pyi file:

def foo(self, *, kwd: bool = False) -> int:
        return 42

FYI: Here is a pretty-print I made of the parse tree used in this example:

Node(funcdef)
:   Leaf(1, 'def')
:   Leaf(1, 'foo')
:   Node(parameters)
:   :   Leaf(7, '(')
:   :   Node(typedargslist)
:   :   :   Leaf(1, 'self')
:   :   :   Leaf(12, ',')
:   :   :   Leaf(16, '*')
:   :   :   Leaf(12, ',')
:   :   :   Node(tname)
:   :   :   :   Leaf(1, 'kwd')
:   :   :   :   Leaf(11, ':')
:   :   :   :   Leaf(1, 'bool')
:   :   :   Leaf(22, '=')
:   :   :   Leaf(1, 'False')
:   :   Leaf(8, ')')
:   Leaf(55, '->')
:   Leaf(1, 'int')
:   Leaf(11, ':')
:   Node(suite)
:   :   Leaf(4, '\n')
:   :   Leaf(5, '\t')
:   :   Node(simple_stmt)
:   :   :   Node(return_stmt)
:   :   :   :   Leaf(1, 'return')
:   :   :   :   Leaf(2, '42')
:   :   :   Leaf(4, '\n')
:   :   Leaf(6, '')```
@martindemello
Copy link
Contributor

thanks! we are currently rewriting merge_pyi completely on top of libcst, but will apply this patch in the mean time so that it's a bit more usable.

@rchen152 rchen152 added bug cat: callables functions and decorators labels Mar 21, 2022
@rchen152 rchen152 added cat: tools pytype-based tools (merge-pyi, annotate-ast, etc.) and removed cat: callables functions and decorators labels Mar 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug cat: tools pytype-based tools (merge-pyi, annotate-ast, etc.)
Projects
None yet
Development

No branches or pull requests

3 participants