"OSError: [WinError 0] The operation completed successfully." (PyCharm, Windows)

[Jonibhoni]

Describe the bug
Hey, first, thanks a lot for enlighten, I love the ideas you have put into it! :)

I have the problem that a call to get_manager() will lead to the following error in my PyCharm console output WITH terminal emulation on Windows:

Traceback (most recent call last):
File "C:\Users\jschoena\AppData\Roaming\JetBrains\PyCharm2022.1\scratches\Test enlighten.py", line 4, in
enlighten_manager = enlighten.get_manager(stream=sys.stdout, companion_stream=sys.stderr)
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\enlighten\manager.py", line 55, in get_manager
return Manager(stream=stream, counter_class=counter_class, **kwargs)
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\enlighten_manager.py", line 70, in init
super(Manager, self).init(**kwargs)
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\enlighten_basemanager.py", line 74, in init
self.term = Terminal(stream=self.stream, kind=kind, force_styling=bool(kind))
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\blessed\terminal.py", line 208, in init
self.__init__keycodes()
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\blessed\terminal.py", line 331, in __init__keycodes
self._encoding = get_console_input_encoding()
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\jinxed\win32.py", line 140, in get_console_input_encoding
encoding = 'cp%d' % KERNEL32.GetConsoleCP()
File "C:\Users\jschoena.conda\envs\main\lib\site-packages\jinxed\win32.py", line 86, in _check_bool
raise ctypes.WinError(ctypes.get_last_error())
OSError: [WinError 0] The operation completed successfully.

Process finished with exit code 1

Even if I would catch the exception, the enlighten manager object is not created, thus I cannot continue from here.

To Reproduce
This snippet is enough:

import enlighten

enlighten_manager = enlighten.get_manager()

print("Doesn't arrive here!")

Environment:

• Enlighten Version: 1.10.2 pyhd8ed1ab_0 conda-forge
• OS and version: Windows Server 2019 Standard, 1809, OS build 17763.2803
• Console application: PyCharm 2022.1, "Emulate terminal in output console" checked
• Special Conditions: None, just executing script within IDE
• Related packages:
  • blessed 1.19.1 py310h5588dad_1 conda-forge
  • python 3.10.4 hbb2ffb3_0 defaults
  • jinxed 1.1.0 py310h5588dad_2 conda-forge

[avylove]

Thanks for reporting! That are some other issues with the PYCharm console described in #32, but you shouldn't seeing this error. I'm wondering if there is another error and the value is being overwritten before it's read.

As an experiment, can you modify line 86 of jinxed.win32? It looks like it's located at C:\Users\jschoena.conda\envs\main\lib\site-packages\jinxed\win32.py

Try changing line 86 from

        raise ctypes.WinError(ctypes.get_last_error())

to

        error = ctypes.get_last_error()
        raise ctypes.WinError(error)

[Jonibhoni]

Wow, fast answer! :)

I replaced it as you suggested, but it's the same outcome just that it raises on the raise ctypes.WinError(error) line.

I also printed the value of error out of curiosity and its just an ordinary int with value 0.

Edit: The function parameter result is also 0.

[avylove]

It's strange behavior. The documentation for GetConsoleCP says a result of 0 is an error and GetLastError() should return more information. Can you run chcp in the same console to see what it returns?

And does everything work as expected outside of PyCharm?

[Jonibhoni]

I added a print into the _check_bool(...) function and did some experiments:

Outside PyCharm, same venv, Anaconda Powershell Prompt:

(main) PS C:\> chcp
Active code page: 850
(main) PS C:\> python
Python 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import enlighten
>>> enlighten.get_manager()
Result: 1
Result: 1
Result: 850
Result: 1
Result: 1
Result: 850
Manager(stream=<_io.TextIOWrapper name='<stdout>' mode='w' encoding='utf-8'>)

Within general "Terminal" tab in PyCharm:

Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\Users\jschoena\PycharmProjects\auswertemepper> chcp
Active code page: 0
PS C:\Users\jschoena\PycharmProjects\auswertemepper> C:\Users\jschoena\.conda\envs\main\python.exe
Python 3.10.4 | packaged by conda-forge | (main, Mar 30 2022, 08:38:02) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import enlighten
>>> enlighten.get_manager()
Result: 1
Result: 1
Result: 0 
Traceback (most recent call last):                                                                               
  File "<stdin>", line 1, in <module>                                                                            
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\enlighten\manager.py", line 55, in get_manager      
    return Manager(stream=stream, counter_class=counter_class, **kwargs)                                         
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\enlighten\_manager.py", line 70, in __init__        
    super(Manager, self).__init__(**kwargs)                                                                      
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\enlighten\_basemanager.py", line 74, in __init__    
    self.term = Terminal(stream=self.stream, kind=kind, force_styling=bool(kind))                                
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\blessed\terminal.py", line 208, in __init__         
    self.__init__keycodes()                                                                                      
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\blessed\terminal.py", line 331, in __init__keycodes 
    self._encoding = get_console_input_encoding() \                                                              
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\jinxed\win32.py", line 141, in get_console_input_encoding
    encoding = 'cp%d' % KERNEL32.GetConsoleCP()
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\jinxed\win32.py", line 87, in _check_bool
    raise ctypes.WinError(error)
OSError: [WinError 0] The operation completed successfully.
>>> exit()
Result: 1
PS C:\Users\jschoena\PycharmProjects\auswertemepper> get-host


Name             : ConsoleHost
Version          : 5.1.17763.2803
InstanceId       : 9b8685ff-bf31-4928-acdb-d097e386a9cf
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : de-DE
CurrentUICulture : en-US
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace

I also ran get-host to see the PowerShell version.

As a PyCharm run configuration with terminal emulation enabled, this script...

import os
import enlighten

os.system("chcp")

enlighten_manager = enlighten.get_manager()

print("Doesn't arrive here!")

...results in:

Active code page: 0
Result: 1
Result: 1
Result: 0                                                                                                              
Traceback (most recent call last):                                                                                     
  File "C:\Users\jschoena\AppData\Roaming\JetBrains\PyCharm2022.1\scratches\Test enlighten.py", line 6, in <module>    
    enlighten_manager = enlighten.get_manager()                                                                        
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\enlighten\manager.py", line 55, in get_manager            
    return Manager(stream=stream, counter_class=counter_class, **kwargs)                                               
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\enlighten\_manager.py", line 70, in __init__              
    super(Manager, self).__init__(**kwargs)                                                                            
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\enlighten\_basemanager.py", line 74, in __init__          
    self.term = Terminal(stream=self.stream, kind=kind, force_styling=bool(kind))                                      
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\blessed\terminal.py", line 208, in __init__               
    self.__init__keycodes()                                                                                            
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\blessed\terminal.py", line 331, in __init__keycodes       
    self._encoding = get_console_input_encoding() \                                                                    
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\jinxed\win32.py", line 141, in get_console_input_encoding 
    encoding = 'cp%d' % KERNEL32.GetConsoleCP()                                                                        
  File "C:\Users\jschoena\.conda\envs\main\lib\site-packages\jinxed\win32.py", line 87, in _check_bool                 
    raise ctypes.WinError(error)                                                                                       
OSError: [WinError 0] The operation completed successfully.                                                            
Result: 1              

Process finished with exit code 1

[Jonibhoni]

It seems to be not just a PyCharm thing, but dependent also on the environment. If I try chcp within the same PyCharm version on the "Terminal" tab on my home PC, it works fine:

Copyright (C) Microsoft Corporation. Alle Rechte vorbehalten.

Lernen Sie das neue plattformübergreifende PowerShell kennen – https://aka.ms/pscore6

PS I:\> chcp
Aktive Codepage: 850.
PS I:\> get-host


Name             : ConsoleHost
Version          : 5.1.19041.1645
InstanceId       : 212016ee-f195-4876-8119-0bf2d5ddf59a
UI               : System.Management.Automation.Internal.Host.InternalHostUserInterface
CurrentCulture   : de-DE
CurrentUICulture : de-DE
PrivateData      : Microsoft.PowerShell.ConsoleHost+ConsoleColorProxy
DebuggerEnabled  : True
IsRunspacePushed : False
Runspace         : System.Management.Automation.Runspaces.LocalRunspace

My home PC is a Windows 10 Education, 21H2, 19044.1645.

[avylove]

I'm wondering if it's a Pycharm bug or if there are any scripts running to initialize the environment that may be affecting this. Do you see anything in the config?

To give some scoping, this is initiated by Blessed.Terminal to determine the encoding to use when decoding keyboard input. It mostly relies on Jinxed for this, but if Jinxed can't determine the encoding, it falls back to locale.getpreferredencoding() and then 'UTF-8'. On Windows those last two are actually likely to return the wrong code encoding. os.device_encoding(fd) should return the correct encoding, and isn't in Blessed mainly because Jinxed worked and it's Python 3 only while Blessed supports 2.7+. I assume it uses the same mechanisms as Jinxed and chcp, so probably wouldn't help in this case anyway.

What I'm trying to decide is if this should be worked around by catching the OSError and returning None or if it masks a problem that shouldn't be there.

Out of curiosity, what does os.device_encoding(0) return?

[Jonibhoni]

Regarding os.device_encoding(0):

• On Windows Server 2019 1809, Anaconda Powershell prompt:
  'cp850'

• On Windows Server 2019 1809, PyCharm 2022.1 builtin Powershell terminal with Anaconda env active:
  None

• On Windows Server 2019 1809, PyCharm 2022.1 builtin Python console with Anaconda env active:
  None

• On Windows Server 2019 1809, PyCharm 2022.1 running a script with Anaconda env without Terminal Emulation:
  None

• On Windows Server 2019 1809, PyCharm 2022.1 running a script with Anaconda env with Terminal Emulation:
  None

• On Windows 10 Education 21H2, Powershell prompt with default system Python 3.10:
  'cp850'

• On Windows 10 Education 21H2, PyCharm 2022.1 builtin Powershell terminal with default system Python 3.10:
  'cp850'

• On Windows 10 Education 21H2, PyCharm 2022.1 builtin Python console with default system Python 3.10:
  None

• On Windows 10 Education 21H2, PyCharm 2022.1 running a script with default system Python 3.10 without Terminal Emulation:
  None

• On Windows 10 Education 21H2, PyCharm 2022.1 running a script with default system Python 3.10 with Terminal Emulation:
  'cp850'

Summary:

Consistent results:

• The normal system Powershell always delivers 'cp850'.
• The PyCharm built-in Python console always delivers None.
• Running a script with PyCharm run config setting Terminal Emulation -> disabled always delivers None.

Inconsistent results:

• Running a script with PyCharm run config setting Terminal Emulation -> enabled
• Using the PyCharm built-in Terminal

[Jonibhoni]

There are settings in PyCharm to config the console encoding, by default it's set to use System encoding:

... But actually, regardless what I set in here, the None output of os.device_encoding(0) in the PyCharm built-in Python console is always the same; probably because the built-in Python Console is not connected to a real terminal.

In contrary, the PyCharm built-in Terminal is using the system Powershell, and thus does return 'cp850' (on up-to-date Windows 10 Edu, regardless of the Console PyCharm setting) or None (on older Windows Server 2019).

It's important to mind the terminology here: PyCharm has a built-in "Terminal" (= embedded Powershell) and a built-in "Python Console" (= ready loaded Python interpreter instance).

[Jonibhoni]

(I updated some stuff in the previous messages - in case you read the content via e-mail, read it again on GitHub, please! ^^)

[Jonibhoni]

So it seems PyCharm has some problems talking to the Windows Powershell 5.1.17763.2803 on Windows Server 2019 1809 (+ some other environment specs that may be involved, for example that I am using Windows Remote Desktop to connect to it). In effect, the usual methods (os.device_encodingor chcp) fail to forward the correct codepage when using the PyCharm Terminal or a run configuration with Terminal emulation enabled.

I guess this is a PyCharm bug, because in the same Powershell outside PyCharm, it works. I'm thinking about filing a PyCharm bug report, if you agree with my thoughts.

Anyway, there may be other editors/situation/cases like this where the codepage is not correctly reported? Maybe it's worth to make enlighten robustly work in such situations anyway, by uttering a warning or so and assuming some default encoding. (?)

[avylove]

Thanks for all that investigation! Yes, it definitely seems like a PyCharm bug and it would be great if you file a bug with them. I think updating Jinxed to handle the error makes the most sense. Working on that now, but there seems to be a long queue for GH actions.

[Jonibhoni]

I've filed a bug report for PyCharm:
https://youtrack.jetbrains.com/issue/PY-54212/chcp-and-os-device-encoding-0-return-0-None-on-some-Windows-syst

[avylove]

GitHub Actions have been stalled for the last day, but Appveyor tests and local tests pass, so I pushed a new Jinxed release. Upgrade to jinxed 1.2.0 and let me know if it works for you.

[Jonibhoni]

With Jinxed 1.2.0 it works wonderfully! 😃
Wow! Thanks for your dedication, Avy!

[avylove]

Good to hear! Thanks for reporting!