Environments needs to be activated before running code on the user's behalf (e.g. unit tests or DS) #3330

mikofski · 2018-11-13T23:32:37Z

Environment data

VS Code version: 1.29.0
Extension version (available under the Extensions sidebar): 2018.10.1
OS and version: Windows10
Python version (& distribution if applicable, e.g. Anaconda): 3.7.1 (Anaconda) and 3.6.7 (Anaconda)
Type of virtual environment used (N/A | venv | virtualenv | conda | ...): conda-4.5.11
Relevant/affected Python packages and their versions: numpy-1.15.3, pytest-3.10, nose-1.3.7

Related Issues:

VS Code does not properly activate anaconda 5.3 -> Cannot import numpy, DLL not found #3040

Actual behavior

Tests are discovered and run in Python-3.6 conda env, but not in Python-3.7, which doesn't discover tests and/or fails in conftests.py when importing numpy, fails to load DLL multiarray. But numpy does work in a terminal, when py37 is activated, and I can import multiarray from numpy.core.

Also I can even run pytest directly from the terminal inside vscode, so it's something specific to how the tests are implemented. Maybe it's not implementing the conda environment correctly?

Same issue with both pytest and nosetests.

Expected behavior

discover and run tests in Python-3.7 same as in Python-3.6

Steps to reproduce:

install anaconda
create py3.6 and py3.7 environments
install vscode, python extension
open a python project with testing (that uses numpy?), enable testing, select each environment and run tests.

Logs

Output for Python in the Output panel (View→Output, change the drop-down the upper-right of the Output panel to Python)

here's the output from pytest:

============================= test session starts =============================
platform win32 -- Python 3.7.1, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: c:\Users\mikm\Projects\pvlib-python, inifile:
collected 0 items / 1 errors

=================================== ERRORS ====================================
______________________________ ERROR collecting  ______________________________
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\config\__init__.py:429: in _importconftest
    return self._conftestpath2mod[conftestpath]
E   KeyError: local('c:\\Users\\mikm\\Projects\\pvlib-python\\pvlib\\test\\conftest.py')

During handling of the above exception, another exception occurred:
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\numpy\core\__init__.py:16: in <module>
    from . import multiarray
E   ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\config\__init__.py:435: in _importconftest
    mod = conftestpath.pyimport()
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\py\_path\local.py:668: in pyimport
    __import__(modname)
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\assertion\rewrite.py:294: in load_module
    six.exec_(co, mod.__dict__)
pvlib\test\conftest.py:3: in <module>
    import numpy as np
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\numpy\__init__.py:142: in <module>
    from . import add_newdocs
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\numpy\add_newdocs.py:13: in <module>
    from numpy.lib import add_newdoc
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\numpy\lib\__init__.py:8: in <module>
    from .type_check import *
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\numpy\lib\type_check.py:11: in <module>
    import numpy.core.numeric as _nx
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\numpy\core\__init__.py:26: in <module>
    raise ImportError(msg)
E   ImportError: 
E   Importing the multiarray numpy extension module failed.  Most
E   likely you are trying to import a failed build of numpy.
E   If you're working with a numpy git repo, try `git clean -xdf` (removes all
E   files not under version control).  Otherwise reinstall numpy.
E   
E   Original error was: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\py\_path\common.py:377: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\py\_path\common.py:429: in gen
    for p in self.gen(subdir):
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\py\_path\common.py:418: in gen
    dirs = self.optsort([p for p in entries
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\py\_path\common.py:419: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\main.py:599: in _recurse
    ihook = self.gethookproxy(dirpath)
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\main.py:417: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\config\__init__.py:413: in _getconftestmodules
    mod = self._importconftest(conftestpath)
..\..\AppData\Local\Continuum\miniconda3\envs\py37\lib\site-packages\_pytest\config\__init__.py:452: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: (local('c:\\Users\\mikm\\Projects\\pvlib-python\\pvlib\\test\\conftest.py'), (<class 'ImportError'>, ImportError("\nImporting the multiarray numpy extension module failed.  Most\nlikely you are trying to import a failed build of numpy.\nIf you're working with a numpy git repo, try `git clean -xdf` (removes all\nfiles not under version control).  Otherwise reinstall numpy.\n\nOriginal error was: DLL load failed: The specified module could not be found.\n"), <traceback object at 0x0000012E97C8C688>))
 generated xml file: C:\Users\mikm\AppData\Local\Temp\tmp-18512deYtowEyIRt8.xml 
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.53 seconds ===========================

Output from Console under the Developer Tools panel (toggle Developer Tools on under Help)

There were some errors in discovering unit tests

t.onDidNotificationChange	@	notificationsAlerts.ts:40
(anonymous)	@	notificationsAlerts.ts:26
e.fire	@	event.ts:145
t.notify	@	notifications.ts:113
t.notify	@	notificationService.ts:52
e._showMessage	@	mainThreadMessageService.ts:83
e._showMessage	@	mainThreadMessageService.ts:44
e.$showMessage	@	mainThreadMessageService.ts:38
t._doInvokeHandler	@	rpcProtocol.ts:439
t._invokeHandler	@	rpcProtocol.ts:424
t._receiveRequest	@	rpcProtocol.ts:341
t._receiveOneMessage	@	rpcProtocol.ts:271
(anonymous)	@	rpcProtocol.ts:146
e.fire	@	event.ts:145
a	@	ipc.net.ts:103
n._socketDataListener	@	ipc.net.ts:132
emitOne	@	events.js:116
emit	@	events.js:211
addChunk	@	_stream_readable.js:263
readableAddChunk	@	_stream_readable.js:250
Readable.push	@	_stream_readable.js:208
onread	@	net.js:594

The text was updated successfully, but these errors were encountered:

brettcannon · 2018-11-15T01:07:04Z

I can discover and run tests fine under conda using Python 3.7 and pytest without numpy, but I do get the same failure if the environment is not activated. It seems conda has chosen to package numpy in such a way that it can't be imported unless the environment is activated (which we don't do when running tests).

With a test of:

import numpy

def test_fake():
    return True

and settings of:

"python.unitTest.pyTestArgs": [
        "."
    ],
    "python.unitTest.unittestEnabled": false,
    "python.unitTest.nosetestsEnabled": false,
    "python.unitTest.pyTestEnabled": true

I end up with:

============================= test session starts =============================
platform win32 -- Python 3.7.0, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
rootdir: c:\Users\brcan\Desktop\Testing\Some 3.7 Bunk, inifile:
collected 0 items / 1 errors

=================================== ERRORS ====================================
______________________ ERROR collecting example_test.py _______________________
ImportError while importing test module 'c:\Users\brcan\Desktop\Testing\Some 3.7 Bunk\example_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\core\__init__.py:16: in <module>
    from . import multiarray
E   ImportError: DLL load failed: The specified module could not be found.

During handling of the above exception, another exception occurred:
example_test.py:1: in <module>
    import numpy
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\__init__.py:142: in <module>
    from . import add_newdocs
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\add_newdocs.py:13: in <module>
    from numpy.lib import add_newdoc
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\lib\__init__.py:8: in <module>
    from .type_check import *
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\lib\type_check.py:11: in <module>
    import numpy.core.numeric as _nx
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\core\__init__.py:26: in <module>
    raise ImportError(msg)
E   ImportError: 
E   Importing the multiarray numpy extension module failed.  Most
E   likely you are trying to import a failed build of numpy.
E   If you're working with a numpy git repo, try `git clean -xdf` (removes all
E   files not under version control).  Otherwise reinstall numpy.
E   
E   Original error was: DLL load failed: The specified module could not be found.
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!
=========================== 1 error in 0.11 seconds ===========================
__________There was an error in identifying unit tests in example_test.py__________
______________________ ERROR collecting example_test.py _______________________
ImportError while importing test module 'c:\Users\brcan\Desktop\Testing\Some 3.7 Bunk\example_test.py'.
Hint: make sure your test modules/packages have valid Python names.
Traceback:
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\core\__init__.py:16: in <module>
    from . import multiarray
E   ImportError: DLL load failed: The specified module could not be found.
During handling of the above exception, another exception occurred:
example_test.py:1: in <module>
    import numpy
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\__init__.py:142: in <module>
    from . import add_newdocs
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\add_newdocs.py:13: in <module>
    from numpy.lib import add_newdoc
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\lib\__init__.py:8: in <module>
    from .type_check import *
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\lib\type_check.py:11: in <module>
    import numpy.core.numeric as _nx
..\..\..\AppData\Local\Continuum\miniconda3\envs\activatetest\lib\site-packages\numpy\core\__init__.py:26: in <module>
    raise ImportError(msg)
E   ImportError: 
E   Importing the multiarray numpy extension module failed.  Most
E   likely you are trying to import a failed build of numpy.
E   If you're working with a numpy git repo, try `git clean -xdf` (removes all
E   files not under version control).  Otherwise reinstall numpy.
E   
E   Original error was: DLL load failed: The specified module could not be found.
!!!!!!!!!!!!!!!!!!! Interrupted: 1 errors during collection !!!!!!!!!!!!!!!!!!!

brettcannon · 2018-11-15T01:21:16Z

I also forgot to mention that I verified this in an Anaconda Prompt by running directly the Python interpreter and then indirectly through an activated terminal and got the same results, so it's definitely based on how they have chosen to package numpy for at least Python 3.7.

We might have to use child_process.exec() and compose the activation + command together to make this work which is not going to be trivial to do as we would have to re-architect how to execute Python just for this one case.

gvellut · 2018-11-21T15:42:14Z

I had the same issue. Until it is fixed, I created a batch file :

@echo off
C:\\Users\\gvellut\\Anaconda3\\Scripts\\activate.bat wbt2 && pytest %*

And configured the "python.unitTest.pyTestPath" option to launch it (instead of pytest directly).

brettcannon · 2018-11-21T19:18:18Z

Just so people know, we're trying to schedule a meeting with Anaconda to figure out how best to handle this. Once we have that meeting we will be in a better place to know how to solve this going forward.

qubitron · 2019-01-02T22:28:38Z

We'll need to check the performance before committing this change.

DonJayamanne · 2019-01-07T22:22:53Z

Findings, running jupyter in pipenv also requires the environment to be activated.
I.e. activation of environments is not limited just to Conda, we'll need to do this for other environments too.

Proposed Solution:

Create a method namd createActivatedEnvironment in IPythonExecutionFactory
This new method will asynchronously return an implementation of IPythonExecutionService, which will in turn ensure the python environment is activated before executing any python code.

brettcannon · 2019-01-07T23:23:23Z

@DonJayamanne How does the failure manifest itself? I would be curious to know why it's occurring (if it's because of virtualenv then I can understand that).

ericsnowcurrently · 2019-01-29T19:17:47Z

I validated this.

mikofski · 2019-01-29T19:56:57Z

yay! thanks so much everyone!

abarajithan11 · 2019-02-21T19:51:57Z

Is this issue solved now? I have the latest version of Anaconda and VS code and I still have this problem.

brettcannon · 2019-02-26T23:47:21Z

@abarajithan11 yes, it's resolved. If you're having issues then please open a new issue.

brettcannon self-assigned this Nov 15, 2018

brettcannon added triage bug Issue identified by VS Code Team member as probable bug area-testing and removed triage labels Nov 15, 2018

brettcannon removed their assignment Nov 15, 2018

brettcannon mentioned this issue Nov 15, 2018

VS Code does not properly activate anaconda 5.3 -> Cannot import numpy, DLL not found #3040

Closed

This comment has been minimized.

Sign in to view

brettcannon changed the title ~~test frameworks fails with python 3.7 from anaconda~~ when using a conda environment, the environment is not activated when running tests Nov 21, 2018

brettcannon added the important Issue identified as high-priority label Jan 2, 2019

qubitron changed the title ~~when using a conda environment, the environment is not activated when running tests~~ Conda needs to be activated starting with Python 3.7 Jan 2, 2019

qubitron changed the title ~~Conda needs to be activated starting with Python 3.7~~ Conda needs to be activated before running code on the user's behalf Jan 2, 2019

DonJayamanne added this to the 2019, week 1 - Jan Sprint 1 milestone Jan 2, 2019

DonJayamanne self-assigned this Jan 7, 2019

This was referenced Jan 8, 2019

Run python code using env variables from activated envs #3930

Merged

Interpreter list reporting global interpreters twice & reporting wrong bugfix version #3953

Closed

DonJayamanne changed the title ~~Conda needs to be activated before running code on the user's behalf~~ Environments needs to be activated before running code on the user's behalf (e.g. unit tests or DS) Jan 10, 2019

DonJayamanne mentioned this issue Jan 10, 2019

Activate environment before running tests #2420

Closed

DonJayamanne added the validate fix label Jan 16, 2019

DonJayamanne modified the milestones: 2019, week 1 - Jan Sprint 1, 2019, week 3 - Jan Sprint 2 Jan 16, 2019

DonJayamanne assigned ericsnowcurrently and unassigned DonJayamanne Jan 29, 2019

DonJayamanne closed this as completed Jan 29, 2019

ghost removed needs PR labels Jan 29, 2019

mbunse mentioned this issue Feb 6, 2019

Activate environment before debugging tests #4300

Closed

microsoft locked as resolved and limited conversation to collaborators Feb 26, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Environments needs to be activated before running code on the user's behalf (e.g. unit tests or DS) #3330

Environments needs to be activated before running code on the user's behalf (e.g. unit tests or DS) #3330

mikofski commented Nov 13, 2018 •

edited

Loading

brettcannon commented Nov 15, 2018

brettcannon commented Nov 15, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

gvellut commented Nov 21, 2018

brettcannon commented Nov 21, 2018

qubitron commented Jan 2, 2019

DonJayamanne commented Jan 7, 2019

brettcannon commented Jan 7, 2019

ericsnowcurrently commented Jan 29, 2019

mikofski commented Jan 29, 2019

abarajithan11 commented Feb 21, 2019

brettcannon commented Feb 26, 2019

Environments needs to be activated before running code on the user's behalf (e.g. unit tests or DS) #3330

Environments needs to be activated before running code on the user's behalf (e.g. unit tests or DS) #3330

Comments

mikofski commented Nov 13, 2018 • edited Loading

Environment data

Related Issues:

Actual behavior

Expected behavior

Steps to reproduce:

Logs

brettcannon commented Nov 15, 2018

brettcannon commented Nov 15, 2018

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

gvellut commented Nov 21, 2018

brettcannon commented Nov 21, 2018

qubitron commented Jan 2, 2019

DonJayamanne commented Jan 7, 2019

brettcannon commented Jan 7, 2019

ericsnowcurrently commented Jan 29, 2019

mikofski commented Jan 29, 2019

abarajithan11 commented Feb 21, 2019

brettcannon commented Feb 26, 2019

mikofski commented Nov 13, 2018 •

edited

Loading