Skip to content
This repository has been archived by the owner on Jun 18, 2019. It is now read-only.

Unable to install python_packages ImportError #135

Closed
Idun0 opened this issue Oct 16, 2018 · 2 comments
Closed

Unable to install python_packages ImportError #135

Idun0 opened this issue Oct 16, 2018 · 2 comments

Comments

@Idun0
Copy link

Idun0 commented Oct 16, 2018

First time so be gentle :D The reason I find myself here is cause im trying to create a airflow build which is dependent upon poise-python since python has been deprecated. At first I was getting an error for setuptools which I have now remedied with override["poise-python"]["options"]["pip_version"] = "18.0" But now I am getting several errors:

Heres my kitchen converge output:
`-----> Starting Kitchen (v1.16.0)
-----> Converging ...
Preparing files for transfer
Preparing dna.json
Resolving cookbook dependencies with Berkshelf 6.3.1...
Removing non-cookbook files before transfer
Preparing validation.pem
Preparing client.rb
-----> Chef Omnibus installation detected (12.19)
Transferring files to
[2018-10-16T21:47:42+00:00] INFO: Forking chef instance to converge...
Starting Chef Client, version 12.19.36
[2018-10-16T21:47:42+00:00] INFO: *** Chef 12.19.36 ***
[2018-10-16T21:47:42+00:00] INFO: Platform: x86_64-linux
[2018-10-16T21:47:42+00:00] INFO: Chef-client pid: 11179
[2018-10-16T21:47:43+00:00] INFO: Setting the run_list to ["recipe[airflow::default]"] from CLI options
[2018-10-16T21:47:43+00:00] INFO: Run List is [recipe[airflow::default]]
[2018-10-16T21:47:43+00:00] INFO: Run List expands to [airflow::default]
[2018-10-16T21:47:43+00:00] INFO: Starting Chef Run for airflow-ubuntu-1404
[2018-10-16T21:47:43+00:00] INFO: Running start handlers
[2018-10-16T21:47:43+00:00] INFO: Start handlers complete.
[2018-10-16T21:47:43+00:00] INFO: HTTP Request Returned 404 Not Found: Object not found:
resolving cookbooks for run list: ["airflow::default"]
[2018-10-16T21:47:43+00:00] INFO: Loading cookbooks [airflow@1.2.2, apt@4.0.1, poise-python@1.2.1, compat_resource@12.19.1, poise@2.8.2, poise-languages@1.4.0, poise-archive@1.5.0]
Synchronizing Cookbooks:
- poise-python (1.2.1)
- airflow (1.2.2)
- poise (2.8.2)
- poise-archive (1.5.0)
- apt (4.0.1)
- compat_resource (12.19.1)
- poise-languages (1.4.0)
Installing Cookbook Gems:
Compiling Cookbooks...
[2018-10-16T21:47:44+00:00] WARN: Chef::Provider::AptRepository already exists! Cannot create deprecation class for LWRP provider apt_repository from cookbook apt
[2018-10-16T21:47:44+00:00] WARN: AptRepository already exists! Deprecation class overwrites Custom resource apt_repository from cookbook apt
Converging 85 resources
Recipe: apt::default
* file[/var/lib/apt/periodic/update-success-stamp] action nothing[2018-10-16T21:47:44+00:00] INFO: Processing file[/var/lib/apt/periodic/update-success-stamp] action nothing (apt::default line 26)
(skipped due to action :nothing)
* apt_update[periodic] action periodic[2018-10-16T21:47:44+00:00] INFO: Processing apt_update[periodic] action periodic (apt::default line 37)
(up to date)
* execute[apt-get update] action nothing[2018-10-16T21:47:44+00:00] INFO: Processing execute[apt-get update] action nothing (apt::default line 42)
(skipped due to action :nothing)
* execute[apt-get autoremove] action nothing[2018-10-16T21:47:44+00:00] INFO: Processing execute[apt-get autoremove] action nothing (apt::default line 51)
(skipped due to action :nothing)
* execute[apt-get autoclean] action nothing[2018-10-16T21:47:44+00:00] INFO: Processing execute[apt-get autoclean] action nothing (apt::default line 61)
(skipped due to action :nothing)
* directory[/var/cache/local] action create[2018-10-16T21:47:44+00:00] INFO: Processing directory[/var/cache/local] action create (apt::default line 68)
(up to date)
* directory[/var/cache/local/preseeding] action create[2018-10-16T21:47:44+00:00] INFO: Processing directory[/var/cache/local/preseeding] action create (apt::default line 68)
(up to date)
* template[/etc/apt/apt.conf.d/10recommends] action create[2018-10-16T21:47:44+00:00] INFO: Processing template[/etc/apt/apt.conf.d/10recommends] action create (apt::default line 77)
(up to date)
* apt_package[apt-transport-https] action install[2018-10-16T21:47:44+00:00] INFO: Processing apt_package[apt-transport-https] action install (apt::default line 85)
(up to date)
Recipe: airflow::user
* group[airflow] action create[2018-10-16T21:47:44+00:00] INFO: Processing group[airflow] action create (airflow::user line 15)
(up to date)
* linux_user[airflow] action create[2018-10-16T21:47:44+00:00] INFO: Processing linux_user[airflow] action create (airflow::user line 19)
(up to date)
Recipe: airflow::directories
* directory[/usr/local/lib/airflow] action create[2018-10-16T21:47:44+00:00] INFO: Processing directory[/usr/local/lib/airflow] action create (airflow::directories line 15)
(up to date)
* directory[/usr/local/lib/airflow/dags] action create[2018-10-16T21:47:44+00:00] INFO: Processing directory[/usr/local/lib/airflow/dags] action create (airflow::directories line 22)
(up to date)
* directory[/usr/local/lib/airflow/plugins] action create[2018-10-16T21:47:44+00:00] INFO: Processing directory[/usr/local/lib/airflow/plugins] action create (airflow::directories line 29)
(up to date)
* directory[/var/run/airflow] action create[2018-10-16T21:47:44+00:00] INFO: Processing directory[/var/run/airflow] action create (airflow::directories line 36)
(up to date)
Recipe: airflow::packages
* python_runtime[2] action install[2018-10-16T21:47:44+00:00] INFO: Processing python_runtime[2] action install (airflow::packages line 15)

       * poise_languages_system[python2.7] action install[2018-10-16T21:47:44+00:00] INFO: Processing poise_languages_system[python2.7] action install (/tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/system/mixin.rb line 32)

   (up to date)
        (up to date)
     * python_runtime_pip[2] action install[2018-10-16T21:47:44+00:00] INFO: Processing python_runtime_pip[2] action install (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 116)

       * python_package[pip] action upgrade[2018-10-16T21:47:44+00:00] INFO: Processing python_package[pip] action upgrade (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb line 145)


         ================================================================================
         Error executing action `upgrade` on resource 'python_package[pip]'
         ================================================================================

         Mixlib::ShellOut::ShellCommandFailed
         ------------------------------------
         Expected process to exit with [0], but received '1'
         ---- Begin output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
         STDOUT:
         STDERR: Traceback (most recent call last):
    File "<stdin>", line 4, in <module>
         ImportError: No module named commands
         ---- End output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
         Ran ["/usr/bin/python2.7", "-", "pip==18.0"] returned 1

         Cookbook Trace:
         ---------------
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:201:in `tap'
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:201:in `language_command_shell_out!'
         /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:216:in `block in language_command_mixin'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:263:in `pip_command'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:287:in `pip_outdated'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:164:in `check_package_versions'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:144:in `load_current_resource'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:143:in `install_pip'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:77:in `action_install'
         /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
         /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

         Resource Declaration:
         ---------------------
         # In /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb

         145:             python_package 'pip' do
         146:               action :upgrade
         147:               parent_python new_resource.parent
         148:               version new_resource.version if new_resource.version
         149:             end
         150:           end

         Compiled Resource:
         ------------------
         # Declared in /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:145:in `block in install_pip'

         python_package("pip") do
    package_name "pip"
    action [:upgrade]
    retries 0
    retry_delay 2
    default_guard_interpreter :default
    declared_type :python_package
    cookbook_name "airflow"
    parent_python python_runtime[2]
    version "18.0"
    timeout 900
         end

         Platform:
         ---------
         x86_64-linux

   [2018-10-16T21:47:45+00:00] INFO: Running queued delayed notifications before re-raising exception

       ================================================================================
       Error executing action `install` on resource 'python_runtime_pip[2]'
       ================================================================================

       Mixlib::ShellOut::ShellCommandFailed
       ------------------------------------
       python_package[pip] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb line 145) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
       ---- Begin output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
       STDOUT:
       STDERR: Traceback (most recent call last):
         File "<stdin>", line 4, in <module>
       ImportError: No module named commands
       ---- End output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
       Ran ["/usr/bin/python2.7", "-", "pip==18.0"] returned 1

       Cookbook Trace:
       ---------------
       /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:201:in `tap'
       /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:201:in `language_command_shell_out!'
       /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:216:in `block in language_command_mixin'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:263:in `pip_command'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:287:in `pip_outdated'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:164:in `check_package_versions'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:144:in `load_current_resource'
       /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:143:in `install_pip'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:77:in `action_install'
       /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
       /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

       Resource Declaration:
       ---------------------
       # In /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb

       116:         python_runtime_pip new_resource.name do
       117:           parent new_resource
       118:           # If the version is `true`, don't pass it at all.
       119:           version pip_version if pip_version.is_a?(String)
       120:           get_pip_url pip_url if pip_url
       121:         end
       122:       end
       123:

       Compiled Resource:
       ------------------
       # Declared in /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:116:in `install_pip'

       python_runtime_pip("2") do
         action [:install]
         retries 0
         retry_delay 2
         default_guard_interpreter :default
         declared_type :python_runtime_pip
         cookbook_name "airflow"
         parent python_runtime[2]
         version "18.0"
       end

       Platform:
       ---------
       x86_64-linux

   [2018-10-16T21:47:45+00:00] INFO: Running queued delayed notifications before re-raising exception

     ================================================================================
     Error executing action `install` on resource 'python_runtime[2]'
     ================================================================================

     Mixlib::ShellOut::ShellCommandFailed
     ------------------------------------
     python_runtime_pip[2] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 116) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[pip] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb line 145) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
     ---- Begin output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
     STDOUT:
     STDERR: Traceback (most recent call last):
       File "<stdin>", line 4, in <module>
     ImportError: No module named commands
     ---- End output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
     Ran ["/usr/bin/python2.7", "-", "pip==18.0"] returned 1

     Cookbook Trace:
     ---------------
     /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:201:in `tap'
     /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:201:in `language_command_shell_out!'
     /tmp/kitchen/cache/cookbooks/poise-languages/files/halite_gem/poise_languages/command/mixin.rb:216:in `block in language_command_mixin'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:263:in `pip_command'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:287:in `pip_outdated'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:164:in `check_package_versions'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_package.rb:144:in `load_current_resource'
     /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:143:in `install_pip'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb:77:in `action_install'
     /tmp/kitchen/cache/cookbooks/poise/files/halite_gem/poise/helpers/notifying_block.rb:69:in `notifying_block'
     /tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb:49:in `action_install'

     Resource Declaration:
     ---------------------
     # In /tmp/kitchen/cache/cookbooks/airflow/recipes/packages.rb

      15: python_runtime node["airflow"]["python_runtime"] do
      16:   version node["airflow"]["python_version"]
      17:   provider :system
      18:   pip_version node["airflow"]["pip_version"]
      19: end
      20:

     Compiled Resource:
     ------------------
     # Declared in /tmp/kitchen/cache/cookbooks/airflow/recipes/packages.rb:15:in `from_file'

     python_runtime("2") do
       provider PoisePython::PythonProviders::System
       action [:install]
       retries 0
       retry_delay 2
       default_guard_interpreter :default
       subresources [python_package[apache-airflow], python_package[greenlet], python_package[eventlet], python_package[celery], python_package[cryptography], python_package[sphinx], python_package[sphinx-argparse], python_package[sphinx-rtd-theme], python_package[Sphinx-PyPI-upload], python_package[docker-py], python_package[druid], python_package[boto3], python_package[httplib2], python_package[google-api-python-client], python_package[oauth2client], python_package[PyOpenSSL], python_package[snakebite], python_package[hdfs[dataframe,avro,kerberos]], python_package[hive-thrift-py], python_package[pyhive], python_package[impyla], python_package[unicodecsv], python_package[jaydebeapi], python_package[pymssql], python_package[mysqlclient], python_package[librabbitmq], python_package[psycopg2], python_package[boto], python_package[filechunkio], python_package[pysmbclient], python_package[slackclient], python_package[statsd], python_package[vertica-python], python_package[ldap3], python_package[pykerberos], python_package[thrift_sasl], python_package[snakebite[kerberos]], python_package[bcrypt], python_package[flask-bcrypt], python_package[Flask-OAuthlib], python_package[qds-sdk], python_package[cloudant], python_package[lxml], python_package[nose], python_package[nose-parameterized], python_package[mock], python_package[click], python_package[jira], python_package[moto], python_package[freezegun], python_runtime_pip[2], python_package[setuptools], python_package[wheel], python_package[virtualenv], python_package[pip]]
       declared_type :python_runtime
       cookbook_name "airflow"
       recipe_name "packages"
       version "2.7"
       pip_version true
       setuptools_version true
       virtualenv_version true
       wheel_version true
     end

     Platform:
     ---------
     x86_64-linux

   [2018-10-16T21:47:45+00:00] INFO: Running queued delayed notifications before re-raising exception

   Running handlers:
   [2018-10-16T21:47:45+00:00] ERROR: Running exception handlers
   Running handlers complete
   [2018-10-16T21:47:45+00:00] ERROR: Exception handlers complete
   Chef Client failed. 0 resources updated in 03 seconds
   [2018-10-16T21:47:45+00:00] FATAL: Stacktrace dumped to /tmp/kitchen/cache/chef-stacktrace.out
   [2018-10-16T21:47:45+00:00] FATAL: Please provide the contents of the stacktrace.out file if you file a bug report
   [2018-10-16T21:47:45+00:00] ERROR: python_runtime[2] (airflow::packages line 15) had an error: Mixlib::ShellOut::ShellCommandFailed: python_runtime_pip[2] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/python_providers/base.rb line 116) had an error: Mixlib::ShellOut::ShellCommandFailed: python_package[pip] (/tmp/kitchen/cache/cookbooks/poise-python/files/halite_gem/poise_python/resources/python_runtime_pip.rb line 145) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
   ---- Begin output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
   STDOUT:
   STDERR: Traceback (most recent call last):
     File "<stdin>", line 4, in <module>
   ImportError: No module named commands
   ---- End output of ["/usr/bin/python2.7", "-", "pip==18.0"] ----
   Ran ["/usr/bin/python2.7", "-", "pip==18.0"] returned 1
   [2018-10-16T21:47:45+00:00] FATAL: Chef::Exceptions::ChildConvergeError: Chef run process exited unsuccessfully (exit code 1)`
@coderanger
Copy link
Member

Duplicate of #134. You previously had 18.1 installed which breaks the system.

@Idun0
Copy link
Author

Idun0 commented Oct 17, 2018

@coderanger Thats the part that is confusing me i guess, cause i set the override and destroyed and recreated my kitchen instance but still get an error, are you seeing in my converge where it is being installed as 18.1? cause if it is i need to stop that from happening

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@coderanger @Idun0