yarn install won't install the sub-dependencies of private git dependencies

[aberigle]

Do you want to request a feature or report a bug?
Report a bug

What is the current behavior?
We have a couple of packages we install from gitlab in our package.json in the following format:

"package" : "git+ssh://gitlab.com/user/package"

Yarn manages to download and link this package but it's not downloading any of its dependencies.

If the current behavior is a bug, please provide the steps to reproduce.

What is the expected behavior?
My package's dependencies are installed

[rally25rs]

Could you make a minimal sample project? I use GitHub references a lot and I've never run into this issue. Not sure if it's specific to GitLab or also happens with GitHub?

[galvarez421]

I am experiencing this as well for a private Git repo.

I have ESLint listed as a dependency in my project's package.json as well as in the package.json for the Git repo dependency. After running yarn, I get the error described in #6285, namely:

error An unexpected error occurred: "could not find a copy of eslint to link in
C:\\myproject\\node_modules\\mydependency\\node_modules".

and a yarn-lock file is not generated. The node_modules folder then contains an eslint folder that contains the version listed in my package.json, while the 'mydependency' folder has an empty node_modules folder, withouth a separate copy of the version of eslint listed in its package.json.

'myproject' package.json:

"dependencies": {
  "eslint": "5.5.0",
  "mydependency": "https://privaterepo.com/mydependency.git#v1.0.0",
}

'mydependency' package.json:

"dependencies": {
  "eslint": "4.19.1",
  "eslint-config-google": "0.9.1"
},
"peerDependencies": {
  "eslint": ">= 4.19.1"
}

I should note that I am experiencing this issue only recently after trying to run a fresh yarn install for a project after having upgraded both yarn and Node.js, so I'm not sure if the updated versions of those tools might be a factor.

Perhaps the fact that eslint is also listed as a peer dependency is preventing the proper version of the dependency's dependency from being installed?

[chrismllr]

@rally25rs I am seeing the same behavior after moving our package registry from Artifactory to Gitlab.

Upon install, the dependencies listed below each package in the yarn.lock do not have top level entries. Switching back to/ using the Artifactory registry fixes the issue.

Trying to debug, but this being Gitlab related could be true.

[chrismllr]

@rally25rs Could this be the issue on their side? https://gitlab.com/gitlab-org/gitlab/issues/9415

It seems if i put the entry in my package.json and simply run yarn:

// package.json
{
  "dependencies": {
    "@myscope/my-package": "4.0.0-alpha.1"
  }
}

The dependencies of my-package are not installed.

But, if I run yarn add @myscope/my-package@4.0.0-alpha.1, it WILL install the sub dependencies.

Very curious

[rally25rs]

@chrismllr if GitLab doesn't include a latest tag then that would likely be an issue, and would explain why installing a specific version (my-package@4.0.0-alpha.1) works. It's been a while since I looked, but IIRC when no version is specified, yarn looks for the latest tag.

However it looks like GitLab should have fixed that months ago.

I don't have a GitLab repo setup that I can try, but you should be able to just see what metadata it returns for your package with a normal GET request from a web browser or curl.

[chrismllr]

@rally25rs Interesting, what would the URL look like for that GET request? I'll check it out. Thanks!

[rally25rs]

Actually, the code here

https://github.com/yarnpkg/yarn/blob/master/src/resolvers/registries/npm-resolver.js#L51-L60

and

https://github.com/yarnpkg/yarn/blob/master/src/package-constraint-resolver.js#L21-L27

it looks like if dist-tags latest doesn't exist then it will just install the newest version according to semver. 🤔

Although, none of that really explains why sub-deps wouldn't be installed, unless those aren't listed in the metadata either

[rally25rs]

@chrismllr the registry URL for a normal package like left-pad from npm's normal registry is http://registry.npmjs.org/left-pad and with a scope it's http://registry.npmjs.org/@babel/core
So bascially just https://registry/@scope/package

You would then expect to see versions.{version}.dependencies for example:

and I'm pretty sure that is what yarn uses to install the transitive/sub dependencies

[chrismllr]

Ah! We're getting somewhere. The dist-tags DOES exist, with the correct latest key.

Although, the registry did not add the dependencies hash inside the version hash.

Looks like:

{
  "name": "@scope/package",
  "dist-tags":{
    "latest":"5.0.0-alpha.3"
  },
  "versions": {
    "1.0.0": {
      "name": "@scope/package",
      "version": "1.0.0",
      "dist": {
        "shasum": "6b587df883b13939fcfb66a13fb934a3985b248e",
        "tarball": "https://registry.url.com/packages/npm/@scope/package/-/@scope/package-1.0.0.tgz"
      }
      ... should be `dependencies`
    }
    ... other versions
  }
}

Wheras Artifactory includes more metadata from package.json, including dependencies and devDependencies.

I could see this likely being a problem 😆 Although I don't know how it is working correctly when installing the package alone using yarn add. Perhaps for a single package install, yarn is falling back to interrogating the package.json for that downloaded package 🤷‍♂

I will open a support ticket with Gitlab and get back to you when I have some more details.

[chrismllr]

Update: Gitlab merged a fix for this within the last week, and it is now live on gitlab.com. We are on an enterprise account and have not received the update yet. New structure from gitlab packages:

{
  "name": "@scope/package",
  "dist-tags": {
    "latest": "1.0.5"
  },
  "versions": {
    "1.0.5": {
      "name": "@scope/package",
      "version": "1.0.5",
      "dist": {
        "shasum": "4cbde301828bd1f3e",
        "tarball": "https://gitlab.com/api/v4/projects/xx/packages/npm/@scope/package/-/@scope/package-1.0.5.tgz"
      },
      "dependencies": {
        "sub-dep-1": "^2.0.0",
        "sub-dep-2": "^15.0.2"
      }
    }
  }
}

I am not sure if this solves the problem for the original reporter, but its something Gitlab has apparently taken care of recently.

[rally25rs]

The addition of dependencies should hopefully fix the original issue.

Perhaps for a single package install, yarn is falling back to interrogating the package.json for that downloaded package

One thing I remembered while investigating another issue; If a lockfile already exists and the specific locked version is in yarn cache, then I believe yarn will use the package.json out of the cache, which would explain why it sometimes works.

I'm going to mark this issue closed in light of this gitlab change. If the problem still occurs, we can reopen. Thanks for checking the gitlab metadata responses @chrismllr that really helps. 👍

[mgara]

Hello,
I'm trying to run my app which has a dependency that has a dependency on bitbucket
When I do yarn everything goes as expected, but when trying to run the application it says that the dependency of the dependency is not available and yet I verified that and inside the node_modules for the module there is such dependency.

This dependency was not found:

* @company/foo in ./node_modules/baz/toz/bar.js

To install it, you can run: npm install --save @company/foo

Any hints ?

• My packages @company/foo is on our private bitbucket git ... its a pointing to a git tag
• My dependency baz is a. dependency on our bitbucket git too.