gatsby-plugin-utils: Missing fs-extra as a dependency.

[fertolg]

Preliminary Checks

• This issue is not a duplicate. Before opening a new issue, please search existing issues: https://github.com/gatsbyjs/gatsby/issues
• This issue is not a question, feature request, RFC, or anything other than a bug report directly related to Gatsby. Please post those things in GitHub Discussions: https://github.com/gatsbyjs/gatsby/discussions

Description

gatsby-plugin-utils depends on the fs-extra module, specifically in this file:

gatsby/packages/gatsby-plugin-utils/src/polyfill-remote-file/placeholder-handler.ts

Line 2 in 29b236b

import { createReadStream, readFile, mkdtemp } from "fs-extra"

However, fs-extra is not listed as a dependency on gatsby-plugin-utils package.json. This normally is not a problem because fs-extra is depended by other gastby modules and it is picked up as a hoisted dependency. However, when building for production (in a CD system for example), it's possible the dependency will not be there and the build will fail.

I originally filed a bug with the npm team, but they determined it's a missing dependency for gastby-plugin-utils. See npm/cli#4178 for more reference.

Reproduction Link

npm/cli#4178

Steps to Reproduce

1. Create this package.json in a new directory:

{
  "name": "missing-fs-extra",
  "private": true,
  "description": "A simple package to reproduce missing dependencies when installing with the --production flag",
  "version": "0.1.0",
  "dependencies": {
    "gatsby": "^4.4.0"
  },
  "devDependencies": {
    "@storybook/react": "^6.4.9"
  },
  "license": "MIT",
  "scripts": {
    "build": "gatsby build"
  }
}

2. Then attempt to run

npm install --production && npm run build

Expected Result

Gatsby builds.

Actual Result

Gatsby build fails with a missing dependency of fs-extra.

Here's my full output:

➜  test npm install --production && npm run build
npm WARN deprecated async-cache@1.1.0: No longer maintained. Use [lru-cache](http://npm.im/lru-cache) version 7.6 or higher, and provide an asynchronous `fetchMethod` option.
npm WARN deprecated source-map-url@0.4.1: See https://github.com/lydell/source-map-url#deprecated
npm WARN deprecated urix@0.1.0: Please see https://github.com/lydell/urix#deprecated
npm WARN deprecated resolve-url@0.2.1: https://github.com/lydell/resolve-url#deprecated
npm WARN deprecated source-map-resolve@0.5.3: See https://github.com/lydell/source-map-resolve#deprecated
npm WARN deprecated querystring@0.2.1: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN deprecated chokidar@2.1.8: Chokidar 2 does not receive security updates since 2019. Upgrade to chokidar 3 with 15x fewer dependencies
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated babel-eslint@10.1.0: babel-eslint is now @babel/eslint-parser. This package will no longer receive updates.
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated subscriptions-transport-ws@0.9.19: The `subscriptions-transport-ws` package is no longer maintained. We recommend you use `graphql-ws` instead. For help migrating Apollo software to `graphql-ws`, see https://www.apollographql.com/docs/apollo-server/data/subscriptions/#switching-from-subscriptions-transport-ws    For general help using `graphql-ws`, see https://github.com/enisdenjo/graphql-ws/blob/master/README.md

added 1773 packages, and audited 1774 packages in 45s

211 packages are looking for funding
  run `npm fund` for details

24 vulnerabilities (6 moderate, 18 high)

To address issues that do not require attention, run:
  npm audit fix

To address all issues (including breaking changes), run:
  npm audit fix --force

Run `npm audit` for details.

> missing-fs-extra@0.1.0 build
> gatsby build


 ERROR 

gatsby build

Build a Gatsby project.

Options:
  --verbose                   Turn on verbose output             [boolean] [default: false]
  --no-color, --no-colors     Turn off the color in output       [boolean] [default: false]
  --json                      Turn on the JSON logger            [boolean] [default: false]
  --prefix-paths              Build site with link paths prefixed with the pathPrefix value
                              in gatsby-config.js. Default is env.PREFIX_PATHS or false.
                                                                 [boolean] [default: false]
  --no-uglify                 Build site without uglifying JS bundles (for debugging).
                                                                 [boolean] [default: false]
  --profile                   Build site with react profiling (this can add some additional
                              overhead). See https://reactjs.org/docs/profiler
                                                                 [boolean] [default: false]
  --graphql-tracing           Trace every graphql resolver, may have performance
                              implications                       [boolean] [default: false]
  --open-tracing-config-file  Tracer configuration file (OpenTracing compatible). See
                              https://gatsby.dev/tracing                           [string]
  -h, --help                  Show help                                           [boolean]
  -v, --version               Show the version of the Gatsby CLI and the Gatsby package in
                              the current project                                 [boolean]


 ERROR 

There was a problem loading the local build command. Gatsby may not be installed. Perhaps
you need to run "npm install"? Cannot find module 'fs-extra'
Require stack:
- /Users/fernandroid/Desktop/test/node_modules/gatsby-plugin-utils/dist/polyfill-remote-fil
e/placeholder-handler.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby-plugin-utils/dist/polyfill-remote-fil
e/graphql/gatsby-image-resolver.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby-plugin-utils/dist/polyfill-remote-fil
e/index.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/utils/start-server.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/services/start-webpack-server.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/services/index.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/bootstrap/index.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/commands/build.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby-cli/lib/create-cli.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby-cli/lib/index.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/bin/gatsby.js
- /Users/fernandroid/Desktop/test/node_modules/gatsby/cli.js



  Error: Cannot find module 'fs-extra'
  Require stack:
  - /Users/fernandroid/Desktop/test/node_modules/gatsby-plugin-utils/dist/polyfill-remote-f
  ile/placeholder-handler.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby-plugin-utils/dist/polyfill-remote-f
  ile/graphql/gatsby-image-resolver.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby-plugin-utils/dist/polyfill-remote-f
  ile/index.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/utils/start-server.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/services/start-webpack-server.
  js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/services/index.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/bootstrap/index.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/commands/build.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby-cli/lib/create-cli.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby-cli/lib/index.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/dist/bin/gatsby.js
  - /Users/fernandroid/Desktop/test/node_modules/gatsby/cli.js
  
  - loader:933 Function.Module._resolveFilename
    node:internal/modules/cjs/loader:933:15
  
  - loader:778 Function.Module._load
    node:internal/modules/cjs/loader:778:27
  
  - loader:999 Module.require
    node:internal/modules/cjs/loader:999:19
  
  - helpers:102 require
    node:internal/modules/cjs/helpers:102:18
  
  - placeholder-handler.js:11 Object.<anonymous>
    [test]/[gatsby-plugin-utils]/dist/polyfill-remote-file/placeholder-handler.js:11:16
  
  - loader:1097 Module._compile
    node:internal/modules/cjs/loader:1097:14
  
  - loader:1149 Object.Module._extensions..js
    node:internal/modules/cjs/loader:1149:10
  
  - loader:975 Module.load
    node:internal/modules/cjs/loader:975:32
  
  - loader:822 Function.Module._load
    node:internal/modules/cjs/loader:822:12
  
  - loader:999 Module.require
    node:internal/modules/cjs/loader:999:19
  
  - helpers:102 require
    node:internal/modules/cjs/helpers:102:18
  
  - gatsby-image-resolver.js:19 Object.<anonymous>
    [test]/[gatsby-plugin-utils]/dist/polyfill-remote-file/graphql/gatsby-image-resolver.js
    :19:27
  
  - loader:1097 Module._compile
    node:internal/modules/cjs/loader:1097:14
  
  - loader:1149 Object.Module._extensions..js
    node:internal/modules/cjs/loader:1149:10
  
  - loader:975 Module.load
    node:internal/modules/cjs/loader:975:32
  
  - loader:822 Function.Module._load
    node:internal/modules/cjs/loader:822:12

Environment

System:
    OS: macOS 12.1
    CPU: (10) arm64 Apple M1 Pro
    Shell: 5.8 - /bin/zsh
  Binaries:
    Node: 17.3.1 - ~/.nvm/versions/node/v17.3.1/bin/node
    npm: 8.3.0 - ~/.nvm/versions/node/v17.3.1/bin/npm
  Languages:
    Python: 2.7.18 - /usr/bin/python
  Browsers:
    Chrome: 99.0.4844.74
    Firefox: 98.0.1
    Safari: 15.2
  npmPackages:
    gatsby: ^4.4.0 => 4.10.0

Config Flags

No response