Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(eslint-plugin): rule against invalid paths #28828

Merged
merged 19 commits into from
Feb 2, 2024
Merged

chore(eslint-plugin): rule against invalid paths #28828

merged 19 commits into from
Feb 2, 2024

Conversation

kaizencc
Copy link
Contributor

This is a follow up to #28658, #28772, and #28760. We had to fix multiple places where that file path extended beyond the package itself into other areas of the local repository (that would not be available after packaging). This caused myriad issues at synth time with file not found errors.

This PR introduces a linter rule with the following specifications:

  • no inefficient paths, i.e. no going backwards multiple times. Ex. path.join(__dirname, '..', 'folder', '..', 'another-folder'). This should and can be easily simplified
  • no paths that go backwards past a package.json file. This should catch the instances we faced next time.

The yarn lint command on aws-cdk-lib took 51.47s seconds without this new rule and 53.32s seconds with the rule enabled. The difference of ~2 seconds shouldn't be a hindrance in this case but I am happy to look for additional efficiencies in the rule I've written.


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team January 23, 2024 18:01
@github-actions github-actions bot added the p2 label Jan 23, 2024
@mergify mergify bot added the contribution/core This is a PR that came from AWS. label Jan 23, 2024
@aws-cdk-automation aws-cdk-automation added the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jan 23, 2024
Copy link
Contributor

@paulhcsun paulhcsun left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, just one minor comment.

tools/@aws-cdk/eslint-plugin/lib/rules/no-invalid-path.ts Outdated Show resolved Hide resolved
tools/@aws-cdk/eslint-plugin/lib/rules/no-invalid-path.ts Outdated Show resolved Hide resolved
}

if (isPathJoinFuncCall(node)) {
// Confirm path does not have unnecessary '..'
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First: is this check against oscillating worth the squeeze? (Or is this just to make the subsequent checks easier? If so, I can dig that. Might want to write that down.)

Second: can this not be simplified in the following way:

  • .. may only occur in the path prefix.
  • So any .. whose index is after the first non-.. is incorrect:
const components = confirmFirstArgumentIsDirnameAndRemove(node.arguments);

const firstDownDir = components.findIndex((p) => p !== '..');
if (components.some((p, i) => p === '..' && i > firstDownDir) {
  // ERROR: Oscillating paths
  return;
}

Then we can observe that package.json may only exist in the very last directory that we ended up with by doing ..: if it occurs anywhere earlier that's an error:

// Note the - 1 to exclude the last directory from the check
// Exclude the case where there are no '..' at all in the path -- those are never invalid
if (firstDownDir > 0) {
  for (let i = 0; i < firstDownDir - 1; i++) {
    const pjFile = path.join(...[path.dirname(currentFile), ...components.slice(0, i), 'package.json']);
    if (existsSync(pjFile)) {
      // ERROR: this path will end up going out of the package.json directory
      return;
    }
  }
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Take a moment to reason through this table to see how this works:

// components    | firstDownDir | closest invalid package.json
['x']              0              /* N/A */
['..', 'x']        1              'package.json'
['..', '..', 'x']  2              '../package.json'

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh you should also add a check that / doesn't occur anywhere inside any of the components, otherwise our checks are going to fail:

path.join(__dirname, '../../../..', 'banana')

Is bad style but works, and anything that accidentally works will occur in our code base 😅. We need to guard against it.

let forward = false; // Determines if the path is officially going forward
let validPath = true;
for (let i=1; i<node.arguments.length; i++) {
const path = node.arguments[i].value;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are we confirming anywhere that the first argument to path.join() is __dirname ? I can't find it

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, in isPathJoinFuncCall:

function isPathJoinFuncCall(node: any): boolean {
  return (
    node.callee?.property?.name === 'join' &&
    node.arguments.length > 1 &&
    node.arguments[0].name &&
    node.arguments[0].name === '__dirname'
  )
}

Copy link
Contributor Author

@kaizencc kaizencc Jan 24, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I assume that we don't have any rule that says path.join must have __dirname as the first argument, but rereading your comment suggests that that may be indeed what you are suggesting

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No, but just saying that we don't need to do any checks on path.joins where the first argument ISN'T __dirname.

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Jan 24, 2024
Comment on lines 82 to 83
// Note i + 1 here to include the current '..' in the path, since Array.slice excludes the end index.
const pjFile = path.join(...[path.dirname(currentFile), ...args.slice(0, i + 1), 'package.json']);
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this is the correct implementation of your algo @rix0rrr

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you are right. Good catch!

I suppose we could also do i in [1..firstDownDir) as opposed to [0..firstDownDir - 1), and that would have avoided this i + 1.

@rix0rrr rix0rrr added the pr/do-not-merge This PR should not be merged at this time. label Jan 25, 2024
Copy link
Contributor

@rix0rrr rix0rrr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conditionally approved! I'll leave you to decide what to do with the i + 1/i - 1 business

Comment on lines 82 to 83
// Note i + 1 here to include the current '..' in the path, since Array.slice excludes the end index.
const pjFile = path.join(...[path.dirname(currentFile), ...args.slice(0, i + 1), 'package.json']);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, you are right. Good catch!

I suppose we could also do i in [1..firstDownDir) as opposed to [0..firstDownDir - 1), and that would have avoided this i + 1.

@mrgrain
Copy link
Contributor

mrgrain commented Jan 29, 2024

@Mergifyio update

Copy link
Contributor

mergify bot commented Jan 29, 2024

update

✅ Branch has been successfully updated

@kaizencc kaizencc removed the pr/do-not-merge This PR should not be merged at this time. label Jan 30, 2024
@kaizencc kaizencc added the pr/do-not-merge This PR should not be merged at this time. label Feb 1, 2024
@kaizencc kaizencc removed the pr/do-not-merge This PR should not be merged at this time. label Feb 2, 2024
Copy link
Contributor

mergify bot commented Feb 2, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: a8e6610
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit e5b7813 into main Feb 2, 2024
11 of 12 checks passed
@mergify mergify bot deleted the conroy/eslint4 branch February 2, 2024 20:19
Copy link
Contributor

mergify bot commented Feb 2, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

SankyRed pushed a commit that referenced this pull request Feb 8, 2024
This is a follow up to #28658, #28772, and #28760. We had to fix multiple places where that file path extended beyond the package itself into other areas of the local repository (that would not be available after packaging). This caused myriad issues at synth time with `file not found` errors.

This PR introduces a linter rule with the following specifications:
- no inefficient paths, i.e. no going backwards multiple times. Ex. `path.join(__dirname, '..', 'folder', '..', 'another-folder')`. This should and can be easily simplified
- no paths that go backwards past a `package.json` file. This should catch the instances we faced next time.

The `yarn lint` command on `aws-cdk-lib` took 51.47s seconds without this new rule and 53.32s seconds with the rule enabled. The difference of ~2 seconds shouldn't be a hindrance in this case but I am happy to look for additional efficiencies in the rule I've written.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
TheRealAmazonKendra pushed a commit that referenced this pull request Feb 9, 2024
This is a follow up to #28658, #28772, and #28760. We had to fix multiple places where that file path extended beyond the package itself into other areas of the local repository (that would not be available after packaging). This caused myriad issues at synth time with `file not found` errors.

This PR introduces a linter rule with the following specifications:
- no inefficient paths, i.e. no going backwards multiple times. Ex. `path.join(__dirname, '..', 'folder', '..', 'another-folder')`. This should and can be easily simplified
- no paths that go backwards past a `package.json` file. This should catch the instances we faced next time.

The `yarn lint` command on `aws-cdk-lib` took 51.47s seconds without this new rule and 53.32s seconds with the rule enabled. The difference of ~2 seconds shouldn't be a hindrance in this case but I am happy to look for additional efficiencies in the rule I've written.

----

*By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license*
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
contribution/core This is a PR that came from AWS. p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants