Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add benchmarks to every type of package in build workflow #249

Closed
NathanielRN opened this issue Dec 11, 2020 · 4 comments
Closed

Add benchmarks to every type of package in build workflow #249

NathanielRN opened this issue Dec 11, 2020 · 4 comments
Labels

Comments

@NathanielRN
Copy link
Contributor

In #243 we add performance tests for the AWS SDK Extension.

The AWS SDK extension falls under the category sdkextension of tests matched in the build workflow. For this reason, when the jq command looks through the entire repo after the workflow for *-benchmark.json files we are guaranteed to find at least ` file. This file is then given to the subsequent Github action to report on benchmarking results.

However, the build workflow also has instrumentation and exporter packages. These don't have performance tests yet so the jq command will fail. Even if we suppress the failure, the subsequent GitHub action will fail if no valid output.json file is created.

A temporary solution was to only run those 2 steps of the workflow if: matrix.package == 'sdkextension'. A more long-term solution will be to add benchmarks to instrumentation and exporter.

# TODO: Add at least one benchmark to every package type to remove this
if: matrix.package == 'sdkextension'

and

# TODO: Add at least one benchmark to every package type to remove this
if: matrix.package == 'sdkextension'

We know only these packages are required because as long as subsequent tests fit under these categories we won't have to keep adding if statements.

@github-actions
Copy link

github-actions bot commented Apr 6, 2021

This issue was marked stale due to lack of activity. It will be closed in 30 days.

@github-actions
Copy link

github-actions bot commented Jun 5, 2021

Closed as inactive. Feel free to reopen if this issue needs resolving.

@github-actions github-actions bot closed this as completed Jun 5, 2021
@NathanielRN
Copy link
Contributor Author

We should probably keep this open until we have benchmarks for all relevant contrib packages (and a relevant way to force new packages to include them)!

@lzchen @codeboten

@lzchen
Copy link
Contributor

lzchen commented Jun 9, 2021

@NathanielRN
If you are interested in adding these benchmarks, feel free to create PRs for them and we'll be happy to review them :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants