Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generate scalability tests based on single user benchmarking build artifacts #256

Closed
dmlemeshko opened this issue May 2, 2022 · 1 comment
Assignees

Comments

@dmlemeshko
Copy link
Member

dmlemeshko commented May 2, 2022

In elastic/kibana#130777 we pull APM traces collected during single user benchmarking journeys run on CI job and normalise into format representing a sequence of Kibana API calls: each journey has its own json files stored as artifact of that CI job.

We need to change current scalability benchmarking job to do the following:

  1. Pull artifacts from the latest single user benchmarking job build
  • we can use buildkite API to get the latest build and fetch artifacts, or try accessing artifacts directly?
  • to use buildkite API we need an API token
  • we will use public bucket to store Kibana build, plugins and jsons with API calls
  1. Generate Gatling simulations using scalability-simulation-generator :
  • node scripts/generate_simulations.js --dir <path to json files> --packageName org.kibanaLoadTest --url <kibana baseUrl>
  1. Download (or build fresh based on commit hash) Kibana build used by single user benchmarking job
  • we can probably start with fresh build based on commit hash of the buildkite build
  1. Build this project with generated simulations inside and run scalability tests
  • copy generated scala files under src/test/scala/org/kibanaLoadTest/simulation
  • build project mvn clean test-compile
  • run test gatling:test -Dgatling.simulationClass=org.kibanaLoadTest.simulation.<generated_simulation_file_name>
@dmlemeshko
Copy link
Member Author

It was implemented in #272 and #279

Simulations files are not generated, instead Gatling reads json file from scalability-simulation-generator and creates simulation scenario in runtime:

mvn gatling:test -Dgatling.simulationClass=org.kibanaLoadTest.simulation.generic.GenericJourney -DjourneyPath=<path to json file> -DKIBANA_HOST=<Kibana host url>

@dmlemeshko dmlemeshko self-assigned this Aug 18, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant