-
Notifications
You must be signed in to change notification settings - Fork 19
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Implementation of basic load test capabilities for websocket functionality on the board. * improve performance of findCards by adding new function getBoardAuthorizables * track number of action calls in separate metric * implementation of a load testing shell script * added option to pass target and scenario as parameters to the shell script * readme.md for board related load testing --------- Co-authored-by: Thomas Feldtkeller <thomas.feldtkeller@dataport.de>
- Loading branch information
1 parent
a82c051
commit 324b9e6
Showing
14 changed files
with
574 additions
and
26 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| # Loadtesting the boards | ||
|
|
||
| The socket.io documentation suggests to use the tool artillery in order to load test a socket-io tool like our board-collaboration service. | ||
|
|
||
| For defining scenarios you need to use/create Yaml-files that define which operations with which parameters need to be executed in which order. | ||
|
|
||
| Some sceneraios were already prepared and are stored in the subfolder scenarios. | ||
|
|
||
| ## install artillery | ||
|
|
||
| To run artillery from your local environment you need to install it first including an adapter that supports socketio-v3-websocket communication: | ||
|
|
||
| ```sh | ||
| npm install -g artillery artillery-engine-socketio-v3 | ||
| ``` | ||
|
|
||
| ## manual execution | ||
|
|
||
| To execute a scenario you can run artillery from the shell / commandline...: | ||
|
|
||
| Using the `--variables` parameter it is possible to define several variables and there values that can be used in the scenerio-yaml-file: | ||
|
|
||
| - **target**: defines the base url for all requests (REST and WebSocket) | ||
| e.g. `https://main.dbc.dbildungscloud.dev` | ||
| - **token**: a valid JWT for the targeted system | ||
| - **board_id**: id of an existing board the tests should be executed on | ||
|
|
||
| ```bash | ||
| npx artillery run --variables "{'target': 'https://main.dbc.dbildungscloud.dev', 'token': 'eJ....', 'board_id': '668d0e03bf3689d12e1e86fb' }" './scenarios/3users.yml' --output artilleryreport.json | ||
| ``` | ||
|
|
||
| On Windows Powershell, the variables value needs to be wrapped in singlequotes, and inside the json you need to use backslash-escaped doublequotes: | ||
|
|
||
| ```powershell | ||
| npx artillery run --variables '{\"target\": \"https://main.dbc.dbildungscloud.dev\", \"token\": \"eJ....\", \"board_id\": \"668d0e03bf3689d12e1e86fb\" }' './scenarios/3users.yml' --output artilleryreport.json | ||
| ``` | ||
|
|
||
| ## visualizing the recorded results | ||
|
|
||
| It is possible to generate a HTML-report based on the recorded data. | ||
|
|
||
| ```powershell | ||
| npx artillery report --output=$board_title.html artilleryreport.json | ||
| ``` | ||
|
|
||
| ## automatic execution | ||
|
|
||
| You can run one of the existing scenarios by executing: | ||
|
|
||
| ```bash | ||
| bash runScenario.sh | ||
| ``` | ||
|
|
||
| This will: | ||
|
|
||
| 1. let you choose from scenario-files | ||
| 2. create a fresh JWT-webtoken | ||
| 3. create a fresh board (in one of the courses) the user has access to | ||
| 4. name the board by a combination of datetime and the scenario name. | ||
| 5. output a link to the generated board (in order open and see the test live) | ||
| 6. start the execution of the scenario against this newly created board | ||
| 7. generate a html report in the end | ||
|
|
||
| You can also provide the target as the first and the name of the scenario as the second parameter - to avoid the need to select those. Here is an example: | ||
|
|
||
| ```bash | ||
| bash runScenario.sh https://bc-6854-basic-load-tests.nbc.dbildungscloud.dev 3users | ||
| ``` | ||
|
|
||
| ## password | ||
|
|
||
| By typeing `export CARL_CORD_PASSWORD=realpassword` the script will not ask you anymore for the password to create a token. | ||
|
|
||
| ## Todos | ||
|
|
||
| - [ ] enable optional parameter course_id |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,139 @@ | ||
| #!/bin/bash | ||
|
|
||
| function select_target() { | ||
| declare -a targets=("https://main.nbc.dbildungscloud.dev" "https://bc-6854-basic-load-tests.nbc.dbildungscloud.dev") | ||
| echo "Please select the target for the test:" >&2 | ||
| select target in "${targets[@]}"; do | ||
| if [[ -n $target ]]; then | ||
| break | ||
| else | ||
| echo "Invalid selection. Please try again." >&2 | ||
| fi | ||
| done | ||
| } | ||
|
|
||
| function select_scenario() { | ||
| # list files in the scenarios directory | ||
| scenarios_dir="./scenarios" | ||
| declare -a scenario_files=($(ls $scenarios_dir)) | ||
|
|
||
| echo "Please select a scenario file for the test:" >&2 | ||
| select scenario_file in "${scenario_files[@]}"; do | ||
| if [[ -n $scenario_file ]]; then | ||
| echo "You have selected: $scenario_file" >&2 | ||
| break | ||
| else | ||
| echo "Invalid selection. Please try again." >&2 | ||
| fi | ||
| done | ||
|
|
||
| scenario_name="${scenario_file%.*}" | ||
| } | ||
|
|
||
| function get_credentials() { | ||
| if [ -z "$CARL_CORD_PASSWORD" ]; then | ||
| echo "Password for Carl Cord is unknown. Provide it as an enviroment variable (CARL_CORD_PASSWORD) or enter it:" | ||
| read CARL_CORD_PASSWORD | ||
| export CARL_CORD_PASSWORD | ||
| fi | ||
| } | ||
|
|
||
| function get_token() { | ||
| response=$(curl -s -f -X 'POST' \ | ||
| "$target/api/v3/authentication/local" \ | ||
| -H 'accept: application/json' \ | ||
| -H 'Content-Type: application/json' \ | ||
| -d "{ | ||
| \"username\": \"lehrer@schul-cloud.org\", | ||
| \"password\": \"$CARL_CORD_PASSWORD\" | ||
| }") | ||
|
|
||
| if [ $? -ne 0 ]; then | ||
| echo "ERROR: Failed to get token. Please check your credentials and target URL." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| token=$(echo $response | sed -n 's/.*"accessToken":"\([^"]*\)".*/\1/p') | ||
| } | ||
|
|
||
| function get_course_id() { | ||
| response=$(curl -s -f -X 'GET' \ | ||
| "$target/api/v3/courses" \ | ||
| -H "Accept: application/json" \ | ||
| -H "Authorization: Bearer $token") | ||
|
|
||
| if [ $? -ne 0 ]; then | ||
| echo "ERROR: Failed to get course list. Please check your credentials and target URL." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| course_id=$(echo $response | sed -n 's/.*"id":"\([^"]*\)".*/\1/p') | ||
| } | ||
|
|
||
| function create_board_title() { | ||
| current_date=$(date +%Y-%m-%d_%H:%M) | ||
| board_title="${current_date}_$1" | ||
| } | ||
|
|
||
| function create_board() { | ||
| response=$(curl -s -f -X 'POST' \ | ||
| "$target/api/v3/boards" \ | ||
| -H 'accept: application/json' \ | ||
| -H 'Content-Type: application/json' \ | ||
| -H "Authorization: Bearer $token" \ | ||
| -d "{ | ||
| \"title\": \"$board_title\", | ||
| \"parentId\": \"$course_id\", | ||
| \"parentType\": \"course\", | ||
| \"layout\": \"columns\" | ||
| }") | ||
|
|
||
| if [ $? -ne 0 ]; then | ||
| echo "ERROR: Failed to create a board." >&2 | ||
| exit 1 | ||
| fi | ||
|
|
||
| board_id=$(echo $response | sed -n 's/.*"id":"\([^"]*\)".*/\1/p' ) | ||
| } | ||
|
|
||
| if [ -z "$1" ]; then | ||
| select_target | ||
| else | ||
| target=$1 | ||
| fi | ||
| echo " " | ||
| echo "target: $target" | ||
|
|
||
|
|
||
| if [ -z "$2" ]; then | ||
| select_scenario | ||
| echo "scenario_name: $scenario_name" | ||
| else | ||
| scenario_name="$2" | ||
| scenario_name=${scenario_name//.yml/} | ||
| fi | ||
| echo "scenario_name: $scenario_name" | ||
|
|
||
| get_credentials | ||
|
|
||
| get_token | ||
| echo "token: ${token:0:50}..." | ||
| echo " " | ||
|
|
||
| get_course_id | ||
| echo "course_id: $course_id" | ||
| echo " " | ||
|
|
||
| create_board_title $scenario_name | ||
| echo "board_title: $board_title" | ||
|
|
||
| create_board | ||
| echo "board_id $board_id" | ||
|
|
||
| echo "board: $target/rooms/$board_id/board" | ||
| echo " " | ||
| echo "Running artillery test..." | ||
|
|
||
| npx artillery run --variables "{\"target\": \"$target\", \"token\": \"$token\", \"board_id\": \"$board_id\" }" "./scenarios/$scenario_name.yml" --output artilleryreport.json | ||
|
|
||
| npx artillery report --output=$board_title.html artilleryreport.json |
75 changes: 75 additions & 0 deletions
75
apps/server/src/modules/board/loadtest/scenarios/30users_5minutes.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| config: | ||
| target: '{{ target }}' | ||
| engines: | ||
| socketio: | ||
| transport: ['websocket', 'polling'] | ||
| path: '/board-collaboration' | ||
| socketio-v3: | ||
| path: '/board-collaboration' | ||
| timeout: 1000000 | ||
| extraHeaders: | ||
| Cookie: 'jwt={{ token }}' | ||
|
|
||
| phases: | ||
| - duration: 300 | ||
| arrivalRate: 10 | ||
| maxVusers: 30 | ||
|
|
||
| scenarios: | ||
| - name: create card | ||
| engine: socketio-v3 | ||
| socketio-v3: | ||
| extraHeaders: | ||
| Cookie: 'jwt={{ token }}' | ||
| flow: | ||
| - think: 1 | ||
|
|
||
| - emit: | ||
| channel: 'fetch-board-request' | ||
| data: | ||
| boardId: '{{ board_id }}' | ||
|
|
||
| - think: 1 | ||
|
|
||
| - emit: | ||
| channel: 'create-column-request' | ||
| data: | ||
| boardId: '{{ board_id }}' | ||
| response: | ||
| on: 'create-column-success' | ||
| capture: | ||
| - json: $.newColumn.id | ||
| as: columnId | ||
|
|
||
| - think: 1 | ||
|
|
||
| - loop: | ||
| - emit: | ||
| channel: 'create-card-request' | ||
| data: | ||
| columnId: '{{ columnId}}' | ||
| response: | ||
| on: 'create-card-success' | ||
| capture: | ||
| - json: $.newCard.id | ||
| as: cardId | ||
|
|
||
| - think: 1 | ||
|
|
||
| - emit: | ||
| channel: 'fetch-card-request' | ||
| data: | ||
| cardIds: | ||
| - '{{ cardId }}' | ||
|
|
||
| - think: 2 | ||
|
|
||
| - emit: | ||
| channel: 'update-card-title-request' | ||
| data: | ||
| cardId: '{{ cardId }}' | ||
| newTitle: 'Card {{ cardId}}' | ||
|
|
||
| - think: 1 | ||
|
|
||
| count: 20 |
Oops, something went wrong.