-
Notifications
You must be signed in to change notification settings - Fork 152
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Windows runner for CI workflows #1275
Changes from 72 commits
3434d63
59e6a36
2e3aae4
02f785c
ba4f774
8c6162d
58dbad1
ad6fe0f
1069183
8b216d9
864bdd7
0a95a68
235ca60
3b3f1e8
93e73c7
6284ec4
4387414
236ad95
1a6def3
0468402
2b117ae
6ad568b
4a62d9f
f7dba31
82e3a9c
a7cd2b2
00986d2
787c9ba
facdfe5
9326f3f
5edfd44
6204d6a
07a2f85
b66d256
9fcb57e
e0880e7
9ea1746
8c7111e
068373d
af37758
aed95c2
95c793f
722851f
50c48f2
fc01542
d6966e9
869cf3b
cc596f4
ca7c6dd
c619e88
9a9b362
bb352bb
b3dc358
8b798eb
a11e5ae
57b9c58
8dde509
af67535
84dd998
9954325
1836b5b
dc40463
a304c2c
5e4542b
895c627
6ae66d6
91da046
f49f3d6
6100ab8
c4f2472
3dbbe4d
9da6a34
0c95b14
677e6c1
6a6c6e2
e9fcee4
edfd2f0
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,53 @@ | ||
name: 'Download Security Plugin' | ||
description: 'Installs OpenSearch Dashboard with a Plugin from github, then checkouts the correct dashboards version for the plugin, configures npm/yarn, and bootstraps Dashboards' | ||
|
||
inputs: | ||
opensearch-version: | ||
description: 'The version of OpenSearch that should be used, e.g "3.0.0"' | ||
required: true | ||
|
||
plugin-name: | ||
description: 'The the name of the plugin to use, such as opensearch-security' | ||
required: true | ||
|
||
plugin-version: | ||
description: 'The version of security plugin that should be used, e.g "3.0.0.0"' | ||
required: true | ||
|
||
runs: | ||
using: "composite" | ||
steps: | ||
- name: Download OpenSearch for Linux | ||
uses: peternied/download-file@v2 | ||
if: ${{ runner.os == 'Linux' }} | ||
with: | ||
url: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${{ inputs.opensearch-version }}/7028/linux/x64/tar/builds/opensearch/plugins/${{ inputs.plugin-name }}-${{ inputs.plugin-version }}.zip | ||
|
||
- name: Download OpenSearch for Windows | ||
uses: peternied/download-file@v2 | ||
if: ${{ runner.os == 'Windows' }} | ||
with: | ||
url: https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${{ inputs.opensearch-version }}/7028/windows/x64/zip/builds/opensearch/plugins/${{ inputs.plugin-name }}-${{ inputs.plugin-version }}.zip | ||
|
||
- name: Rename the Plugin Files | ||
run: mv opensearch-security-${{ inputs.plugin-version }}.zip opensearch-security.zip | ||
shell: bash | ||
|
||
- name: Create Setup Script for Linux | ||
if: ${{ runner.os == 'Linux' }} | ||
run: | | ||
cat > setup.sh <<'EOF' | ||
chmod +x ./opensearch-${{ inputs.opensearch-version}}-SNAPSHOT/plugins/${{ inputs.plugin-name }}/tools/install_demo_configuration.sh | ||
/bin/bash -c "yes | ./opensearch-${{ inputs.opensearch-version}}-SNAPSHOT/plugins/${{ inputs.plugin-name }}/tools/install_demo_configuration.sh" | ||
echo "plugins.security.unsupported.restapi.allow_securityconfig_modification: true" >> ./opensearch-${{ inputs.opensearch-version }}-SNAPSHOT/config/opensearch.yml | ||
EOF | ||
shell: bash | ||
|
||
- name: Create Setup Script for Windows | ||
if: ${{ runner.os == 'Windows' }} | ||
run: | | ||
New-Item .\setup.bat -type file | ||
Set-Content .\setup.bat -Value "powershell.exe -noexit -command `".\opensearch-${{ inputs.opensearch-version}}-SNAPSHOT\plugins\${{ inputs.plugin-name }}\tools\install_demo_configuration.bat -y -i -c`"" | ||
Add-Content -Path .\setup.bat -Value "echo plugins.security.unsupported.restapi.allow_securityconfig_modification: true >> .\opensearch-${{ inputs.opensearch-version}}-SNAPSHOT\config\opensearch.yml" | ||
Get-Content .\setup.bat | ||
shell: pwsh |
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -12,28 +12,53 @@ env: | |
jobs: | ||
tests: | ||
name: Run integration tests | ||
runs-on: ubuntu-latest | ||
strategy: | ||
fail-fast: false | ||
matrix: | ||
os: [ ubuntu-latest , windows-latest ] | ||
jdk: [ 11, 17 ] | ||
runs-on: ${{ matrix.os }} | ||
|
||
steps: | ||
- name: Checkout Branch | ||
uses: actions/checkout@v2 | ||
uses: actions/checkout@v3 | ||
|
||
- name: Set up JDK | ||
uses: actions/setup-java@v1 | ||
with: | ||
java-version: ${{ matrix.jdk }} | ||
|
||
- uses: browser-actions/setup-geckodriver@latest | ||
RyanL1997 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
- run: geckodriver --version | ||
|
||
- uses: browser-actions/setup-firefox@latest | ||
- name: Set up Firefox browser | ||
if: ${{ runner.os == 'Linux' }} | ||
uses: browser-actions/setup-firefox@v1 | ||
|
||
- run: firefox --version | ||
|
||
- name: Download OpenSearch Security Plugin | ||
run: wget --progress=bar:force:noscroll -O opensearch-security.zip https://ci.opensearch.org/ci/dbc/distribution-build-opensearch/${{ env.OPENSEARCH_VERSION }}/latest/linux/x64/tar/builds/opensearch/plugins/${{ env.PLUGIN_NAME }}-${{ env.PLUGIN_VERSION }}.zip | ||
|
||
- name: Create Setup Script | ||
if: ${{ runner.os == 'Linux' }} | ||
run: | | ||
cat > setup.sh <<'EOF' | ||
chmod +x ./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/plugins/${{ env.PLUGIN_NAME }}/tools/install_demo_configuration.sh | ||
/bin/bash -c "yes | ./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/plugins/${{ env.PLUGIN_NAME }}/tools/install_demo_configuration.sh" | ||
echo "plugins.security.unsupported.restapi.allow_securityconfig_modification: true" >> ./opensearch-${{ env.OPENSEARCH_VERSION }}-SNAPSHOT/config/opensearch.yml | ||
EOF | ||
|
||
#- name: Set up Firefox browser for Linux | ||
RyanL1997 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
# if: ${{ runner.os == 'Linux' }} | ||
# uses: browser-actions/setup-firefox@latest | ||
# with: | ||
# browser: firefox | ||
# version: latest | ||
|
||
# Browser-action version does not work on Windows | ||
- name: Set up Firefox browser for Windows | ||
if: ${{ runner.os == 'Windows' }} | ||
uses: RyanL1997/setup-browser@main | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see any changes in this fork, lets use There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I don't see any changes in your fork, can you link the commit with the changes you needed? I'm fine using your fork and should use the same fork for both linux/windows workflows |
||
with: | ||
browser: firefox | ||
version: latest | ||
|
||
- name: Download security plugin and create setup scripts | ||
uses: ./.github/actions/download-plugin | ||
with: | ||
opensearch-version: ${{ env.OPENSEARCH_VERSION }} | ||
plugin-name: ${{ env.PLUGIN_NAME }} | ||
plugin-version: ${{ env.PLUGIN_VERSION }} | ||
|
||
- name: Run Opensearch with A Single Plugin | ||
uses: opensearch-project/security/.github/actions/start-opensearch-with-one-plugin@main | ||
|
@@ -51,9 +76,18 @@ jobs: | |
run: node scripts/build_opensearch_dashboards_platform_plugins.js | ||
working-directory: ${{ steps.install-dashboards.outputs.dashboards-directory }} | ||
|
||
- name: Run integration tests | ||
- name: Run integration tests on Linux | ||
if: ${{ runner.os == 'Linux' }} | ||
run: | | ||
echo "check if opensearch is ready" | ||
curl -XGET https://localhost:9200 -u 'admin:admin' -k | ||
yarn test:jest_server --coverage | ||
working-directory: ${{ steps.install-dashboards.outputs.plugin-directory }} | ||
|
||
- name: Run integration tests on Windows | ||
if: ${{ runner.os == 'Windows' }} | ||
run: | | ||
echo "check if opensearch is ready" | ||
curl -XGET https://localhost:9200 -u 'admin:admin' -k | ||
node .\test\run_jest_tests.js --config .\test\jest.config.server.js --testPathIgnorePatterns saml_auth.test.ts | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Are all of the tests in |
||
working-directory: ${{ steps.install-dashboards.outputs.plugin-directory }} |
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
name: Unit Tests | ||
|
||
on: [push, pull_request] | ||
on: [ push, pull_request ] | ||
|
||
jobs: | ||
unit-tests: | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hardcoded build
7028
?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, I was checking with infra team this morning and seems like the
latest
doesnt include security plugin yet and that's also the reason for the failing of pre-requisite test. I will change it back tolatest
once everything is ready. This is only for now.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Please link the issue where this is being worked on.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I just linked issue in the general comment section.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If the security plugin is building on 3.0.0 why don't we have a fresh build of it available? Should we be publishing our own builds to maven for consumption so we aren't stuck because of a separate plugin, or should the build process automatically publish binaries for components that do function?
@RyanL1997 Please create a new issue in opensearch-build and start a thread in our release channel linking that issue, we need to be pushing for a solution to the systemic problem.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@peternied here we go opensearch-project/opensearch-build#3172
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Update: They have fixed the build