Add Jenkins Webhook and RFS E2E job on PR

[lewijacn]

Description

This change is hopefully the final chapter in the Jenkins setup saga and a huge leap in the integration testing story. It adds support for triggering a Jenkins job through a configured Github Action and monitoring its completion to pass/fail the action. This design is one of many that can be taken, but a benefit of this approach is that it keeps the Github action in control of triggering Jenkins jobs and status updates, as well as doesn't require Jenkins to update the PR (though this may be a useful addition for logs or other data in the future). As a part of this pattern, a general python webhook was created to trigger Jenkins jobs and monitor for a terminal state, this could also potentially become its own GHA plugin separate from this repo.

Additionally this adds our first GHA which triggers a Jenkins job for our RFS E2E integration test. This GHA is planned to be monitored as any bugs/issues are worked out and should not currently be restrictive for PRs to be merged.

Jenkins pipeline files have also been updated to attain a lock on deployment stages to prevent different pipeline executions from using the same stage. The pipeline files have also been largely moved to the shared library, to allow any changes to the context options or other settings for the pipeline to be tested in PR actions.

Issues Resolved

https://opensearch.atlassian.net/browse/MIGRATIONS-1441

Testing

See testing GHA on fork: lewijacn#18
Manual testing of webhook trigger

Check List

• New functionality includes testing
  • All tests pass, including unit test, integration test and doctest
• New functionality has been documented
• Commits are signed per the DCO using --signoff

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 80.90%. Comparing base (e64957a) to head (05461c5).
Report is 130 commits behind head on main.

Additional details and impacted files

@@             Coverage Diff              @@
##               main     #938      +/-   ##
============================================
+ Coverage     78.73%   80.90%   +2.16%     
- Complexity     2367     2732     +365     
============================================
  Files           368      400      +32     
  Lines         14243    15404    +1161     
  Branches        827      967     +140     
============================================
+ Hits          11214    12462    +1248     
+ Misses         2510     2368     -142     
- Partials        519      574      +55     

Flag	Coverage Δ
gradle-test	76.95% <ø> (+1.91%)	⬆️
python-test	92.87% <ø> (+4.24%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[peternied]

Nice that you got this working, but I think this introduces a maintainance burden for the jenkins trigger - it looks like the GHA marketplace has some actions that can handle our use case. Can we us them instead?

[peternied]

Much cleaner, thanks. Just waiting to see a clean run on your fork and then I'll approve.

[peternied]

Minor comments, you are good to merge when ready - nicely done!