Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(ci): use unified Gitlab pipeline for APM libraries [backport 2.9] #10529

Merged
merged 3 commits into from
Sep 9, 2024

Conversation

emmettbutler
Copy link
Collaborator

@emmettbutler emmettbutler commented Sep 5, 2024

What does this PR do?
Converts the Gitlab workflow to use the unified pipeline.

By using the unified pipeline across all APM libraries, there is less maintenance hassle, fewer mistakes, and a more uniform creation of artifacts.

Additionally, this change creates OCI packages and lib inject images for every dd-trace-py build. Previously, only released versions to pypi were built and tested.

The changes can be summarized as follows:

  • Removing github actions that are now taken over by the unified pipeline
  • Removing Gitlab jobs taken over by the unified pipeline
  • Adding a job to download artifacts from the github build
  • Merging the download artifacts with dependencies from pypi

.gitlab-ci.yml is best reviewed in its final form rather than as a diff.

How to test the change?
The changes were tested manually. CI/CD workflows are notoriously hard to test. Final products the final lib-init image was run through automated onboarding tests. dd-trace-py doesn't have automated tests for OCI images

Checklist

  • PR author has checked that all the criteria below are met
  • The PR description includes an overview of the change
  • The PR description articulates the motivation for the change
  • The change includes tests OR the PR description describes a testing strategy
  • The PR description notes risks associated with the change, if any
  • Newly-added code is easy to change
  • The change follows the library release note guidelines
  • The change includes or references documentation updates if necessary
  • Backport labels are set (if applicable)

Reviewer Checklist

  • Reviewer has checked that all the criteria below are met
  • Title is accurate
  • All changes are related to the pull request's stated goal
  • Avoids breaking API changes
  • Testing strategy adequately addresses listed risks
  • Newly-added code is easy to change
  • Release note makes sense to a user of the library
  • If necessary, author has acknowledged and discussed the performance implications of this PR as reported in the benchmarks PR comment
  • Backport labels are set in a manner that is consistent with the release branch maintenance policy

**What does this PR do?**
Converts the Gitlab workflow to use the unified pipeline.

By using the unified pipeline across all APM libraries, there is less
maintenance hassle, fewer mistakes, and a more uniform creation of
artifacts.

Additionally, this change creates OCI packages and lib inject images for
every `dd-trace-py` build. Previously, only released versions to pypi
were built and tested.

The changes can be summarized as follows:
* Removing github actions that are now taken over by the unified
pipeline
* Removing Gitlab jobs taken over by the unified pipeline
* Adding a job to download artifacts from the github build
* Merging the download artifacts with dependencies from pypi

`.gitlab-ci.yml` is best reviewed in its final form rather than as a
diff.

**How to test the change?**
The changes were tested manually. CI/CD workflows are notoriously hard
to test. Final products the final lib-init image was run through
automated onboarding tests. `dd-trace-py` doesn't have automated tests
for OCI images

- [x] PR author has checked that all the criteria below are met
- The PR description includes an overview of the change
- The PR description articulates the motivation for the change
- The change includes tests OR the PR description describes a testing
strategy
- The PR description notes risks associated with the change, if any
- Newly-added code is easy to change
- The change follows the [library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
- The change includes or references documentation updates if necessary
- Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))

- [x] Reviewer has checked that all the criteria below are met
- Title is accurate
- All changes are related to the pull request's stated goal
- Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- Testing strategy adequately addresses listed risks
- Newly-added code is easy to change
- Release note makes sense to a user of the library
- If necessary, author has acknowledged and discussed the performance
implications of this PR as reported in the benchmarks PR comment
- Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
@christophe-papazian christophe-papazian added the changelog/no-changelog A changelog entry is not required for this PR. label Sep 5, 2024
@brettlangdon brettlangdon enabled auto-merge (squash) September 6, 2024 13:44
@brettlangdon
Copy link
Member

Unable to create pipeline
'promote-oci-to-staging' job needs 'onboarding_tests_installer' job, but 'onboarding_tests_installer' is not in any previous stage
'promote-oci-to-prod-beta' job needs 'onboarding_tests_installer' job, but 'onboarding_tests_installer' is not in any previous stage
'promote-oci-to-prod' job needs 'onboarding_tests_installer' job, but 'onboarding_tests_installer' is not in any previous stage
'publish-lib-init-pinned-tags' job needs 'onboarding_tests_installer' job, but 'onboarding_tests_installer' is not in any previous stage
'publish-lib-init-ghcr-tags' job needs 'onboarding_tests_installer' job, but 'onboarding_tests_installer' is not in any previous stage

Seems there is something missing. Probably because there have been updates to the one-pipeline since the initial PR which introduced it.... this is going to be a bit of a pain.....

@emmettbutler
Copy link
Collaborator Author

emmettbutler commented Sep 9, 2024

Blocked on #10563 cc @erikayasuda

@taegyunkim
Copy link
Contributor

taegyunkim commented Sep 9, 2024

cp: cannot stat '../min_compatible_versions.csv': No such file or directory
cp: cannot stat '../lib-injection/telemetry-forwarder.sh': No such file or directory

Gitlab jobs can't find files on 2.9

@emmettbutler emmettbutler merged commit 8bc2696 into 2.9 Sep 9, 2024
104 of 105 checks passed
@emmettbutler emmettbutler deleted the backport-9930-to-2.9 branch September 9, 2024 17:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
changelog/no-changelog A changelog entry is not required for this PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants