Skip to content

Commit

Permalink
ci(telemetry): fix some flaky tests (#8372)
Browse files Browse the repository at this point in the history
This change makes the following failed assertions more resilient to
unexpected test agent states:


https://app.circleci.com/pipelines/github/DataDog/dd-trace-py/55561/workflows/dc853e23-5e79-42d3-869c-bb1e6c83d7c3/jobs/3536110

https://app.circleci.com/pipelines/github/DataDog/dd-trace-py/55552/workflows/de8ca2cf-5f26-4eab-a09e-53a44717686b/jobs/3535813

## Checklist

- [x] Change(s) are motivated and described in the PR description
- [x] Testing strategy is described if automated tests are not included
in the PR
- [x] Risks are described (performance impact, potential for breakage,
maintainability)
- [x] Change is maintainable (easy to change, telemetry, documentation)
- [x] [Library release note
guidelines](https://ddtrace.readthedocs.io/en/stable/releasenotes.html)
are followed or label `changelog/no-changelog` is set
- [x] Documentation is included (in-code, generated user docs, [public
corp docs](https://github.com/DataDog/documentation/))
- [x] Backport labels are set (if
[applicable](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting))
- [x] If this PR changes the public interface, I've notified
`@DataDog/apm-tees`.
- [x] If change touches code that signs or publishes builds or packages,
or handles credentials of any kind, I've requested a review from
`@DataDog/security-design-and-guidance`.

## Reviewer Checklist

- [ ] Title is accurate
- [ ] All changes are related to the pull request's stated goal
- [ ] Description motivates each change
- [ ] Avoids breaking
[API](https://ddtrace.readthedocs.io/en/stable/versioning.html#interfaces)
changes
- [ ] Testing strategy adequately addresses listed risks
- [ ] Change is maintainable (easy to change, telemetry, documentation)
- [ ] Release note makes sense to a user of the library
- [ ] Author has acknowledged and discussed the performance implications
of this PR as reported in the benchmarks PR comment
- [ ] Backport labels are set in a manner that is consistent with the
[release branch maintenance
policy](https://ddtrace.readthedocs.io/en/latest/contributing.html#backporting)
  • Loading branch information
emmettbutler committed Feb 14, 2024
1 parent 213f5b0 commit 56422f3
Showing 1 changed file with 7 additions and 6 deletions.
13 changes: 7 additions & 6 deletions tests/telemetry/test_telemetry.py
Original file line number Diff line number Diff line change
Expand Up @@ -158,6 +158,7 @@ def test_enable_fork_heartbeat(test_agent_session, run_python_code_in_subprocess
telemetry_writer.disable()
"""

initial_requests_count = len(test_agent_session.get_requests())
stdout, stderr, status, _ = run_python_code_in_subprocess(code)
assert status == 0, stderr
assert stderr == b"", stderr
Expand All @@ -167,11 +168,10 @@ def test_enable_fork_heartbeat(test_agent_session, run_python_code_in_subprocess
requests = test_agent_session.get_requests()

# We expect events from the parent process to get sent, but none from the child process
# flaky
# assert len(requests) == 1
# Validate that the runtime id sent for every event is the parent processes runtime id
assert requests[0]["body"]["runtime_id"] == runtime_id
assert requests[0]["body"]["request_type"] == "app-heartbeat"
assert len(requests) == initial_requests_count + 1
matching_requests = [r for r in requests if r["body"]["runtime_id"] == runtime_id]
assert len(matching_requests) == 1
assert matching_requests[0]["body"]["request_type"] == "app-heartbeat"


def test_heartbeat_interval_configuration(run_python_code_in_subprocess):
Expand Down Expand Up @@ -245,13 +245,14 @@ def process_trace(self, trace):
# generate and encode span
tracer.trace("hello").finish()
"""
initial_event_count = len(test_agent_session.get_events())
_, stderr, status, _ = run_python_code_in_subprocess(code)
assert status == 0, stderr
assert b"Exception raised in trace filter" in stderr

events = test_agent_session.get_events()

assert len(events) == 3
assert len(events) == initial_event_count + 3

# Same runtime id is used
assert events[0]["runtime_id"] == events[1]["runtime_id"]
Expand Down

0 comments on commit 56422f3

Please sign in to comment.