ref(replay): try 5s rage/dead click timeout for sentry orgs

[aliu39]

See slack thread https://sentry.slack.com/archives/C03USURCFBJ/p1725645076151569. This flag is only enabled for internal orgs

Ref #77284 and https://github.com/getsentry/sentry-options-automator/pull/2279

[JoshFerge]

you could also make this an option if you wanted to have it be more controllable. but this makes sense!

[codecov]

❌ 6 Tests Failed:

Tests completed	Failed	Passed	Skipped
21529	6	21523	205

View the top 3 failed tests by shortest run time

tests.sentry.replays.unit.test_ingest_dom_index test_emit_click_negative_node_id

Stack Traces | 0.052s run time

#x1B[1m#x1B[.../replays/unit/test_ingest_dom_index.py#x1B[0m:1139: in test_emit_click_negative_node_id
    user_actions = get_user_actions(1, uuid.uuid4().hex, events, None)
#x1B[1m#x1B[.../usecases/ingest/dom_index.py#x1B[0m:188: in get_user_actions
    click = _handle_breadcrumb(event, project_id, replay_id, replay_event)
#x1B[1m#x1B[.../usecases/ingest/dom_index.py#x1B[0m:399: in _handle_breadcrumb
    project = Project.objects.get(id=project_id)
#x1B[1m#x1B[31m.venv/lib/python3.12.../db/models/manager.py#x1B[0m:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.12.../db/models/query.py#x1B[0m:649: in get
    raise self.model.DoesNotExist(
#x1B[1m#x1B[31mE   sentry.models.project.Project.DoesNotExist: Project matching query does not exist.#x1B[0m

tests.sentry.replays.unit.test_ingest_dom_index test_get_user_actions_missing_node

Stack Traces | 0.052s run time

#x1B[1m#x1B[.../replays/unit/test_ingest_dom_index.py#x1B[0m:121: in test_get_user_actions_missing_node
    user_actions = get_user_actions(1, uuid.uuid4().hex, events, None)
#x1B[1m#x1B[.../usecases/ingest/dom_index.py#x1B[0m:188: in get_user_actions
    click = _handle_breadcrumb(event, project_id, replay_id, replay_event)
#x1B[1m#x1B[.../usecases/ingest/dom_index.py#x1B[0m:399: in _handle_breadcrumb
    project = Project.objects.get(id=project_id)
#x1B[1m#x1B[31m.venv/lib/python3.12.../db/models/manager.py#x1B[0m:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.12.../db/models/query.py#x1B[0m:649: in get
    raise self.model.DoesNotExist(
#x1B[1m#x1B[31mE   sentry.models.project.Project.DoesNotExist: Project matching query does not exist.#x1B[0m

tests.sentry.replays.unit.test_ingest_dom_index test_get_user_actions_str_payload

Stack Traces | 0.052s run time

#x1B[1m#x1B[.../replays/unit/test_ingest_dom_index.py#x1B[0m:99: in test_get_user_actions_str_payload
    user_actions = get_user_actions(1, uuid.uuid4().hex, events, None)
#x1B[1m#x1B[.../usecases/ingest/dom_index.py#x1B[0m:188: in get_user_actions
    click = _handle_breadcrumb(event, project_id, replay_id, replay_event)
#x1B[1m#x1B[.../usecases/ingest/dom_index.py#x1B[0m:399: in _handle_breadcrumb
    project = Project.objects.get(id=project_id)
#x1B[1m#x1B[31m.venv/lib/python3.12.../db/models/manager.py#x1B[0m:87: in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
#x1B[1m#x1B[31m.venv/lib/python3.12.../db/models/query.py#x1B[0m:649: in get
    raise self.model.DoesNotExist(
#x1B[1m#x1B[31mE   sentry.models.project.Project.DoesNotExist: Project matching query does not exist.#x1B[0m

To view individual test run time comparison to the main branch, go to the Test Analytics Dashboard

[cmanallen]

This will blow up the database. You're running the query for every segment ingested (2k RPS) multiplied by every breadcrumb in the recording data. This is going to query the database anywhere from 20k times per second to 200k times per second. Or more realistically, we won't query at that rate and will begin to backlog as the database becomes the primary choke point. This approach won't work.

[cmanallen]

Let's not use a feature flag here. Let's set a list-type option. In options-automator you can specify the org-ids/project-ids you want to allow to use the new timeout. You can do the same for the timeout itself as mentioned by Josh.