feat(escalating-issues): Detect when an issue starts escalating #47843

jangjodi · 2023-04-24T18:01:41Z

WOR-2762

Acceptance Criteria:

this should happen in the process_snoozes pipeline step
- this step currently handles the ignored logic
Read the forecast from the cache first and then from the nodestore
- Cache it if we had to hit the nodestore
Check if the current event occurence is greater than the forecast
- If yes, then:
  - we should update the GroupSubStatus to Escalating
  - we should update the GroupStatus to UNRESOLVED
  - call the add_group_to_inbox method and set to the GroupInboxReason.ESCALATING

WOR-2762

codecov · 2023-04-25T00:10:37Z

Codecov Report

Merging #47843 (4bc9099) into master (1127c14) will increase coverage by 3.31%.
The diff coverage is 87.80%.

❗ Current head 4bc9099 differs from pull request most recent head 417dc8e. Consider uploading reports for the commit 417dc8e to get more accurate results

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #47843      +/-   ##
==========================================
+ Coverage   77.50%   80.81%   +3.31%     
==========================================
  Files        4759     4763       +4     
  Lines      201207   201371     +164     
  Branches    11594    11594              
==========================================
+ Hits       155948   162745    +6797     
+ Misses      45003    38370    -6633     
  Partials      256      256

Impacted Files	Coverage Δ
src/sentry/tasks/post_process.py	`92.32% <28.57%> (+16.53%)`	⬆️
src/sentry/issues/escalating.py	`100.00% <100.00%> (ø)`
src/sentry/snuba/referrer.py	`100.00% <100.00%> (ø)`

... and 399 files with indirect coverage changes

wedamija · 2023-04-25T17:16:12Z

src/sentry/issues/escalating.py

+        cache.set(forecast_cache_key, escalating_forecast, forecast_cache_duration)
+
+    # Check if current event occurance is greater than forecast for today's date
+    group_daily_count = get_group_daily_count(group.project.id, group.id)


Let's keep an eye on this and make sure it doesn't add too much load to snuba. If it does, one option we have is to only check is_escalating for an issue at most once per minute.

(This might be a lot more important when we start auto archiving issues as ignore until escalating)

armenzg · 2023-04-25T17:05:21Z

src/sentry/tasks/post_process.py

+            return
+        job["has_reappeared"] = False
+        return
+
    with metrics.timer("post_process.process_snoozes.duration"):


It would be great if we could refactor this block into a new function in a different PR.

armenzg · 2023-04-25T17:27:45Z

src/sentry/issues/escalating.py

+        forecast_cache_duration = (
+            (escalating_forecast[0] + timedelta(days=ONE_WEEK_DURATION)).date() - date_now
+        ).total_seconds()
+        cache.set(forecast_cache_key, escalating_forecast, forecast_cache_duration)


I was thinking that we may want to do this as part of the save call in here:

sentry/src/sentry/issues/escalating_group_forecast.py

Lines 39 to 44 in 0aaef53

def save(self) -> None:

nodestore.set(

self.build_storage_identifier(self.project_id, self.group_id),

self.to_dict(),

ttl=timedelta(TWO_WEEKS_IN_DAYS_TTL),

)

armenzg · 2023-04-25T17:28:44Z

src/sentry/issues/escalating.py

+    escalating_forecast = cache.get(forecast_cache_key)
+    date_now = datetime.now().date()
+    if escalating_forecast is None:
+        escalating_forecast = EscalatingGroupForecast.fetch(group.project.id, group.id)


You can add the getting from cache directly to the fetch method, thus, simplifying the is_escalating method.

armenzg · 2023-04-26T14:34:08Z

src/sentry/issues/escalating.py

 __all__ = ["query_groups_past_counts", "parse_groups_past_counts"]

 REFERRER = "sentry.issues.escalating"
 ELEMENTS_PER_SNUBA_PAGE = 10000  # This is the maximum value for Snuba
 # The amount of data needed to generate a group forecast
 BUCKETS_PER_GROUP = 7 * 24
+ONE_WEEK_DURATION = 7
+IS_ESCALATING_REFERRER = "sentry.issues.escalating.is_escalating"


I realized it is better that we import the referrers from referrer.py.

feat(escalating-issues): Detect when an issue starts escalating

0df5a4e

WOR-2762

github-actions bot added the Scope: Backend Automatically applied to PRs that change backend components label Apr 24, 2023

vercel bot deployed to Preview April 24, 2023 18:02 View deployment

github-actions bot added the Scope: Frontend Automatically applied to PRs that change frontend components label Apr 24, 2023

vercel bot deployed to Preview April 24, 2023 18:19 View deployment

jangjodi force-pushed the jodi/wor-2762 branch from 664cb06 to 0df5a4e Compare April 24, 2023 18:20

jangjodi removed the Scope: Frontend Automatically applied to PRs that change frontend components label Apr 24, 2023

getsentry deleted a comment from github-actions bot Apr 24, 2023

Merge branch 'master' into jodi/wor-2762

0e26709

vercel bot deployed to Preview April 24, 2023 18:25 View deployment

fix: Fix merge conflicts

4bc9099

vercel bot deployed to Preview April 24, 2023 18:40 View deployment

jangjodi marked this pull request as ready for review April 24, 2023 18:44

jangjodi requested review from a team April 24, 2023 18:44

NisanthanNanthakumar approved these changes Apr 25, 2023

View reviewed changes

Merge branch 'master' into jodi/wor-2762

417dc8e

vercel bot deployed to Preview April 25, 2023 16:09 View deployment

jangjodi merged commit 2dd8afe into master Apr 25, 2023

jangjodi deleted the jodi/wor-2762 branch April 25, 2023 16:37

wedamija reviewed Apr 25, 2023

View reviewed changes

armenzg reviewed Apr 25, 2023

View reviewed changes

armenzg added this to the Escalating Issues V1 milestone Apr 25, 2023

jangjodi mentioned this pull request Apr 25, 2023

ref(escalating-issues): Remove cache from escalating issue detection #47916

Merged

armenzg reviewed Apr 26, 2023

View reviewed changes

github-actions bot locked and limited conversation to collaborators May 12, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(escalating-issues): Detect when an issue starts escalating #47843

feat(escalating-issues): Detect when an issue starts escalating #47843

jangjodi commented Apr 24, 2023 •

edited

Loading

codecov bot commented Apr 25, 2023 •

edited

Loading

wedamija Apr 25, 2023

wedamija Apr 25, 2023

armenzg Apr 25, 2023

armenzg Apr 25, 2023

armenzg Apr 25, 2023

armenzg Apr 26, 2023

	def save(self) -> None:
	nodestore.set(
	self.build_storage_identifier(self.project_id, self.group_id),
	self.to_dict(),
	ttl=timedelta(TWO_WEEKS_IN_DAYS_TTL),
	)

feat(escalating-issues): Detect when an issue starts escalating #47843

feat(escalating-issues): Detect when an issue starts escalating #47843

Conversation

jangjodi commented Apr 24, 2023 • edited Loading

codecov bot commented Apr 25, 2023 • edited Loading

Codecov Report

wedamija Apr 25, 2023

Choose a reason for hiding this comment

wedamija Apr 25, 2023

Choose a reason for hiding this comment

armenzg Apr 25, 2023

Choose a reason for hiding this comment

armenzg Apr 25, 2023

Choose a reason for hiding this comment

armenzg Apr 25, 2023

Choose a reason for hiding this comment

armenzg Apr 26, 2023

Choose a reason for hiding this comment

jangjodi commented Apr 24, 2023 •

edited

Loading

codecov bot commented Apr 25, 2023 •

edited

Loading