Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add additional News P3A metrics #35434

Closed
Tracked by #35960
DJAndries opened this issue Jan 19, 2024 · 9 comments · Fixed by brave/brave-core#21639
Closed
Tracked by #35960

Add additional News P3A metrics #35434

DJAndries opened this issue Jan 19, 2024 · 9 comments · Fixed by brave/brave-core#21639
Assignees
@DJAndries
Labels
feature/new-metric Proposing a new metric for Privacy Preserving Product Analytics (P3A) features/P3A OS/Android Fixes related to Android browser functionality OS/Desktop QA Pass - Android ARM QA Pass-Win64 QA/Yes release-notes/exclude
Milestone
1.64.x - Release

Comments

@DJAndries
Copy link
Collaborator

DJAndries commented Jan 19, 2024 •
edited
Loading

Add five metrics for News:

If you have used Brave News in the last week and clicked on at least one card, how many cards did you click on? (reported weekly, desktop & Android)
Brave.Today.WeeklyTotalCardClicks

  1. 1-2
  2. 3-5
  3. 6 -10
  4. 11-15
  5. 16-20
  6. 21-25
  7. 26+

If you have clicked on 5 or more cards in the last week, what is the average rank (depth) of the cards you clicked on in the For You/Following feeds? (reported weekly, desktop & Android)
Brave.Today.ClickCardDepth

  1. 1-3 cards deep
  2. 4 to 6 cards deep
  3. 7 to 10 cards deep
  4. 11 to 15 cards deep
  5. 16 to 20 cards deep
  6. 21+

If you are a monthly active user, how many channels are you subscribed to? (reported monthly, desktop & Android only)
Brave.Today.ChannelCount

  1. 0-1
  2. 2-4
  3. 5-7
  4. 8-10
  5. 11+

If you are a monthly active user, how many publishers are you subscribed to (excluding direct feeds)? (reported monthly, desktop & Android only)
Brave.Today.PublisherCount

  1. 0-1
  2. 2-4
  3. 5-7
  4. 8-10
  5. 11+

If you have used Brave News in the last week and you used the sidebar filter at least once, how many sessions included the usage of the sidebar filters? (reported weekly, desktop only)
Brave.Today.SidebarFilterUsages

  1. 1
  2. 2-4
  3. 5-7
  4. 8-10
  5. 11+

Also, replace Brave.Today.DirectFeedsTotal with Brave.Today.DirectFeedsTotal.2:

As a Brave News user this month, how many external feeds do you have in total? (reported monthly, with same buckets)

cc @LorenzoMinto
@DJAndries DJAndries added QA/Yes release-notes/exclude OS/Android Fixes related to Android browser functionality features/P3A OS/Desktop feature/new-metric Proposing a new metric for Privacy Preserving Product Analytics (P3A) labels Jan 19, 2024
@DJAndries DJAndries self-assigned this Jan 19, 2024
@DJAndries DJAndries mentioned this issue Jan 19, 2024
Merged
24 tasks
@LorenzoMinto
Copy link
Member

@DJAndries for (1) and (2), could we use Click instead of Visit. Visit could be mistaken as a feed visit. It also matches the language in the question

@LorenzoMinto LorenzoMinto mentioned this issue Feb 12, 2024
Open
@brave-builds brave-builds added this to the 1.64.x - Nightly milestone Feb 14, 2024
@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Feb 26, 2024 •
edited
Loading

Verification PASSED on

Brave | 1.64.74 Chromium: 122.0.6261.43 (Official Build) beta (64-bit)
-- | --
Revision | b03db955df4f1c679cb7c2ae8ebfd331d5c29276
OS | Windows 10 Version 22H2 (Build 19045.4046)

Brave.Today.WeeklyTotalCardClicks

Brave.Today.WeeklyTotalCardClicks_metrics is not shown_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and don't click on any card
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks isn't shown in local state file
Brave.Today.WeeklyTotalCardClicks_0_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on any one card
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 0 as expected (as user clicked on one card)
Example Example Example
image image image

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on two cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 0 as expected (as user clicked on two cards)
Example Example
image image
Brave.Today.WeeklyTotalCardClicks_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on three cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 1 as expected (as user clicked on three cards)
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on five cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 1 as expected (as user clicked on five cards)

image

Brave.Today.WeeklyTotalCardClicks_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on six cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 2 as expected (as user clicked on six cards)
    image

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on nine cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 2 as expected (as user clicked on nine cards)
    image

Case 3:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on ten cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 2 as expected (as user clicked on ten cards)
    image
Brave.Today.WeeklyTotalCardClicks_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 11 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 3 as expected (as user clicked on 11 cards)
    image

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 15 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 3 as expected (as user clicked on 15 cards)

image

Brave.Today.WeeklyTotalCardClicks_4_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 16 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 4 as expected (as user clicked on 16 cards)
    image

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 20 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 4 as expected (as user clicked on 20 cards)
    image
Brave.Today.WeeklyTotalCardClicks_5_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 21 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 5 as expected (as user clicked on 21 cards)
    image

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 25 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 5 as expected (as user clicked on 25 cards)
    image
Brave.Today.WeeklyTotalCardClicks_6_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 26 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 6 as expected (as user clicked on 26 cards)
    image

Brave.Today.ClickCardDepth

Note: Minium 5 cards needs to be clicked to see the response value

Brave.Today.ClickCardDepth_0_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Click on 1st card twice
  4. Click on 2nd card twice
  5. Click on 3rd card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 0 as expected as user clicked on 1 to 3 cards deep [1 to 3 cards deep]

image

Brave.Today.ClickCardDepth_1_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 4th card and click on 4th card twice
  4. Click on 5th card twice
  5. Click on 6th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 1 as expected as user clicked on 4 to 6 cards deep [4 to 6 cards deep]
    image
Brave.Today.ClickCardDepth_2_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 7th card and click on 7th card twice
  4. Click on 8th card twice
  5. Click on 9th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 2 as expected as user clicked on 7 to 10 cards deep [7 to 10 cards deep]

image

Brave.Today.ClickCardDepth_3_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 11th card and click on 11th card twice
  4. Click on 12th card twice
  5. Click on 13th card once, skip 14th card click on 15th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 3 as expected as user clicked on 11 to 15 cards deep [11 to 15 cards deep]

image

Brave.Today.ClickCardDepth_4_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 16th card and click on 16th card twice
  4. Skip 17th card and click on 18th card twice
  5. Click on 20th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 4 as expected as user clicked on 16 to 20 cards deep [16 to 20 cards deep]

image

Brave.Today.ClickCardDepth_5_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 21st card and click on 21st card once
  4. Skip 22nd card and click on 23rdth card twice
  5. Click on 24th card once and 26th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 5 as expected as user clicked on 20+ cards deep [20+ cards deep]

image

Brave.Today.ChannelCount

Note: "Top Sources" channel (which is enabled by default) is included in the channel count.

Brave.Today.ChannelCount_0_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 0 as expected [No channels are subscribed]
Example Example
image image
Brave.Today.ChannelCount_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 1 channel
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 1 as expected [One channels is subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 3 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 1 as expected [three channels are subscribed]
Example Example
image image
Brave.Today.ChannelCount_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 4 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 2 as expected [Four channels are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 6 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 2 as expected [Six channels are subscribed]
Example Example
image image
Brave.Today.ChannelCount_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 7 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 3 as expected [Seven channels are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 9 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 3 as expected [Nine channels are subscribed]
Example Example
image image
Brave.Today.ChannelCount_4_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 10 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 4 as expected [Ten channels are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 10 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 4 as expected [Ten channels are subscribed]
Example Example
image image

Brave.Today.PublisherCount

Brave.Today.PublisherCount_0_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 0 as expected [No publishers are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to a publisher
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 0 as expected [One publishers is subscribed]
Example Example
image image
Brave.Today.PublisherCount_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 2 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 1 as expected [Two publishers are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 4 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 1 as expected [Four publishers are subscribed]
Example Example
image image
Brave.Today.PublisherCount_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 5 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 2 as expected [Five publishers are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 7 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 2 as expected [Seven publishers are subscribed]
Example Example
image image
Brave.Today.PublisherCount_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 8 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 3 as expected [Eight publishers are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 10 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 3 as expected [Ten publishers are subscribed]
Example Example
image image
Brave.Today.PublisherCount_4_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 11 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 4 as expected [Eleven publishers are subscribed]
Example Example
image image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 15 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 4 as expected [Fifteen publishers are subscribed]
Example Example
image image

Brave.Today.SidebarFilterUsages

Brave.Today.SidebarFilterUsages_0_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filter
  6. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 0 as expected
    image
Brave.Today.SidebarFilterUsages_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filter
  6. Open 2nd NTP and click on brave news card via "For You" or "Following" filter
  7. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 1 as expected

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filter
  6. Open 2nd NTP and click on brave news card via "For You" or "Following" filter
  7. Open 3rd NTP (3rd Brave news session) and click on brave news card via "For You" or "Following" filter
  8. Open 4th NTP (4th Brave news session) and click on brave news card via "For You" or "Following" filter
  9. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 1 as expected
    image
Brave.Today.SidebarFilterUsages_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filters
  6. Open 2nd NTP and click on each Brave news card via "For You" and "Following" filters
  7. Repeat the step 6 in 3rd, 4th and 5th NTP's, total 5 brave news sessions
  8. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 2 as expected
    image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filters
  6. Open 2nd NTP and click on each Brave news card via "For You" and "Following" filters
  7. Repeat the step 6 in 3rd, 4th, 5th, 6th and 7th NTP's, total 7 brave news sessions
  8. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 2 as expected

image

Brave.Today.SidebarFilterUsages_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filters
  6. Open 2nd NTP and click on each Brave news card via "For You" and "Following" filters
  7. Repeat the step 6 in 3rd, 4th, 5th, 6th, 7th and 8th NTP's, total 8 brave news sessions in 8 NTP's
  8. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 3 as expected

image

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filters
  6. Open 2nd NTP and click on each Brave news card via "For You" and "Following" filters
  7. Repeat the step 6 in 3rd, 4th, 5th, 6th, 7th,8th, 9th and 10th NTP's, total 10 brave news sessions in 10 NTP's
  8. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 3 as expected

image

Brave.Today.SidebarFilterUsages_4_PASSSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Look for the metrics Brave.Today.SidebarFilterUsages
  4. Confirmed that the metrics is not shown
  5. Open a NTP enable brave news and click on each Brave news card via "For You" and "Following" filters
  6. Open 2nd NTP and click on each Brave news card via "For You" and "Following" filters
  7. Repeat the step 6 in 3rd, 4th, 5th, 6th, 7th,8th, 9th, 10th and 11th NTP's, total 11 brave news sessions in 11 NTP's
  8. Reload brave://local-state and ensured that the metrics Brave.Today.SidebarFilterUsages response value is 4 as expected

image

Brave.Today.DirectFeedsTotal.2

Brave.Today.DirectFeedsTotal.2_0_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 0 as expected
Example Example
image image
Brave.Today.DirectFeedsTotal.2_1_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import a external feed
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 1 as expected
Example Example
image image
Brave.Today.DirectFeedsTotal.2_2_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 2 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 2 as expected
Example Example
image image
Brave.Today.DirectFeedsTotal.2_3_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 3 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 3 as expected
Example Example
image image
Brave.Today.DirectFeedsTotal.2_4_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 4 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 4 as expected
Example Example
image image
Brave.Today.DirectFeedsTotal.2_5_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 5 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 5 as expected
Example Example
image image
Brave.Today.DirectFeedsTotal.2_6_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 6 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 6 as expected
Example Example
image image

@brave-builds brave-builds mentioned this issue Feb 27, 2024
Closed
7 tasks
@GeetaSarvadnya GeetaSarvadnya added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Feb 27, 2024
@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Feb 28, 2024 •
edited
Loading

@DJAndries @LorenzoMinto: I need a clarifcation on few questions. Please let me know if you need more details. Thanks!

[Q 1] Brave.Today.ChannelCount - Could you update the Brave.Today.ChannelCount metrics response bucket as below? this works as per the response bucket as below. Thanks!

Brave.Today.ChannelCount

  1. 0
  2. 1-3
  3. 4-6
  4. 7-9
  5. 10+

[Q 2] Brave.Today.WeeklyTotalCardClicks - If I click on 1 or 2 cards I get the metrics response as 0 as expected. But the response bucket range for 1 and 2 are different when I test these cases, click on upto 6 cards I get response 1 but the description say the repsonse 1 should be shown upto 5 cards click, similarly for response bucket 2 the range varies, click upto 13 cards I get the response 2 but the description says 6-10. The response buckets needs to be updated or it's a bug?

  1. 1-2
  2. 3-6
  3. 7-13

[Q 3] Brave.Today.ClickCardDepth - The metrics response value is 0 is shown only when 5-6 cards are clicked, but the response bucket range is 1-3 cards deep in the issues description, similarly 7 -12 cards click yields 1 response value. For remaining response values also the cards click rang varies. Is this a bug or cards deep range needs to be updated?

Brave.Today.ClickCardDepth

  1. 1-3 cards deep
  2. 4 to 6 cards deep
  3. 7 to 10 cards deep
  4. 11 to 15 cards deep
  5. 16 to 20 cards deep
  6. 21+

@GeetaSarvadnya GeetaSarvadnya mentioned this issue Mar 1, 2024
Closed
@DJAndries
Copy link
Collaborator Author

Could you update the Brave.Today.ChannelCount metrics response bucket as below? this works as per the response bucket as below. Thanks!

The original buckets are correct. Please keep in mind that the "Top Sources" channel (which is enabled by default) is included in the channel count.

If I click on 1 or 2 cards I get the metrics response as 0 as expected. But the response bucket range for 1 and 2 are different when I test these cases, click on upto 6 cards I get response 1 but the description say the repsonse 1 should be shown upto 5 cards click, similarly for response bucket 2 the range varies, click upto 13 cards I get the response 2 but the description says 6-10. The response buckets needs to be updated or it's a bug?

The metrics response value is 0 is shown only when 5-6 cards are clicked, but the response bucket range is 1-3 cards deep in the issues description, similarly 7 -12 cards click yields 1 response value. For remaining response values also the cards click rang varies. Is this a bug or cards deep range needs to be updated?

I suspect that this may be due to issues with the click handler, which was fixed in brave/brave-core#22365. I recommend performing testing on Nightly, or clicking on the article title text directly if testing on Beta.

@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Mar 6, 2024 •
edited
Loading

image

@DJAndries The metrics Brave.Today.ClickCardDepth is shown with response value 0 when 5 or more cads are clicked, but the metrics description says the value 0 should be shown for the 1-3 cards deep. This needs to be changed correct?
image

Also, click on 7 cards gives the metrics response value as 1, but the metrics description says the value 1 should be shown for the 4-6 cards deep

image

@LorenzoMinto
Copy link
Member

@GeetaSarvadnya card depth of a click is the ordinal number of the card that has been clicked counting from the top. The card depth metric is an average across the 5 or more clicks. So if I click the first card in the feed for 5 times that should return bucket 0. In practice, we're taking the average of the single clicks' depths. Correct me if I'm wrong @DJAndries

@GeetaSarvadnya GeetaSarvadnya added QA Pass-Win64 and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue labels Mar 7, 2024
@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Mar 15, 2024 •
edited
Loading

Verification PASSED on Samsung Galaxy version 12 running Bravemonoarm64.apk_1.64.104

Brave.Today.WeeklyTotalCardClicks

Brave.Today.WeeklyTotalCardClicks_metrics is not shown_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and don't click on any card
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks isn't shown in local state file
Brave.Today.WeeklyTotalCardClicks_0_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on any one card
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 0 as expected (as user clicked on one card)

Screenshot_20240319_203528

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on two cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 0 as expected (as user clicked on two cards)

Screenshot_20240319_203834

Brave.Today.WeeklyTotalCardClicks_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on three cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 1 as expected (as user clicked on three cards)

Screenshot_20240319_204933

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on five cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 1 as expected (as user clicked on five cards)
Brave.Today.WeeklyTotalCardClicks_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on six cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 2 as expected (as user clicked on six cards)
    Screenshot_20240319_205441

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on nine cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 2 as expected (as user clicked on nine cards)

Case 3:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on ten cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 2 as expected (as user clicked on ten cards)
Brave.Today.WeeklyTotalCardClicks_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 11 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 3 as expected (as user clicked on 11 cards)
    Screenshot_20240319_212734

Case 2:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 15 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 3 as expected (as user clicked on 15 cards)
Brave.Today.WeeklyTotalCardClicks_4_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. View Brave News cards and click on 16 cards
  4. Open brave://local-state
  5. Look for the metrics Brave.Today.WeeklyTotalCardClicks
  6. Confirmed that the metrics Brave.Today.WeeklyTotalCardClicks response value is 3 as expected (as user clicked on 11 cards)

Screenshot_20240319_213405

Brave.Today.DirectFeedsTotal.2

Brave.Today.DirectFeedsTotal.2_0_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 0 as expected

Screenshot_20240318_144219_Brave

Brave.Today.DirectFeedsTotal.2_1_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import a external feed
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 1 as expected
    Screenshot_20240318_145923_Brave
Brave.Today.DirectFeedsTotal.2_2_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 2 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 2 as expected

Screenshot_20240318_150249_Brave

Brave.Today.DirectFeedsTotal.2_3_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 3 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 3 as expected

Screenshot_20240318_150530_Brave

Brave.Today.DirectFeedsTotal.2_4_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 4 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 4 as expected
Example Example
Screenshot_20240318_150905_Brave Screenshot_20240318_150947_Brave
Brave.Today.DirectFeedsTotal.2_5_PASSED
  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 is not shown on the local state file
  4. Open a NTP and enable the Brave News and import 5 external feeds
  5. Reload brave://local-state
  6. Confirmed that the metrics Brave.Today.DirectFeedsTotal.2 value is 5 as expected
Example Example
Screenshot_20240318_151016_Brave Screenshot_20240318_151058_Brave

Brave.Today.ChannelCount

Note: "Top Sources" channel (which is enabled by default) is included in the channel count.

Brave.Today.ChannelCount_0_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 0 as expected [No channels are subscribed]

Screenshot_20240318_180929

Brave.Today.ChannelCount_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 1 channel
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 1 as expected [One channels is subscribed]
Example Example
Screenshot_20240318_181035 Screenshot_20240318_181017

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 3 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 1 as expected [three channels are subscribed]
Example Example
Screenshot_20240318_181116 Screenshot_20240318_181104
Brave.Today.ChannelCount_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 4 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 2 as expected [Four channels are subscribed]
Example Example
Screenshot_20240318_181134 Screenshot_20240318_181148

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 6 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 2 as expected [Six channels are subscribed]
Example Example
Screenshot_20240318_181210 Screenshot_20240318_181227
Brave.Today.ChannelCount_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 7 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 3 as expected [Seven channels are subscribed]
Example Example
Screenshot_20240318_181249 Screenshot_20240318_181305

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 9 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 3 as expected [Nine channels are subscribed]
Example Example
Screenshot_20240318_181347 Screenshot_20240318_181407
Brave.Today.ChannelCount_4_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 10 channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 4 as expected [Ten channels are subscribed]
Example Example
Screenshot_20240318_181430 Screenshot_20240318_181446

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.ChannelCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 10+ channels
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.ChannelCount response value is 4 as expected [Ten channels are subscribed]
Example Example
Screenshot_20240318_181606 Screenshot_20240318_181621

Brave.Today.PublisherCount

Brave.Today.PublisherCount_0_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 0 as expected [No publishers are subscribed]

Screenshot_20240318_205324

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to a publisher
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 0 as expected [One publishers is subscribed]
Example Example
Screenshot_20240318_205440 Screenshot_20240318_205455
Brave.Today.PublisherCount_1_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 2 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 1 as expected [Two publishers are subscribed]

Screenshot_20240318_205534

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 4 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 1 as expected [Four publishers are subscribed]

Screenshot_20240318_205751

Brave.Today.PublisherCount_2_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 5 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 2 as expected [Five publishers are subscribed]
Example Example
Screenshot_20240318_205959 Screenshot_20240318_210016

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 7 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 2 as expected [Seven publishers are subscribed]

Screenshot_20240318_210111

Brave.Today.PublisherCount_3_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 8 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 3 as expected [Eight publishers are subscribed]

Screenshot_20240318_210205

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 10 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 3 as expected [Ten publishers are subscribed]

Screenshot_20240318_210305

Brave.Today.PublisherCount_4_PASSED

Case 1:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 11 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 4 as expected [Eleven publishers are subscribed]
Example Example
Screenshot_20240318_210316 Screenshot_20240318_210351

Case 2:

  1. Clean profile 1.64.x
  2. Open brave://local-state
  3. Confirmed that the metrics Brave.Today.PublisherCount isn't shown as expected
  4. Open a NTP
  5. Enable Brave News and subscribe to 15 publishers
  6. Reload brave://local-state
  7. Confirmed that the metrics Brave.Today.PublisherCount response value is 4 as expected [Fifteen publishers are subscribed]
Example Example
Screenshot_20240318_210335 Screenshot_20240318_210351

Brave.Today.SidebarFilterUsages

Note: This metrics is implemented only for Desktop not for Android as per #35434 (comment). Hence the metrics is not verified on Android

Brave.Today.ClickCardDepth

Note: Minium 5 cards needs to be clicked to see the response value

Brave.Today.ClickCardDepth_0_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Click on 1st card twice
  4. Click on 2nd card twice
  5. Click on 3rd card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 0 as expected as user clicked on 1 to 3 cards deep [1 to 3 cards deep]

Screenshot_20240319_173533

Brave.Today.ClickCardDepth_1_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 4th card and click on 4th card twice
  4. Click on 5th card twice
  5. Click on 6th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 1 as expected as user clicked on 4 to 6 cards deep [4 to 6 cards deep]
    Screenshot_20240319_175152
Brave.Today.ClickCardDepth_2_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 7th card and click on 7th card twice
  4. Click on 8th card twice
  5. Click on 9th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 2 as expected as user clicked on 7 to 10 cards deep [7 to 10 cards deep]

Screenshot_20240319_180218

Brave.Today.ClickCardDepth_3_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 11th card and click on 11th card twice
  4. Click on 12th card twice
  5. Click on 13th card once, skip 14th card click on 15th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 3 as expected as user clicked on 11 to 15 cards deep [11 to 15 cards deep]

Screenshot_20240319_194534

Brave.Today.ClickCardDepth_4_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 16th card and click on 16th card twice
  4. Skip 17th card and click on 18th card twice
  5. Click on 20th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 4 as expected as user clicked on 16 to 20 cards deep [16 to 20 cards deep]

Screenshot_20240319_192710

Brave.Today.ClickCardDepth_5_PASSED
  1. Clean profile 1.64.x
  2. Open a NTP and enable Brave News
  3. Scroll down till 21st card and click on 21st card once
  4. Skip 22nd card and click on 23rdth card twice
  5. Click on 24th card once and 26th card once
  6. Open brave://local-state and look for the metrics Brave.Today.ClickCardDepth
  7. Confirmed that the metrics respone value is 5 as expected as user clicked on 20+ cards deep [20+ cards deep]

Screenshot_20240319_194949

@GeetaSarvadnya GeetaSarvadnya added the QA/In-Progress Indicates that QA is currently in progress for that particular issue label Mar 15, 2024
@GeetaSarvadnya GeetaSarvadnya mentioned this issue Mar 18, 2024
Open
@GeetaSarvadnya
Copy link

GeetaSarvadnya commented Mar 19, 2024 •
edited
Loading

@DJAndries: Few metrics are not shown on the Android, please see the questions below:

Q1. The metrics Brave.Today.SidebarFilterUsages isn't shown on Android. "For you" and "Following" filters don't show up brave news cards like Desktop. There is no "For you" filter available on Android. I have verified the metrics by clicking on the Brave news cards on NTP but metrics isn't shown on the brave://local-state file.

Destop Android
image Screenshot_20240319_134948

@GeetaSarvadnya GeetaSarvadnya added QA Pass - iPhone QA Pass - Android ARM and removed QA/In-Progress Indicates that QA is currently in progress for that particular issue QA Pass - iPhone labels Mar 19, 2024
@DJAndries
Copy link
Collaborator Author

The metrics Brave.Today.SidebarFilterUsages isn't shown on Android

@GeetaSarvadnya this is expected, the metric is only for desktop

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@DJAndries DJAndries

Labels
feature/new-metric Proposing a new metric for Privacy Preserving Product Analytics (P3A) features/P3A OS/Android Fixes related to Android browser functionality OS/Desktop QA Pass - Android ARM QA Pass-Win64 QA/Yes release-notes/exclude
Projects
None yet
Milestone
1.64.x - Release
Development

Successfully merging a pull request may close this issue.

Add additional News P3A metrics for News V2 brave/brave-core
4 participants
@DJAndries @LorenzoMinto @GeetaSarvadnya @brave-builds