[Discover] Display Cache Time and Clear Cache Button #8214

sejli · 2024-09-18T02:30:47Z

Description

Stores last cache updated time in session storage. Last updated time refers to the last time that the cache was updated across all datasets (overall cache rather than specific types).
Adds display for last cache update time in Advanced Dataset Selector
Adds refresh cache button which clears session storage

Still early implementation, open to suggestions

Issues Resolved

Screenshot

Empty Cache State:

Cache has been populated:

Testing the changes

Changelog

feat: Add last updated time and cache refresh button to Discover Advanced Dataset Selector

Check List

All tests pass
- yarn test:jest
- yarn test:jest_integration
New functionality includes testing.
New functionality has been documented.
Update CHANGELOG.md
Commits are signed per the DCO using --signoff

Signed-off-by: Sean Li <lnse@amazon.com>

codecov · 2024-09-18T02:46:05Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 64.05%. Comparing base (b826df8) to head (511ee18).
Report is 19 commits behind head on main.

Additional details and impacted files

@@           Coverage Diff           @@
##             main    #8214   +/-   ##
=======================================
  Coverage   64.05%   64.05%           
=======================================
  Files        3741     3741           
  Lines       88629    88688   +59     
  Branches    13801    13818   +17     
=======================================
+ Hits        56771    56813   +42     
- Misses      31260    31263    +3     
- Partials      598      612   +14

Flag	Coverage Δ
Linux_1	`30.04% <0.00%> (-0.02%)`	⬇️
Linux_2	`58.85% <0.00%> (+0.02%)`	⬆️
Linux_3	`40.34% <100.00%> (-0.04%)`	⬇️
Linux_4	`31.43% <0.00%> (-0.03%)`	⬇️
Windows_1	`30.06% <0.00%> (-0.02%)`	⬇️
Windows_2	`58.80% <0.00%> (+0.02%)`	⬆️
Windows_3	`40.35% <100.00%> (-0.03%)`	⬇️
Windows_4	`31.43% <0.00%> (-0.03%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

virajsanghvi

should we add tests for new methods + snapshot test to bump up patch cov?

virajsanghvi · 2024-09-18T22:09:43Z

src/plugins/data/public/query/query_string/dataset_service/dataset_service.ts

+    this.sessionStorage.clear();
+  }
+
+  public getLastCacheTime(): string | undefined {


want to convert to a Date given its passed a date? otherwise consumers just need to handle this

yeah we should make sure to do this.

@sejli
we should probably include the day and should also pulling from UI Settings to get the format of the date

virajsanghvi · 2024-09-18T22:10:49Z

src/plugins/data/public/ui/dataset_selector/dataset_explorer.tsx

@@ -92,6 +92,29 @@ export const DatasetExplorer = ({
              </EuiLink>
            </p>
          </EuiText>
+          {queryString.getDatasetService().getLastCacheTime() && (
+            <EuiText>


src/plugins/data/public/query/query_string/dataset_service/dataset_service.ts

kavilla · 2024-09-18T22:42:55Z

a good catch all but did we want to consider making it more granular?

for example:

if i press the refresh on Data then it clears out the full cache. If i press refresh on Database then it will refresh all the Databases and Tables and not the Connections and Data Sources?

This way to avoid destroying the cache if only one source is out of date?

If so then we can consider extending this function https://github.com/opensearch-project/OpenSearch-Dashboards/blob/main/src/plugins/data/public/query/query_string/dataset_service/dataset_service.ts#L91 to have a new boolean as a param like refresh = false but if the user presses the refresh button it will clear the cache related to the data structure passed in and it's children and refetch.

To get the date we could actually shove that in the data structure meta for when it was first fetched.

kavilla

+1 to @virajsanghvi comments. nothing blocking

Signed-off-by: Sean Li <lnse@amazon.com>

AMoo-Miki · 2024-09-19T17:27:20Z

src/plugins/data/public/query/query_string/dataset_service/dataset_service.ts

@@ -138,6 +138,7 @@ export class DatasetService {
  }

  private cacheDataStructure(dataType: string, dataStructure: DataStructure) {
+    this.setLastCacheTime(new Date());


Since this is getting set in the sessionStorage, the Date object gets converted to a string. To consume this, we will need to parse it and what not. To programmatically use date, it is best to stick with the unix timestamp of the date which is number and can be easily parsed and formatted.

Suggested change

this.setLastCacheTime(new Date());

this.setLastCacheTime(Date.now());

AMoo-Miki · 2024-09-19T17:36:27Z

src/plugins/data/public/query/query_string/dataset_service/dataset_service.ts

+  }
+
+  public getLastCacheTime(): string | undefined {
+    return this.sessionStorage.get('lastCacheTime');


Assuming you do use Date.now() this line changes to

Suggested change

return this.sessionStorage.get('lastCacheTime');

return Number(this.sessionStorage.get('lastCacheTime')) || undefined;

The reason I propose Number and not parseInt is that parseInt will try to parse the numeric part of a mixed alphanumeric string which we don't want because the value should be fully numeric, and if it is not, we need to fail to parse.

The reason for || undefined is that we should return undefined for all invalid values: NaN and 0.

Signed-off-by: Sean Li <lnse@amazon.com>

kavilla · 2024-09-20T08:01:10Z

src/plugins/data/public/query/query_string/dataset_service/dataset_service.ts

+    this.sessionStorage.clear();
+  }
+
+  public getLastCacheTime(): number | undefined {


nit i think there is precedent with updatedAt

kavilla

agreeing with @virajsanghvi with some snapshots will be awesome.

one nit with the naming of the field

one fast follow we should do is pull from ui settings for the date format

initial commit for cache time and clearing cache

ce5b21e

Signed-off-by: Sean Li <lnse@amazon.com>

github-actions bot added the seasoned-contributor label Sep 18, 2024

Changeset file for PR opensearch-project#8214 created/updated

3686825

sejli added backport 2.x discover for discover reinvent 2.17.1 labels Sep 18, 2024

virajsanghvi reviewed Sep 18, 2024

View reviewed changes

kavilla reviewed Sep 18, 2024

View reviewed changes

updating UI to address some comments

956921e

Signed-off-by: Sean Li <lnse@amazon.com>

AMoo-Miki reviewed Sep 19, 2024

View reviewed changes

addressing comments, adding tests

511ee18

Signed-off-by: Sean Li <lnse@amazon.com>

kavilla reviewed Sep 20, 2024

View reviewed changes

kavilla approved these changes Sep 21, 2024

View reviewed changes

kavilla added the backport 2.17 label Sep 21, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Discover] Display Cache Time and Clear Cache Button #8214

[Discover] Display Cache Time and Clear Cache Button #8214

sejli commented Sep 18, 2024 •

edited

Loading

codecov bot commented Sep 18, 2024 •

edited

Loading

virajsanghvi left a comment

virajsanghvi Sep 18, 2024

kavilla Sep 18, 2024

virajsanghvi Sep 18, 2024

kavilla commented Sep 18, 2024 •

edited

Loading

kavilla left a comment

AMoo-Miki Sep 19, 2024

AMoo-Miki Sep 19, 2024

kavilla Sep 20, 2024

kavilla left a comment

	this.setLastCacheTime(new Date());
	this.setLastCacheTime(Date.now());

	return this.sessionStorage.get('lastCacheTime');
	return Number(this.sessionStorage.get('lastCacheTime')) \|\| undefined;

[Discover] Display Cache Time and Clear Cache Button #8214

Are you sure you want to change the base?

[Discover] Display Cache Time and Clear Cache Button #8214

Conversation

sejli commented Sep 18, 2024 • edited Loading

Description

Issues Resolved

Screenshot

Testing the changes

Changelog

Check List

codecov bot commented Sep 18, 2024 • edited Loading

Codecov Report

virajsanghvi left a comment

Choose a reason for hiding this comment

virajsanghvi Sep 18, 2024

Choose a reason for hiding this comment

kavilla Sep 18, 2024

Choose a reason for hiding this comment

virajsanghvi Sep 18, 2024

Choose a reason for hiding this comment

kavilla commented Sep 18, 2024 • edited Loading

kavilla left a comment

Choose a reason for hiding this comment

AMoo-Miki Sep 19, 2024

Choose a reason for hiding this comment

AMoo-Miki Sep 19, 2024

Choose a reason for hiding this comment

kavilla Sep 20, 2024

Choose a reason for hiding this comment

kavilla left a comment

Choose a reason for hiding this comment

sejli commented Sep 18, 2024 •

edited

Loading

codecov bot commented Sep 18, 2024 •

edited

Loading

kavilla commented Sep 18, 2024 •

edited

Loading