Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(metastore-cache): add codec support #24586

Merged
merged 2 commits into from
Jul 4, 2023

Conversation

villebro
Copy link
Member

@villebro villebro commented Jul 4, 2023

SUMMARY

This PR adds the option to use a custom codec for serializing/deserializing values in the SupersetMetastoreCache. In addition, the default Explore form data and Dashboard filter state cache config is changed to use the JSON key-value codec. This will not affect deployments that are using Redis or another cache. In addition, we're already emitting a warning when using it in prod, and since it will only cause a problem if a user happens to refresh their browser right after the state has last been updated, there is minimal risk that this will cause issues for users.

Tests are added to ensure that the cache works as expected with both the JSON and Pickle codecs.

TESTING INSTRUCTIONS

ADDITIONAL INFORMATION

  • Has associated issue:
  • Required feature flags:
  • Changes UI
  • Includes DB Migration (follow approval process in SIP-59)
    • Migration is atomic, supports rollback & is backwards-compatible
    • Confirm DB migration upgrade and downgrade tested
    • Runtime estimates and downtime expectations provided
  • Introduces new feature or API
  • Removes existing feature or API

@codecov
Copy link

codecov bot commented Jul 4, 2023

Codecov Report

Merging #24586 (7d08c7e) into master (226c7f8) will decrease coverage by 0.02%.
The diff coverage is 90.90%.

❗ Current head 7d08c7e differs from pull request most recent head af1ce10. Consider uploading reports for the commit af1ce10 to get more accurate results

@@            Coverage Diff             @@
##           master   #24586      +/-   ##
==========================================
- Coverage   69.08%   69.06%   -0.02%     
==========================================
  Files        1906     1906              
  Lines       74168    74175       +7     
  Branches     8164     8164              
==========================================
- Hits        51239    51232       -7     
- Misses      20807    20821      +14     
  Partials     2122     2122              
Flag Coverage Δ
hive 53.93% <9.09%> (-0.01%) ⬇️
mysql ?
postgres 79.47% <63.63%> (-0.02%) ⬇️
presto 53.82% <9.09%> (-0.01%) ⬇️
python 83.44% <90.90%> (-0.04%) ⬇️
sqlite 78.04% <63.63%> (-0.01%) ⬇️
unit 54.69% <90.90%> (+<0.01%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Impacted Files Coverage Δ
superset/extensions/metastore_cache.py 96.61% <90.00%> (-1.51%) ⬇️
superset/config.py 92.06% <100.00%> (+0.02%) ⬆️

... and 7 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@villebro villebro merged commit a4880ca into apache:master Jul 4, 2023
29 checks passed
@villebro villebro deleted the villebro/metastore-codec branch July 4, 2023 13:13
@dpgaspar dpgaspar added the 2.1.1 label Jul 4, 2023
@michael-s-molina michael-s-molina added the v3.0 Label added by the release manager to track PRs to be included in the 3.0 branch label Jul 5, 2023
@eschutho eschutho added the v2.1 label Jul 8, 2023
michael-s-molina pushed a commit that referenced this pull request Jul 26, 2023
@eschutho eschutho mentioned this pull request Aug 12, 2023
9 tasks
@eschutho
Copy link
Member

@michael-s-molina I see that this PR is flagged for 3.0. You'll probably also want to bring in #24970

@michael-s-molina
Copy link
Member

@michael-s-molina I see that this PR is flagged for 3.0. You'll probably also want to bring in #24970

Thanks @eschutho. Will do.

@mistercrunch mistercrunch added 🍒 3.0.1 🍒 3.0.2 🍒 3.0.3 🍒 3.0.4 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 3.1.0 labels Mar 8, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
2.1.1 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L v2.1 v3.0 Label added by the release manager to track PRs to be included in the 3.0 branch 🍒 2.1.1 🍒 2.1.2 🍒 2.1.3 🍒 3.0.0 🍒 3.0.1 🍒 3.0.2 🍒 3.0.3 🍒 3.0.4 🚢 3.1.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants