Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WiP] feat: backend for thumnail computation + caching + endpoints #7716

Closed
wants to merge 1 commit into from

Conversation

mistercrunch
Copy link
Member

@mistercrunch mistercrunch commented Jun 17, 2019

This PR is the backend subset of WiP PR #6601. It includes:

  • refactor of the Selenium abstractions used for email schedules,
    generalized to also work with thumbs
  • new dependency on PIL, the common way of doing image processing in
    python, we use it to resize selenium screenshots into thumbs
  • CLI utilities to compute-thumbnails
  • the addition of an extra caching backend for thumbnail, in most cases
    we assume it should point to the same backend as the one used for
    chart JSON caching
  • 2 new endpoints to retrieve dashboard and chart thumbs

CATEGORY

Choose one### SUMMARY

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

TODO / extra thoughts

  • The security on viewing thumbnails should be the same as viewing the chart/dashboard itself
  • We should perf/load test viewing a pageful of thumbnail. Assuming ~50 thumbs are displayed on a card list view, how does the metadata database react? It can represent a fair amount of database lookups depending on the way that perms are setup (if the user is Gamma, and there are lots of dashboards on the system). We should make sure that the worst case is ok.
  • Caching for thumbs can be lax (doesn't have to be done daily for instance), but it's different for the image service (say for Slack or email scheduled delivery). Caching for thumb is currently forced on save events (async, and that's the way we'd tell users to get a new fresh thumb), or we could add a feature to force on the endpoint. There's also the CLI. For caching for other purposes, I think that the caching policy for charts should apply for the image the same way as it does for the JSON caching (same logic/policy).

This PR is the backend subset of WiP PR apache#6601. It includes

* refactor of the Selenium abstractions used for email schedules,
  generalized to also work with thumbs
* new dependency on PIL, the common way of doing image processing in
  python, we use it to resize selenium screenshots into thumbs
* CLI utilities to compute-thumbnails
* the addition of an extra caching backend for thumbnail, in most cases
  we assume it should point to the same backend as the one used for
  chart JSON caching
* 2 new endpoints to retrieve dashboard and chart thumbs
@stale
Copy link

stale bot commented Oct 8, 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label Oct 8, 2019
@stale stale bot closed this Oct 16, 2019
@mistercrunch mistercrunch reopened this Jan 6, 2020
@stale stale bot removed the inactive Inactive for >= 30 days label Jan 6, 2020
@stale
Copy link

stale bot commented Mar 9, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label Mar 9, 2020
@stale stale bot closed this Mar 16, 2020
@dpgaspar dpgaspar removed the inactive Inactive for >= 30 days label Mar 16, 2020
@dpgaspar dpgaspar reopened this Mar 16, 2020
@stale
Copy link

stale bot commented May 15, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions. For admin, please label this issue .pinned to prevent stale bot from closing the issue.

@stale stale bot added the inactive Inactive for >= 30 days label May 15, 2020
@stale stale bot closed this May 22, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
inactive Inactive for >= 30 days size/XL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants