Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Put consumers in their own scheduling group #10398

Merged

Commits on Apr 26, 2023

  1. Make _scheduling_groups public in application

    By analogy with smp_groups which is already public we need to
    access this outside of application.cc to pass the right scheduling
    groups through to instantiations of services in our redpanda fixture.
    travisdowns committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    d505693 View commit details
    Browse the repository at this point in the history
  2. Add fetch scheduling group

    This scheduling group is used for consumer fetch processing. We assign
    it the same priority as the default group (where most other kafka
    handling takes place), but by putting it into its own group we prevent
    non-fetch requests from being significantly delayed when fetch requests
    use all the CPU.
    
    Issue redpanda-data/core-internal#435
    travisdowns committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    d133843 View commit details
    Browse the repository at this point in the history
  3. Use fetch scheduling group to handle fetches

    Add the with_scheduling_group call to switch to the kafka scheduling
    group in the fetch handling flow. See prior change for context.
    
    Issue redpanda-data/core-internal/redpanda-data#435.
    travisdowns committed Apr 26, 2023
    Configuration menu
    Copy the full SHA
    9a93a9c View commit details
    Browse the repository at this point in the history

Commits on Apr 27, 2023

  1. Make the fetch scheduling group configurable

    Introduce a config tunable which would let users set the scheduling
    group to use for kafka fetch request back to the default group.
    
    Also useful for quick A/B testing of the perf effect of the change.
    travisdowns committed Apr 27, 2023
    Configuration menu
    Copy the full SHA
    bc72f5e View commit details
    Browse the repository at this point in the history