Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Uniform message distribution among partitions in tests #4920

Closed
wants to merge 2 commits into from
Closed

Uniform message distribution among partitions in tests #4920

wants to merge 2 commits into from

Commits on May 25, 2022

  1. tests: enable round robin distribudion of records 

    when using kafka-producer-perf-test.sh via the KafkaCliTools utility,
provide an option to force round robin distribution of records. In
modern versions of kafka the producer by default uses sticky assignment
of records where the producer sticks to a partition until batch size is
reached or linger.ms is up. The next partition is sent messages when the
batch is full. This provides better performance in real world uses but
in some test cases we rely on messages being distributed equally among
partitions for testing and assertions, eg when triggering shadow
indexing uploads.

The option round_robin_partition enables to use the older partitioner,
which is potentially slower but ensures that each partition gets a
roughly equal amount of messages.
    @abhijat
    abhijat committed May 25, 2022
    Configuration menu
    Copy the full SHA
    14bf51e View commit details
    Browse the repository at this point in the history

  2. tests: missionpartition uses roundrobin scheme

    @abhijat
    abhijat committed May 25, 2022
    Configuration menu
    Copy the full SHA
    a43196f View commit details
    Browse the repository at this point in the history