Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feature: TopN window operator #16394

Open
sundy-li opened this issue Sep 5, 2024 · 1 comment
Open

Feature: TopN window operator #16394

sundy-li opened this issue Sep 5, 2024 · 1 comment
Assignees
Labels
A-query Area: databend query C-feature Category: feature

Comments

@sundy-li
Copy link
Member

sundy-li commented Sep 5, 2024

Summary

Description for this feature.

Example Query:

select * from (select number, rank() over( partition by number % 3   order by number ) c  from   numbers(1000000) ) where c < 3;

🐳 :) explain  pipeline select * from (select number, rank() over( partition by number % 3   order by number ) c  from   numbers(1000000) ) where c < 3;
-[ EXPLAIN ]-----------------------------------
CompoundBlockOperator(Project) × 16
  TransformFilter × 16
    Transform Window × 16
      TransformWindowPartitionSort × 16
        TransformWindowPartitionSpillReader × 16
          Merge to Resize × 16
            Merge to TransformWindowPartitionBucket × 1
              TransformWindowPartitionSpillWriter × 16
                TransformWindowPartitionScatter × 16
                  CompoundBlockOperator(Map) × 16
                    NumbersSourceTransform × 16

11 rows explain in 0.044 sec. Processed 0 rows, 0B (0 row/s, 0B/s)

Since we only need Top 3 rank results partitioned by number % 3 and order by number, we can introduce a TopK window operator after CompoundBlockOperator and filter the results in advance.

This optimization also works for Q44 of TPCDS.

@sundy-li sundy-li added the C-feature Category: feature label Sep 5, 2024
@sundy-li
Copy link
Member Author

sundy-li commented Sep 5, 2024

cc @Dousir9 want this optimization.

assigned to @forsaken628

@sundy-li sundy-li changed the title Feature: TopK window operator Feature: TopN window operator Sep 5, 2024
@dosubot dosubot bot added the A-query Area: databend query label Sep 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-query Area: databend query C-feature Category: feature
Projects
None yet
Development

No branches or pull requests

2 participants