[FEATURE] : Search Phase Results Processor in Search Pipeline

[navneet1v]

Description

For Normalization and Score Combination feature, we need a way to do the Normalization of scores received for different sub-queries from different shards at Coordinator node, before we can start combing the scores. This needs to be done after the Query phase is completed and before Fetch Phase is started in a _search api call.

Solution

The solution I am proposing is to extend the Search Pipeline Processor to create a new Processor Interface called as Search Phase Processors that can run between Phases of Search(There are many phases Search like DFS, Query, Fetch, Expand etc).

As first use-case we will create a new SearchPhase Processor that will run between Query and Fetch phase, which do the Normalization of Scores and combine the Scores after normalization.

Alternatives

The alternative approaches are defined in the Section: Obtaining Relevant Information for Normalization and score Combination in #126

Task

• Create RFC for New Processor Interface : [RFC]: Search Phase Results Processor  #152
• Implementation of Search Phase Processor Interface
• Implementation of Normalization Processor
• Testing

Reference Links

1. [RFC] High Level Approach and Design For Normalization and Score Combination #126
2. [RFC] Search pipelines search-processor#80

[navneet1v]

PR: opensearch-project/OpenSearch#7283

[navneet1v]

Resolving this issue as this feature is launched in 2.10 version of OpenSearch