Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CT-1178] Only emit finalized subaccount update #2181

Merged
merged 1 commit into from
Sep 9, 2024

Conversation

teddyding
Copy link
Contributor

@teddyding teddyding commented Aug 30, 2024

Changelist

Context

Test Plan

Ran client against dev5

Author/Reviewer Checklist

  • If this PR has changes that result in a different app state given the same prior state and transaction list, manually add the state-breaking label.
  • If the PR has breaking postgres changes to the indexer add the indexer-postgres-breaking label.
  • If this PR isn't state-breaking but has changes that modify behavior in PrepareProposal or ProcessProposal, manually add the label proposal-breaking.
  • If this PR is one of many that implement a specific feature, manually label them all feature:[feature-name].
  • If you wish to for mergify-bot to automatically create a PR to backport your change to a release branch, manually add the label backport/[branch-name].
  • Manually add any of the following labels: refactor, chore, bug.

Summary by CodeRabbit

Summary by CodeRabbit

  • New Features

    • Enhanced GitHub Actions workflows to trigger on branches matching the 'td/*' pattern.
    • Introduced a method for retrieving subaccount snapshots for stream initialization.
  • Improvements

    • Refined logic for sending finalized subaccount updates, ensuring clarity in operations.
    • Streamlined subscription management and initialization processes for better efficiency.
  • Bug Fixes

    • Removed outdated logic for fetching subaccount IDs, simplifying operations processing.
  • Documentation

    • Updated comments to clarify the new logic and conditions for emitting subaccount updates.

Copy link

linear bot commented Aug 30, 2024

Copy link
Contributor

coderabbitai bot commented Aug 30, 2024

Walkthrough

The changes involve updates across multiple files, including the addition of new branch triggers in GitHub Actions workflows and modifications to method signatures in various interfaces and implementations. Enhancements have been made to the handling of subaccount updates and snapshots, particularly focusing on ensuring that updates are sent in a finalized state. These updates aim to improve overall functionality and streamline processes within the system.

Changes

Files Change Summary
.github/workflows/protocol-build-and-push-snapshot.yml Added trigger for branches matching pattern td/*.
.github/workflows/protocol-build-and-push.yml Added trigger for branches matching pattern td/*.
protocol/lib/metrics/metric_keys.go Added constant GrpcSendFinalizedSubaccountUpdatesLatency.
protocol/mocks/ClobKeeper.go Updated InitializeNewStreams method signature to include subaccountSnapshots parameter.
protocol/streaming/full_node_streaming_manager.go Replaced sync.Once with atomic.Bool for initialization; renamed SendSubaccountUpdates to SendFinalizedSubaccountUpdates. Added GetSubaccountSnapshotsForInitStreams.
protocol/streaming/noop_streaming_manager.go Renamed SendSubaccountUpdates to SendFinalizedSubaccountUpdates; updated InitializeNewStreams signature. Added GetSubaccountSnapshotsForInitStreams.
protocol/streaming/types/interface.go Updated InitializeNewStreams and SendSubaccountUpdates signatures; added GetSubaccountSnapshotsForInitStreams.
protocol/x/clob/abci.go Introduced subaccountSnapshots variable; updated InitializeNewStreams call.
protocol/x/clob/keeper/keeper.go Added GetSubaccountSnapshotsForInitStreams method; updated InitializeNewStreams signature.
protocol/x/clob/types/clob_keeper.go Updated InitializeNewStreams method signature to include subaccountSnapshots parameter.
protocol/x/clob/types/expected_keepers.go Renamed SendSubaccountUpdates to SendFinalizedSubaccountUpdates.
protocol/x/subaccounts/keeper/subaccount.go Renamed SendSubaccountUpdates to SendFinalizedSubaccountUpdates; added context checks.
protocol/x/subaccounts/types/types.go Renamed SendSubaccountUpdates to SendFinalizedSubaccountUpdates.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant Workflow
    participant StreamingManager
    participant Keeper

    User->>Workflow: Trigger on branch td/*
    Workflow->>StreamingManager: InitializeNewStreams(subaccountSnapshots)
    StreamingManager->>Keeper: GetSubaccountSnapshotsForInitStreams()
    Keeper-->>StreamingManager: Return subaccount snapshots
    StreamingManager-->>Workflow: Complete initialization
Loading

🐇 In the meadow, I hop with glee,
New updates are here, oh what a spree!
With snapshots and streams, we dance and play,
Finalized updates brighten the day!
So let’s celebrate, with joy and cheer,
The code is now better, oh dear, oh dear! 🐇✨


Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@teddyding teddyding marked this pull request as ready for review September 6, 2024 15:29
@teddyding teddyding requested a review from a team as a code owner September 6, 2024 15:29
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Outside diff range, codebase verification and nitpick comments (1)
protocol/x/clob/abci.go (1)

Line range hint 127-252: Approved: Changes to PrepareCheckState function.

The modifications in the PrepareCheckState function, including the addition of subaccountSnapshots and its conditional initialization, are aligned with the PR's objectives to handle subaccount updates more effectively. The function's updated signature and the integration of the new variable into the stream initialization process are well-implemented.

However, it is crucial to ensure that these changes are covered by unit tests to verify their functionality and integration with existing systems.

Would you like assistance in writing the unit tests for these changes, or should I open a GitHub issue to track this task?

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between ffde811 and d1d847d.

Files selected for processing (14)
  • .github/workflows/protocol-build-and-push-snapshot.yml (1 hunks)
  • .github/workflows/protocol-build-and-push.yml (1 hunks)
  • protocol/lib/metrics/metric_keys.go (1 hunks)
  • protocol/mocks/ClobKeeper.go (2 hunks)
  • protocol/streaming/full_node_streaming_manager.go (7 hunks)
  • protocol/streaming/noop_streaming_manager.go (2 hunks)
  • protocol/streaming/types/interface.go (2 hunks)
  • protocol/x/clob/abci.go (3 hunks)
  • protocol/x/clob/keeper/keeper.go (3 hunks)
  • protocol/x/clob/keeper/process_operations.go (3 hunks)
  • protocol/x/clob/types/clob_keeper.go (1 hunks)
  • protocol/x/clob/types/expected_keepers.go (1 hunks)
  • protocol/x/subaccounts/keeper/subaccount.go (3 hunks)
  • protocol/x/subaccounts/types/types.go (1 hunks)
Additional comments not posted (21)
protocol/streaming/types/interface.go (3)

25-25: Approved change but verify integration.

The change from a function to a direct map for subaccount snapshots in InitializeNewStreams should improve efficiency. However, ensure that all parts of the system that interact with this method are updated to handle this new method signature.

Run the following script to verify the integration:

Verification successful

Integration verified successfully.

The change to use a direct map for subaccount snapshots in InitializeNewStreams has been consistently applied across the codebase. All relevant parts of the system appear to be updated to handle the new method signature.

  • Files updated include: protocol/x/clob/abci.go, protocol/x/clob/keeper/keeper.go, protocol/x/clob/types/clob_keeper.go, protocol/streaming/noop_streaming_manager.go, protocol/streaming/types/interface.go, protocol/mocks/ClobKeeper.go, and protocol/streaming/full_node_streaming_manager.go.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that all parts of the system that interact with `InitializeNewStreams` are updated.

# Test: Search for the method usage. Expect: Only occurrences of the new signature.
rg --type go -A 5 $'InitializeNewStreams'

Length of output: 3859


29-31: Approved addition but verify implementation.

The addition of GetSubaccountSnapshotsForInitStreams is a positive step towards a more structured handling of subaccount data. Ensure that this method is properly implemented across all classes that implement this interface.

Run the following script to verify the implementation:

Verification successful

Verification successful: Method implemented across relevant classes.

The method GetSubaccountSnapshotsForInitStreams is implemented in multiple classes, ensuring its integration into the system. It is actively used, as evidenced by its invocation in protocol/x/clob/abci.go. No further action is required.

  • Implementations found in:
    • protocol/x/clob/keeper/keeper.go
    • protocol/streaming/noop_streaming_manager.go
    • protocol/streaming/full_node_streaming_manager.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that `GetSubaccountSnapshotsForInitStreams` is implemented across all classes that use this interface.

# Test: Search for the method implementation. Expect: Implementations in all classes that use the interface.
rg --type go -A 5 $'GetSubaccountSnapshotsForInitStreams'

Length of output: 2838


48-48: Approved renaming but verify usage.

Renaming SendSubaccountUpdates to SendFinalizedSubaccountUpdates aligns with the PR's objective to ensure only finalized updates are emitted. Verify that this change is consistently applied across the system.

Run the following script to verify the usage:

Verification successful

Renaming consistently applied across the codebase.

The method SendFinalizedSubaccountUpdates has been consistently renamed and applied throughout the codebase, aligning with the PR's objective to ensure only finalized updates are emitted. No further action is required.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that `SendFinalizedSubaccountUpdates` is used consistently across the system.

# Test: Search for the method usage. Expect: Only occurrences of the new name.
rg --type go -A 5 $'SendFinalizedSubaccountUpdates'

Length of output: 5341

protocol/x/clob/types/expected_keepers.go (1)

88-88: Method Renaming Approved

The renaming of SendSubaccountUpdates to SendFinalizedSubaccountUpdates is clear and aligns with the PR's objectives to ensure only finalized updates are emitted. This change should help in reducing unnecessary emissions and improving system efficiency.

Run the following script to verify that all references to the old method name are updated:

Verification successful

Method Renaming Verified

The renaming of SendSubaccountUpdates to SendFinalizedSubaccountUpdates has been successfully implemented across the codebase. The new method name is present in multiple files, confirming that all references have been updated. This change aligns with the PR's objectives to ensure only finalized updates are emitted, improving system efficiency.

  • Verified occurrences in:
    • protocol/x/subaccounts/types/types.go
    • protocol/x/subaccounts/keeper/subaccount.go
    • protocol/x/clob/types/expected_keepers.go
    • protocol/streaming/noop_streaming_manager.go
    • protocol/streaming/full_node_streaming_manager.go
    • protocol/streaming/types/interface.go
    • protocol/lib/metrics/metric_keys.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that all references to `SendSubaccountUpdates` are updated to `SendFinalizedSubaccountUpdates`.

# Test: Search for the old function name. Expect: No occurrences.
rg --type go -A 5 $'SendSubaccountUpdates'

Length of output: 41


Script:

#!/bin/bash
# Description: Verify that the new function name `SendFinalizedSubaccountUpdates` is present in the codebase.

# Test: Search for the new function name. Expect: At least one occurrence.
rg --type go -A 5 $'SendFinalizedSubaccountUpdates'

Length of output: 5341

protocol/x/clob/types/clob_keeper.go (1)

140-143: Method Parameter Addition Approved

The addition of the subaccountSnapshots parameter to InitializeNewStreams enhances the method's functionality by allowing it to process subaccount snapshots alongside the context. This change is likely necessary for handling more complex operations related to stream initialization.

Run the following script to verify that all implementations of the ClobKeeper interface are updated to accommodate the new parameter:

Verification successful

Verification Successful: Method Parameter Addition

All implementations and interface definitions of the InitializeNewStreams method have been updated to include the subaccountSnapshots parameter, confirming that the change has been consistently applied across the codebase.

  • protocol/x/clob/abci.go: Method call includes the new parameter.
  • protocol/x/clob/types/clob_keeper.go: Method signature includes the new parameter.
  • protocol/x/clob/keeper/keeper.go: Method implementation includes the new parameter.
  • protocol/mocks/ClobKeeper.go: Mock implementation includes the new parameter.
  • protocol/streaming/noop_streaming_manager.go: Method implementation includes the new parameter.
  • protocol/streaming/full_node_streaming_manager.go: Method implementation includes the new parameter.
  • protocol/streaming/types/interface.go: Interface definition includes the new parameter.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that all implementations of the `ClobKeeper` interface accommodate the new parameter in `InitializeNewStreams`.

# Test: Search for implementations of the `InitializeNewStreams` method. Expect: All implementations include the new parameter.
rg --type go -A 5 $'InitializeNewStreams'

Length of output: 3859

protocol/lib/metrics/metric_keys.go (1)

74-74: New Metric Key Addition Approved

The addition of the GrpcSendFinalizedSubaccountUpdatesLatency metric key is a positive change that enhances the granularity of metrics available for monitoring gRPC interactions, particularly in the context of subaccount updates. This addition aligns with the system's focus on finalized updates.

Run the following script to verify that the new metric key is being used in the codebase:

Verification successful

Verification Successful: Metric Key Usage Confirmed

The new metric key GrpcSendFinalizedSubaccountUpdatesLatency is being utilized in the codebase, specifically in the protocol/streaming/full_node_streaming_manager.go file. This confirms its integration into the system for monitoring purposes.

  • File: protocol/streaming/full_node_streaming_manager.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that the new metric key `GrpcSendFinalizedSubaccountUpdatesLatency` is being used in the codebase.

# Test: Search for the new metric key. Expect: At least one occurrence.
rg --type go -A 5 $'GrpcSendFinalizedSubaccountUpdatesLatency'

Length of output: 1275

.github/workflows/protocol-build-and-push.yml (1)

9-9: Approved: New trigger pattern added.

The addition of the 'td/*' trigger pattern is approved as it allows for more flexible workflow execution. However, it's important to monitor the impact this might have on CI resources, considering the potential increase in workflow executions.

Consider monitoring the CI system for any significant changes in resource utilization or queuing times.

.github/workflows/protocol-build-and-push-snapshot.yml (1)

9-9: Approved: New trigger pattern added.

The addition of the 'td/*' trigger pattern is approved as it allows for more flexible workflow execution. However, it's important to monitor the impact this might have on CI resources, considering the potential increase in workflow executions.

Consider monitoring the CI system for any significant changes in resource utilization or queuing times.

protocol/x/clob/keeper/keeper.go (2)

255-272: Well-implemented function for fetching subaccount snapshots.

The use of a callback function to fetch subaccount updates from subaccountsKeeper is a flexible and clean approach, allowing for easy modifications if the fetching logic needs to change in the future.


Line range hint 276-289: Efficient use of pre-fetched subaccount snapshots.

Accepting subaccountSnapshots as a parameter is a significant improvement that enhances the modularity and performance of stream initialization. This change allows the function to operate without the overhead of fetching snapshots internally, which can be critical in performance-sensitive operations.

protocol/streaming/full_node_streaming_manager.go (4)

59-60: Improved initialization logic in OrderbookSubscription.

Replacing sync.Once with *atomic.Bool simplifies the initialization logic, allowing for more straightforward checks of the subscription's state. This change enhances the clarity and efficiency of the subscription initialization process.


79-81: Correct implementation of IsInitialized.

The function correctly checks the initialization state using the new initialized field. This method provides a clean and straightforward public interface for checking the subscription's state.


519-535: Enhanced control in SendFinalizedSubaccountUpdates.

Renaming the method to SendFinalizedSubaccountUpdates and adding a panic condition to ensure it is only called in ExecModeFinalize are significant improvements. These changes enhance the robustness of the method by enforcing stricter control over its execution context, which is crucial for maintaining the integrity of the application's logic.


682-703: Useful addition of GetSubaccountSnapshotsForInitStreams.

The method enhances the functionality of the streaming manager by allowing it to collect snapshots for uninitialized subscriptions. This addition is particularly useful for optimizing the initialization process of streams, ensuring that only necessary data is fetched and processed.

protocol/x/subaccounts/keeper/subaccount.go (1)

828-837: Robust implementation of SendFinalizedSubaccountUpdates.

Renaming the method to SendFinalizedSubaccountUpdates and including a check to ensure it is only called in DeliverTx mode are significant improvements. These changes enhance the robustness of the method by enforcing stricter control over its execution context, which is crucial for maintaining the integrity of the application's logic.

protocol/x/clob/keeper/process_operations.go (1)

61-61: Verify subaccount update handling after function removal.

The removal of fetchSubaccountIdsInvolvedInOpQueue simplifies the code but requires verification to ensure that subaccount updates are still correctly handled. Please confirm that all necessary updates are managed by other parts of the system or through new mechanisms introduced in this PR.

protocol/mocks/ClobKeeper.go (1)

709-711: Approve the function signature change and verify dependent code.

The addition of subaccountSnapshots to InitializeNewStreams enhances its functionality. Please ensure that all dependent code has been updated to accommodate this change.

Verification successful

Function signature change verified; ensure all dependent code is updated.

The addition of subaccountSnapshots to InitializeNewStreams has been propagated to several key areas in the codebase. Please manually verify that all dependent code, including tests and indirect usages, has been updated to accommodate this change.

  • Instances found in:
    • protocol/x/clob/abci.go
    • protocol/x/clob/keeper/keeper.go
    • protocol/x/clob/types/clob_keeper.go
    • protocol/streaming/noop_streaming_manager.go
    • protocol/streaming/types/interface.go
    • protocol/streaming/full_node_streaming_manager.go
    • protocol/mocks/ClobKeeper.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that all dependent code has been updated to accommodate the new parameter in InitializeNewStreams.

# Test: Search for usages of InitializeNewStreams to ensure the new parameter is being passed correctly.
rg --type go -A 5 $'InitializeNewStreams'

Length of output: 3859

protocol/x/subaccounts/types/types.go (1)

80-80: Method Renaming Approved: Reflects Finalized Updates

The renaming of SendSubaccountUpdates to SendFinalizedSubaccountUpdates is approved as it aligns with the PR's objective to emit only finalized updates. This change should help in reducing unnecessary emissions and improving the system's efficiency.

Run the following script to verify the impact of this change across the system:

Verification successful

Method Renaming Verified: Successful Integration Across Codebase

The method SendFinalizedSubaccountUpdates has been successfully integrated across the codebase, replacing the old method name. This confirms that the renaming was comprehensive and aligns with the intended functionality changes.

  • The method is found in multiple files, including types.go, subaccount.go, expected_keepers.go, metric_keys.go, noop_streaming_manager.go, full_node_streaming_manager.go, and interface.go.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the impact of renaming `SendSubaccountUpdates` to `SendFinalizedSubaccountUpdates` across the system.

# Test: Search for the old method usage. Expect: No occurrences of the old method name.
rg --type go -A 5 $'SendSubaccountUpdates'

Length of output: 41


Script:

#!/bin/bash
# Description: Verify the integration of the new method name `SendFinalizedSubaccountUpdates` across the system.

# Test: Search for the new method usage. Expect: Occurrences of the new method name.
rg --type go -A 5 $'SendFinalizedSubaccountUpdates'

Length of output: 5341

protocol/streaming/noop_streaming_manager.go (3)

65-69: New Method Added: GetSubaccountSnapshotsForInitStreams

The addition of the GetSubaccountSnapshotsForInitStreams method is approved as it enhances the functionality of the NoopGrpcStreamingManager by providing a mechanism to obtain snapshots for initializing streams. This should be crucial for managing state in streaming operations.


73-73: Signature Change Approved: InitializeNewStreams

The signature change in the InitializeNewStreams method, where the parameter getSubaccountSnapshot has been replaced with subaccountSnapshots, is approved. This change indicates a shift from a function-based approach to a more direct data structure approach for handling subaccount snapshots, potentially improving efficiency and clarity.


54-54: Method Renaming Approved: Reflects Finalized Updates

The renaming of SendSubaccountUpdates to SendFinalizedSubaccountUpdates is approved as it aligns with the PR's objective to emit only finalized updates. This change should help in reducing unnecessary emissions and improving the system's efficiency.

Run the following script to verify the impact of this change across the system:

Verification successful

Renaming and Integration Verified: Method Transition Successful

The method SendFinalizedSubaccountUpdates is correctly integrated across the codebase, replacing the old method name. The renaming aligns with the intended functionality and is consistently applied in various contexts, confirming the successful transition. No further action is required.

  • Files where the new method is used:
    • protocol/x/subaccounts/types/types.go
    • protocol/x/subaccounts/keeper/subaccount.go
    • protocol/x/clob/types/expected_keepers.go
    • protocol/lib/metrics/metric_keys.go
    • protocol/streaming/full_node_streaming_manager.go
    • protocol/streaming/noop_streaming_manager.go
    • protocol/streaming/types/interface.go
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the impact of renaming `SendSubaccountUpdates` to `SendFinalizedSubaccountUpdates` across the system.

# Test: Search for the old method usage. Expect: No occurrences of the old method name.
rg --type go -A 5 $'SendSubaccountUpdates'

Length of output: 41


Script:

#!/bin/bash
# Description: Verify the usage of the new method `SendFinalizedSubaccountUpdates` across the system.

# Test: Search for the new method usage. Expect: Occurrences of the new method name.
rg --type go -A 5 $'SendFinalizedSubaccountUpdates'

Length of output: 5341

@teddyding teddyding force-pushed the td/fns-subaccounts-finalize-only branch 2 times, most recently from 92726f3 to a3401b8 Compare September 6, 2024 20:27
@teddyding teddyding force-pushed the td/fns-subaccounts-finalize-only branch from a3401b8 to fb9309d Compare September 6, 2024 20:31
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between d1d847d and a3401b8.

Files selected for processing (7)
  • protocol/mocks/ClobKeeper.go (2 hunks)
  • protocol/streaming/full_node_streaming_manager.go (7 hunks)
  • protocol/streaming/noop_streaming_manager.go (2 hunks)
  • protocol/streaming/types/interface.go (2 hunks)
  • protocol/x/clob/abci.go (3 hunks)
  • protocol/x/clob/keeper/keeper.go (3 hunks)
  • protocol/x/clob/types/clob_keeper.go (1 hunks)
Files skipped from review as they are similar to previous changes (6)
  • protocol/mocks/ClobKeeper.go
  • protocol/streaming/noop_streaming_manager.go
  • protocol/streaming/types/interface.go
  • protocol/x/clob/abci.go
  • protocol/x/clob/keeper/keeper.go
  • protocol/x/clob/types/clob_keeper.go
Additional comments not posted (7)
protocol/streaming/full_node_streaming_manager.go (7)

Line range hint 1-6: Approved import changes.

The addition of sync/atomic aligns with the changes to use atomic.Bool for managing initialization state in OrderbookSubscription.


59-60: Structural change approved: OrderbookSubscription initialization.

Replacing sync.Once with *atomic.Bool for the initialized field simplifies the initialization check and ensures thread safety without the overhead of a function call each time the state is checked.


79-81: Method implementation approved: IsInitialized.

The method correctly uses atomic.Bool.Load() to check the initialization state, ensuring thread safety and efficiency.


167-167: Initialization logic approved in Subscribe.

Setting initialized to &atomic.Bool{} ensures that each new subscription starts in an uninitialized state, aligning with the new management strategy.


519-534: Method changes approved: SendFinalizedSubaccountUpdates.

Renaming the method and adding a panic condition to ensure it is only called in ExecModeFinalize are crucial for enforcing the intended behavior of only emitting finalized subaccount updates.


682-704: New method approved: GetSubaccountSnapshotsForInitStreams.

This method efficiently handles the retrieval of subaccount snapshots for subscriptions that are not yet initialized, facilitating the initial setup of streams and ensuring that all necessary data is available before they become active.


722-756: Initialization logic approved in InitializeNewStreams.

Using initialized.Swap(true) to set the state simplifies the initialization logic and provides a clear, thread-safe method to manage the subscription's state. This approach is more straightforward and efficient compared to the previous sync.Once method.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Outside diff range, codebase verification and nitpick comments (1)
protocol/streaming/full_node_streaming_manager.go (1)

682-704: New Method: GetSubaccountSnapshotsForInitStreams

This new method supports the initialization of streams with subaccount snapshots, enhancing the functionality of the streaming manager. It iterates through subscriptions and collects snapshots for those not yet initialized, which is crucial for the PR's focus on finalized updates.

Ensure that the method handles potential race conditions or inconsistencies during snapshot collection, possibly by locking mechanisms or revisiting the logic for concurrent access.

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between a3401b8 and fb9309d.

Files selected for processing (12)
  • protocol/lib/metrics/metric_keys.go (1 hunks)
  • protocol/mocks/ClobKeeper.go (2 hunks)
  • protocol/streaming/full_node_streaming_manager.go (7 hunks)
  • protocol/streaming/noop_streaming_manager.go (2 hunks)
  • protocol/streaming/types/interface.go (2 hunks)
  • protocol/x/clob/abci.go (3 hunks)
  • protocol/x/clob/keeper/keeper.go (3 hunks)
  • protocol/x/clob/keeper/process_operations.go (3 hunks)
  • protocol/x/clob/types/clob_keeper.go (1 hunks)
  • protocol/x/clob/types/expected_keepers.go (1 hunks)
  • protocol/x/subaccounts/keeper/subaccount.go (3 hunks)
  • protocol/x/subaccounts/types/types.go (1 hunks)
Files skipped from review as they are similar to previous changes (8)
  • protocol/lib/metrics/metric_keys.go
  • protocol/mocks/ClobKeeper.go
  • protocol/streaming/noop_streaming_manager.go
  • protocol/streaming/types/interface.go
  • protocol/x/clob/abci.go
  • protocol/x/clob/keeper/keeper.go
  • protocol/x/clob/types/expected_keepers.go
  • protocol/x/subaccounts/types/types.go
Additional comments not posted (6)
protocol/x/clob/types/clob_keeper.go (1)

140-143: Method Signature Change: InitializeNewStreams

The method InitializeNewStreams now includes an additional parameter to handle subaccount snapshots. This change is aligned with the PR's objectives to ensure that only finalized updates are processed. Ensure that all implementations of this interface are updated to accommodate this new parameter.

Run the following script to verify the method usage:

Verification successful

Verification Successful: Method Signature Updated

All implementations of the InitializeNewStreams method have been updated to include the new parameter subaccountSnapshots, as required. The changes align with the PR's objectives to process only finalized updates.

  • protocol/mocks/ClobKeeper.go at line 710
  • protocol/x/clob/keeper/keeper.go at line 276
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all implementations of `InitializeNewStreams` are updated.

# Test: Search for the method implementation. Expect: Only occurrences of the new signature.
ast-grep --lang go --pattern $'func ($_ $_) InitializeNewStreams($_, $_) { $$$ }'

Length of output: 1534

protocol/streaming/full_node_streaming_manager.go (1)

519-535: Method Renaming and Enhancement: SendFinalizedSubaccountUpdates

The method SendSubaccountUpdates has been renamed to SendFinalizedSubaccountUpdates to better reflect its purpose of handling only finalized subaccount updates. This change includes a panic condition to ensure it is only called in ExecModeFinalize, which enforces stricter control over its execution context.

Run the following script to verify the method usage:

Verification successful

Verification Successful: SendFinalizedSubaccountUpdates Usage

The method SendFinalizedSubaccountUpdates is correctly used with ExecModeFinalize. The codebase includes checks to ensure that the execution mode is sdk.ExecModeFinalize when this method is called, aligning with the intended functionality.

  • The panic condition in the method enforces this requirement.
  • The ctx.ExecMode() method is used appropriately to pass the execution mode.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all calls to `SendFinalizedSubaccountUpdates` are in the correct execution mode.

# Test: Search for the method usage. Expect: Only occurrences in `ExecModeFinalize`.
rg --type go -A 5 $'SendFinalizedSubaccountUpdates'

Length of output: 5341


Script:

#!/bin/bash
# Verify the definition and usage of `ctx.ExecMode()` to ensure it returns `ExecModeFinalize`.

# Search for the definition of `ExecMode` in the sdk package.
rg --type go 'ExecMode' -A 3

# Search for the implementation of `ExecMode()` method in the sdk.Context.
rg --type go 'ExecMode()' -A 3

Length of output: 31841

protocol/x/subaccounts/keeper/subaccount.go (1)

828-837: Method Renaming and Logic Enhancement: SendFinalizedSubaccountUpdates

The method SendSubaccountUpdates has been renamed to SendFinalizedSubaccountUpdates, and now includes a check to assert DeliverTx mode before proceeding. This change ensures that updates are only emitted when they are finalized and the system is in the correct state, aligning with the PR's objectives.

Run the following script to verify the method usage:

Verification successful

Verification Successful: SendFinalizedSubaccountUpdates Method Usage

The method SendFinalizedSubaccountUpdates is correctly used with a DeliverTx mode check in its direct call. Other occurrences in the codebase are not direct calls and do not require this check. The review comment's concern is addressed.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all calls to `SendFinalizedSubaccountUpdates` check for `DeliverTx` mode.

# Test: Search for the method usage. Expect: Only occurrences with `DeliverTx` mode check.
rg --type go -A 5 $'SendFinalizedSubaccountUpdates'

Length of output: 5341

protocol/x/clob/keeper/process_operations.go (3)

Line range hint 31-115: Review of ProcessProposerOperations: Simplification and Focus on Finalized Updates.

The changes in ProcessProposerOperations reflect the PR's objective to only emit finalized subaccount updates. The removal of the fetchSubaccountIdsInvolvedInOpQueue function simplifies the handling of these updates, potentially reducing complexity and improving performance. This aligns well with the PR's goals of enhancing efficiency and accuracy by focusing on finalized states.


Line range hint 117-248: Review of ProcessInternalOperations: Comprehensive Handling of Operations.

ProcessInternalOperations effectively handles various types of operations, ensuring that each is processed correctly based on its type. The function's implementation appears robust, with appropriate error handling and state updates. This is crucial for maintaining the integrity of the system's state.


61-61: Clarification needed on the comment regarding optimistic orderbook updates.

The comment at line 61 mentions reverting optimistic orderbook updates during the CheckTx phase. It would be beneficial to clarify whether this change is directly related to the removal of the fetchSubaccountIdsInvolvedInOpQueue function and how it impacts the overall system behavior.

Comment on lines +59 to +60
// Whether the subscription is initialized with snapshot.
initialized *atomic.Bool
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Refactor: Initialization Tracking in OrderbookSubscription

The OrderbookSubscription struct now uses *atomic.Bool for the initialized field, replacing the previous *sync.Once. This change simplifies the initialization logic and is more appropriate for the use case where the initialization state needs to be checked and updated multiple times.

Consider adding more detailed comments explaining why atomic.Bool is preferred over sync.Once in this context, especially for future maintainers.

Also applies to: 79-81

// Initialize the subscription with orderbook snapshots.
initialize *sync.Once
// Whether the subscription is initialized with snapshot.
initialized *atomic.Bool
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Switched to atomic.Bool since I need to query it here

@teddyding teddyding merged commit 4780b4c into main Sep 9, 2024
22 checks passed
@teddyding teddyding deleted the td/fns-subaccounts-finalize-only branch September 9, 2024 15:33
time.Now(),
)

if execMode != sdk.ExecModeFinalize {
panic("SendFinalizedSubaccountUpdates should only be called in ExecModeFinalize")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should panic be before defer?

// 2. A new subaccount is subscribed to by a new subscription.
// 3. InitializeNewStreams is called.
// Then the new subaccount would not be included in the snapshot.
// We are okay with this behavior.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In this case, the client would be notified of the subaccount's position on the next update since it won't be sent out in the snapshot right?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Development

Successfully merging this pull request may close these issues.

3 participants