Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix race condition when slots are re-calculated #2731

Merged
merged 1 commit into from
May 16, 2024
Merged

Conversation

jjsimps
Copy link
Contributor

@jjsimps jjsimps commented Apr 2, 2024

Description

I was getting an error similar to #2704.

What was happening for me is that I was creating a redis cluster client and then calling client.eval(). Since eval doesn't seem to be cluster-aware, it returned a MOVE error since it was routed to an incorrect node. This seems to cause a call to discover() to re-discover the hash slot mapping.

The bug I am trying to address is a race condition between when the internal data is cleared to when it's re-populated again. In the current implementation, there's an asynchronous command being run in-between, which means if any other command was run in between, would cause this.slots[val] of cluster-slots.ts (such as in getSlotRandomNode()) to return undefined and throw an exception.

 async #discover(rootNode?: RedisClusterClientOptions) {
        this.#resetSlots(); // !!! Reset is being called here
        const addressesInUse = new Set<string>();

        try {
            const shards = await this.#getShards(rootNode), // !!! But there's an async command here
                promises: Array<Promise<unknown>> = [],
                eagerConnect = this.#options.minimizeConnections !== true;
            for (const { from, to, master, replicas } of shards) {

This changes moves when it is cleared so that it's cleared then populated, without any async stuff inbetween.


Checklist

  • Does npm test pass with this change (including linting)?
  • Is the new or changed code fully tested?
  • Is a documentation update included (if this change modifies existing APIs, or introduces new ones)?

@jjsimps
Copy link
Contributor Author

jjsimps commented May 3, 2024

Hi, can this be taken a look at? I can't seem to add people as reviewers. Thank you.

@leibale leibale self-assigned this May 3, 2024
@leibale
Copy link
Collaborator

leibale commented May 3, 2024

@jjsimps sorry for the delay. I'll take a look at the start of next week. If i forget - ping me.. ;)

Copy link
Collaborator

@leibale leibale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch! Sorry about the (way too long) delay

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants