-
Notifications
You must be signed in to change notification settings - Fork 608
Description
Search before asking
- I had searched in the issues and found no similar issues.
Motivation
There has been ongoing discussion in the community regarding Graceful Failover (see apache/kvrocks#2848). While a specific solution was previously proposed in #3295, the process has remained stalled due to several unresolved questions.
To move forward, I have implemented a simplified failover scheme previously discussed in the community using the CLIENT PAUSE and CLIENT UNPAUSE commands. This approach serves two purposes: it implements missing Redis protocols and prevents sequence number (seq) mismatches during the Failover (Takeover) process. Ultimately, this enhances Kvrocks' manual failover capabilities for production environments.
Solution
By introducing CLIENT PAUSE/UNPAUSE, the controller can now manage external writes to Kvrocks, allowing for a secure Failover once the master and slave sequences (
The behavior of CLIENT PAUSE/UNPAUSE is consistent with Redis:
- Blocking Mechanism: After executing CLIENT PAUSE, Kvrocks will block connections attempting to run restricted commands.
- Release/Error Handling: These commands are released once the timeout expires or CLIENT UNPAUSE is called.
- Role Transition: If a role change occurs (e.g., Master
$\rightarrow$ Slave) during the pause, any previously blocked write requests will return a READONLY error upon resumption.
While simply returning an immediate error for write requests during a failover might be a "cleaner" or more concise implementation, I chose the blocking approach to maintain strict parity with Redis behavior and remain faithful to the standard CLIENT PAUSE semantics.
Are you willing to submit a PR?
- I'm willing to submit a PR!