The node will stop updating its liveness record. Process terminationĪfter draining completes, the node process is automatically terminated (unless the node was manually drained).Ī node process termination stops the cockroach process on the node. It is only stopped by process termination. A client attempting to connect to a decommissioned node and run a query will get an error.Īt this point, the cockroach process is still running. This node is now cut off from communicating with the rest of the cluster. Status changeĪfter decommissioning and draining are both complete, the node membership changes from decommissioning to decommissioned. When draining manually, if the above steps have not completed after 10 minutes by default, node draining will stop and must be restarted to continue. Since all range replicas were already removed from the node during the decommissioning stage, this step immediately resolves. This phase completes when all range leases and Raft leaderships have been transferred. Any range leases or Raft leaderships must be transferred to other nodes. Lease transfer phase: The node's is_draining field is set to true, which removes the node as a candidate for replica rebalancing, lease transfers, and query planning. This phase completes either when all transactions have been processed or the maximum duration set by _wait is reached. This phase completes either when all transactions have been processed or the maximum duration set by _wait is reached.ĭistSQL drain phase: All distributed statements initiated on other gateway nodes are allowed to complete, and DistSQL requests from other nodes are no longer accepted. After this phase completes, CockroachDB closes all remaining SQL client connections to the node. SQL drain phase: All active transactions and statements for which the node is a gateway are allowed to complete, and CockroachDB closes the SQL client connections immediately afterward. This phase completes either when all SQL client connections are closed or the maximum duration set by _wait is reached. SQL wait phase: New SQL client connections are no longer permitted, and any remaining SQL client connections are allowed to close or time out. This phase completes when the fixed duration set by _wait is reached. Unready phase: The node's /health?ready=1 endpoint returns an HTTP 503 Service Unavailable response code, which causes load balancers and connection managers to reroute traffic to other nodes.
Node drain consists of the following consecutive phases:
Draining a node disconnects clients after active queries are completed, and transfers any range leases and Raft leaderships to other nodes, but does not move replicas or data off of the node.Īfter all replicas on a decommissioning node are rebalanced, the node is automatically drained.
For more information, see Perform node shutdown.Īn operator initiates the draining process on the node. However, to avoid possible disruptions in query performance, we recommend manually draining before decommissioning. After this stage, the node is automatically drained.
0 kommentar(er)
