How to Perform a Rolling Restart of MySQL NDB Cluster?
When you need to restart all of the nodes in a cluster for a change to take effect, you’ll need to use a rolling restart. Instead of shutting down the entire cluster, you can restart each node individually to keep the cluster serving requests while the changes are made.
Most processes, such as changing config.ini settings, can be performed using this method (ie. DataMemory, MaxNoOfConcurrentTransactions, etc.). Even so, there is one scenario in which you cannot use this procedure: changing the layout of an NDB cluster node.
When you change the topography of a cluster, for example, by changing the NoOfReplicas or removing data nodes. In the case above, you’ll need to shut down and restart the cluster completely. You may also need to dump and re-import data in some cases.
Note that Schema changes (DDL) for NDBCluster are not permitted while a data node restart is in progress. If you try to run a DDL statement, the cluster will give you the error “711 – System busy with node restart, schema operations not allowed.”
The following steps should be followed in order to perform a rolling restart:
- Install the new software version or configure the new config.ini.
- Using ndb mgmd, restart the management node. This causes the management server to read in the new configuration file, which the data and mysqld nodes will retrieve in the following steps. For later versions, don’t forget to include the –reload option.
- One by one, restart each data node with ndbd (or ndbmtd). This requires a full restart, which must be performed on each machine rather than through the Management Server. This causes the node to read the remote configuration and then use the new settings and software version.
- Each SQL node should be restarted (mysqld). You may need to reconfigure the applications to reuse each one as it is brought up, based on how they handle failover conditions with the mysqld nodes.
After you’ve completed the steps above, all of the new configuration data should have been propagated to the nodes. You should also be running the latest version of the cluster software if you upgraded it. During the upgrade process, the complete cluster should never refuse to serve requests during the steps above.
Note: If you have multiple management nodes in 7.0.8 or later, you must stop them all before restarting.
Are you looking for an answer to another query? Contact our technical support team.
