GBASE数据库 | GBase 8a MPP Cluster Cluster Management Tool (2)

GBase Database (GBase数据库) is designed to meet the demands of big data analytics, offering powerful MPP capabilities and efficient cluster management tools to streamline complex data processing.

Node Management

Adding a Specified Data Node

You can add data nodes to a specific cluster by modifying the gcChangeInfo.xml file. This file contains information about new nodes. After a successful cluster installation, the system will automatically call this command, so users do not need to manually execute the addnodes command.

Syntax:

gcadmin addnodes gcChangeInfo.xml [vc_name | single_vc_add_to_rc]

Parameters:

• gcChangeInfo.xml: This file is similar in format to the node installation file and describes the information for newly installed nodes. After installing the GBase 8a software package on a new data node, the gcadmin addnodes command is automatically called to add the node to the root cluster as a Free Node.
• vc_name: If you specify a vc_name, the Free Node is added to the specified VC. Otherwise, it is added to the root cluster.
• single_vc_add_to_rc: In compatibility mode, this parameter adds the node to the root cluster as a Free Node. This option is only valid in compatibility mode.

Example:

[gbase@node131 ~]$ gcadmin addnodes gcChangeInfo.xml vc2
[gbase@node131 ~]$ gcadmin showcluster vc vc2

Output:

CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL
===============================================
|      GBASE VIRTUAL CLUSTER INFORMATION      |
===============================================
|    VcName    | DistributionId |   comment   |
-----------------------------------------------
|     vc2      |        3       | vc2 cluster |
-----------------------------------------------
=========================================================================================================
|                                 VIRTUAL CLUSTER DATA NODE INFORMATION                                 |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |               172.16.4.133               |       3        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node2   |               172.16.4.134               |       3        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node3   |               172.16.4.151               |                | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------

Modify gcChangeInfo.xml to Include Only New Nodes:

<?xml version="1.0" encoding="utf-8"?>
<servers>
 <rack>
   <node ip="172.16.4.151"/>
 </rack>
</servers>

Scenarios for Adding Nodes to the Cluster:

Scenario 1:
Add Free Node Data Node to Root Cluster (Automatic After Installation)
When adding a new node, the node is automatically added to the root cluster as a Free Node after installing the 8a database software. In a multi-VC environment, the new node is automatically added to the root cluster as a Free Node.

Note: In a single VC compatibility mode, the new node is added to the default cluster as a data node.

Scenario 2:
Add Free Node to a Specified VC (Requires the Add Nodes Command)
To add a Free Node from the root cluster to a specified VC, use the addnodes command to make the Free Node part of the VC’s data node set.

Command Example:

gcadmin addnodes gcChangeInfo.xml vc_name

Scenario 3:
Add New Coordinator Node to the Cluster (Automatic After Installation)
When adding a new node, it automatically joins the coordinator cluster as a coordinator node after installing the GBase 8a software.

Example for Scenario 1: Add Free Node Data Node to Root Cluster

Set Up New Node Installation Environment (Example for RHEL7):

1) Create a DBA user for the new node, specify the password, and grant the database installation directory ownership to the DBA user (same as other cluster nodes).

# useradd gbase 
# passwd gbase           
# chown -R gbase:gbase /opt      

2) Use the SetSysEnv.py script from the gcinstall directory to set system parameters on the new node.

# scp root@main-install-node-ip:/opt/gcinstall/SetSysEnv.py /opt      
# python SetSysEnv.py --dbaUser=gbase --installPrefix=/opt --cgroup

Parameters:

• --installPrefix: The database installation directory, which must match the installPrefix in the demo.options file.
• --dbaUser: The DBA user for the cluster installation. It must match the dbaUser in demo.options.
• --cgroup: Used for modifying resource management configurations. By default, this is not set.
• Modify demo.options file: Set the dataHost, existCoordinateHost, and existDataHost parameters.

installPrefix= /opt
dataHost = 172.16.4.15
existCoordinateHost = 172.16.4.131,172.16.4.132,172.16.4.133
existDataHost = 172.16.4.131,172.16.4.132,172.16.4.133,172.16.4.134
dbaUser = gbase
dbaGroup = gbase
dbaPwd = gbase
rootPwd = 111111
#mcastAddr = 226.94.1.39
mcastPort = 5493

Execute the Installation:

./gcinstall.py --silent=demo.options --license_file hostslicense.txt

Viewing the Expansion Result in Multi-VC Mode:

[gbase@node131 gcinstall]$ gcadmin

Output:

CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL

===============================================================
|           GBASE COORDINATOR CLUSTER INFORMATION            |
===============================================================
|   NodeName   |  IpAddress   | gcware | gcluster | DataState |
---------------------------------------------------------------
| coordinator1 | 172.16.4.131 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator2 | 172.16.4.132 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator3 | 172.16.4.133 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator4 | 172.16.4.149 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------

===============================================================
|       GBASE VIRTUAL CLUSTER INFORMATION                     |
===============================================================
|   VcName    | DistributionId |   comment   |
-----------------------------------------------
|     vc1      |       2        | vc1 cluster |
-----------------------------------------------
|     vc2      |       3        | vc2 cluster |
-----------------------------------------------

===============================================================
|      GBASE DATA CLUSTER INFORMATION                        |
===============================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------
|  node1   |               172.16.4.142               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------
|  node2   |               172.16.4.143               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------
|  node3   |               172.16.4.144               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------

Output Details:

• 2 Virtual Clusters: vc1, vc2
• 3 Coordinator Nodes
• 1 Free Node

Example for Scenario 2: Adding a Free Node to a Specified Virtual Cluster (VC)

Modify the gcChangeInfo.xml file, only including the new node.

[gbase@131 gcinstall]$ cat gcChangekuorong2.xml 
<?xml version="1.0" encoding="utf-8"?>
<servers>
  <rack>
    <node ip="172.16.4.151"/>
  </rack>
</servers>

Use the gcadmin tool to add the new node to the specified VC.

[gbase@node131 ~]$ gcadmin addnodes gcChangekuorong2.xml vc2

Check the addition results:

[gbase@node131 ~]$ gcadmin showcluster vc vc2 
CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL

===============================================
|      GBASE VIRTUAL CLUSTER INFORMATION      |
===============================================

|    VcName    | DistributionId |   comment   |
-----------------------------------------------
|     vc2      |        3       | vc2 cluster |
-----------------------------------------------

=========================================================================================================
|                                 VIRTUAL CLUSTER DATA NODE INFORMATION                                 |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |               172.16.4.133               |       3        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node2   |               172.16.4.134               |       3        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node3   |               172.16.4.151               |                | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------

3 data nodes

Example for Scenario 3: Adding a New Coordinator Node to the Cluster

Set up the installation environment for the new node (explained using RHEL7 as an example):
1) Create the DBA user for the new node, assign a password, and give ownership of the database installation directory to the DBA user (matching other cluster nodes).

# useradd gbase  and # passwd gbase
# chown -R gbase:gbase /opt

2) Use the SetSysEnv.py script provided in the gcinstall directory to set system parameters for the new node.

Copy the SetSysEnv.py script from the main installation node to the new cluster node’s /opt directory and execute the environment setup (matching other cluster nodes).

# scp root@main_install_node_ip:/opt/gcinstall/SetSysEnv.py /opt
# python SetSysEnv.py --dbaUser=gbase --installPrefix=/opt --cgroup

Parameters:

• --installPrefix: The database installation directory specified by the user, which must match the installPrefix set in the demo.options file.
• --dbaUser: The DBA user, which must be the same as specified in the demo.options file.
• --cgroup: Used for modifying resource management configuration files when enabling resource management (default is not configured).

Stop the 8a services on all cluster nodes ($gcluster_services all stop).

Modify the demo.options file:

• Set CoordinateHost to the IP of the new Coordinator node.
• Set existCoordinateHost to the IPs of existing Coordinator nodes.
• Set existDataHost to the IPs of all existing data nodes.

Example configuration:

installPrefix = /opt
coordinateHost = 172.16.4.149
# dataHost = 172.16.83.14
existCoordinateHost = 172.16.4.142,172.16.4.143,172.16.4.144
existDataHost = 172.16.4.142,172.16.4.143,172.16.4.144
dbaUser = gbase
dbaGroup = gbase
dbaPwd = gbase
rootPwd = 111111

Use the gbase user to execute the installation:

./gcinstall.py --silent=demo.options --license_file hostslicense.txt

Check the cluster information:

[gbase@node143 gcinstall]$ gcadmin
CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL

===============================================================
|            GBASE COORDINATOR CLUSTER INFORMATION            |
===============================================================
|   NodeName   |  IpAddress   | gcware | gcluster | DataState |
---------------------------------------------------------------
| coordinator1 | 172.16.4.142 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator2 | 172.16.4.143 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator3 | 172.16.4.144 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator4 | 172.16.4.149 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------

=========================================================================================================
|                                    GBASE DATA CLUSTER INFORMATION                                     |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |               172.16.4.142               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node2   |               172.16.4.143               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node3   |               172.16.4.144               |       1        | OPEN  |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------

---

Removing a Specified Data Node

To remove a specific data node from the cluster, use the gcChangeInfo.xml file, which contains the node details.

Syntax:

gcadmin rmnodes gcChangeInfo.xml [vc_name | single_vc_rm_to_rc]

Parameters:

• gcChangeInfo.xml: The file containing the data node details to be removed. If no optional parameters are specified, it will remove the Free Node from the root cluster.
• vc_name: Specify the VC name to remove the node from the VC and make it a Free Node in the root cluster.
• single_vc_rm_to_rc: In compatibility mode, this parameter removes the node from the default cluster and makes it a Free Node in the root cluster.

Scenarios for Removing Nodes from the Cluster:

• Scenario 1: Remove Coordinator Node (Uninstall the Node)
  Uninstalling the GBase 8a software from the coordinator node will remove it from the cluster.

• Scenario 2: Remove Data Node from VC (Use the Node Removal Command)
  To remove a data node from a VC, use the rmnodes command to move the node to the Free Node status in the root cluster.

• Scenario 3: Remove Free Node from Root Cluster
  To remove a Free Node from the root cluster, use the removal commands, which can then be followed by the uninstallation operation.

Example for Scenario 1: Removing a Coordinator Node

Uninstalling the Coordinator Node

1) Stop the cluster services on all nodes:

   $gcluster_services all stop

2) Modify the demo.options file:

• Set CoordinateHost to the IP of the Coordinator node to be uninstalled.
• Modify the existCoordinateHost and existDataHost parameters to include only the remaining nodes.

Example configuration:

   installPrefix = /opt
   coordinateHost = 172.16.4.133

   # dataHost = 

   existCoordinateHost = 172.16.4.131,172.16.4.132
   existDataHost = 172.16.4.133,172.16.4.134

   dbaUser = gbase
   dbaGroup = gbase
   dbaPwd = gbase
   rootPwd = 111111

   # mcastAddr = 226.94.1.39
   mcastPort = 5493

3) Run the uninstallation command:

   ./unInstall.py --silent=demo.options

Note:

When using the uninstallation command to remove a node, do not use the --Force parameter. Using this option will bypass the script's check to confirm whether the node being removed is currently in use by the cluster.

Example for Scenario 2: Removing a Data Node from the VC

1) Modify the gcChangeInfo.xml file (only include the node to be removed):

Rearrange the rack by placing the node to be removed in the appropriate rack:

<?xml version="1.0" encoding="utf-8"?> 
<servers>
 <rack>
   <node ip="172.16.9.165"/> 
 </rack> 
</servers>

2) In compatibility mode, run the command to remove the cluster node:

gcadmin rmnodes gcChangeInfo.xml single_vc_rm_to_rc

3) Check the cluster information:

[gbase@163 gcinstall]$ gcadmin
CLUSTER STATE:         ACTIVE
VIRTUAL CLUSTER MODE:  NORMAL
===============================================================
|            GBASE COORDINATOR CLUSTER INFORMATION            |
===============================================================
|   NodeName   |  IpAddress   | gcware | gcluster | DataState |
---------------------------------------------------------------
| coordinator1 | 172.16.9.163 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
| coordinator2 | 172.16.9.164 |  OPEN  |   OPEN   |     0     |
---------------------------------------------------------------
=========================================================================================================
|                                    GBASE DATA CLUSTER INFORMATION                                     |
=========================================================================================================
| NodeName |                IpAddress                 | DistributionId | gnode | syncserver | DataState |
---------------------------------------------------------------------------------------------------------
|  node1   |               172.16.9.163               |       1        |  OPEN |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
|  node2   |               172.16.9.164               |       1        |  OPEN |    OPEN    |     0     |
---------------------------------------------------------------------------------------------------------
=============================================================
|         GBASE CLUSTER FREE DATA NODE INFORMATION          |
=============================================================
| NodeName  |  IpAddress   | gnode | syncserver | DataState |
-------------------------------------------------------------
| FreeNode1 | 172.16.9.165 |  OPEN |    OPEN    |     0     |
-------------------------------------------------------------

Example for Scenario 3: Removing a Free Node from the Root Cluster

1) Stop the services on the data node to be uninstalled ($gcluster_services all stop).

2) Modify the demo.options file:

Set the dataHost parameter to the IP of the data node to be uninstalled; update the existCoordinateHost and existDataHost parameters to retain only the remaining nodes.

installPrefix= /opt
#coordinateHost =
dataHost = 172.16.9.165
existCoordinateHost = 172.16.9.163,172.16.9.164
existDataHost = 172.16.9.163,172.16.9.164
dbaUser = gbase
dbaGroup = gbase
dbaPwd = gbase
rootPwd = 111111
#mcastAddr = 226.94.1.39
mcastPort = 5493

---

Set Node State

Sets the status of a node.

Syntax:

gcadmin setnodestate <ip> <state>

Parameters:

• IP: IP address of the node to set the status.
• state: The state to set the node to. There are three possible node states:
  • unavailable: Marks the node as unavailable, without logging an event (fevent). After setting this state, the node must be replaced, and its state is restored to normal upon completion.
  • failure: Indicates a cluster failure, equivalent to offline, where DML and DDL commands will not be dispatched to this node but will be logged in fevent. The node can be manually restored to normal once repaired.
  • normal: Once the node failure is resolved, it can be set back to normal, which reactivates it (online), allowing gcrecover to restore previously logged fevents. New DDL and DML commands will again be dispatched to this node.

---

Distribution Information Management

Overview

Distribution information management involves managing data shard distribution, including creating, deleting, viewing, and exporting distribution tables.

The “distribution table,” also called the “Distribution Table,” is a core mapping table in the GBase 8a database that stores data. It records key information on the layout of VCs, racks, nodes, and shards, determining data storage and access locations, which impact database performance and high availability.

Create Distribution Information

There are two ways to create a distribution table:

1) Automatic Creation: The administrator selects a built-in sharding distribution rule based on business needs, and the system automatically generates the distribution table.
2) Manual Creation: The administrator manually specifies the locations for primary and standby shards by editing an XML configuration file to create the distribution table.

Syntax:

gcadmin distribution <gcChangeInfo.xml> <p number> [d number] [extension] [pattern 1|2] [db_user user_name] [db_pwd password] [vc vc_name]

Parameters:

• gcChangeInfo.xml: File describing node information within the rack.
• p number: Number of shards per data node, where p * node count <= 65535. (In pattern 1 mode, this must be less than the number of nodes within each rack (1 <= p < nodes per rack)).
• d number: Number of backups per shard, with a value of 0, 1, or 2. If d is not specified, the default value is 1.
• extension: Distributes the new distribution shards as much as possible to the original nodes.
• pattern number: pattern 1 is the load balancing mode, and pattern 2 is the high availability mode. The default is pattern 1.
• db_user user_name: When generating a new distribution and the data nodes of the new and old distributions differ, pass in the database username.
• db_pwd password: The password for the specified database username.
• vc vc_name: Specifies the virtual cluster for generating the distribution.

Core Parameters for Automatic Distribution Table Creation:

Data Distribution Template Parameters:
1) pattern 1 – Load Balancing Mode

"Pattern 1" is the system’s default mode. It adjusts the distribution of shards based on rack settings. A "rack" is a logical grouping of Data nodes within the database, and each rack affects the primary and standby shard locations.

Multiple Racks

Single Rack

Note: Rack configurations are independent of VC, as they only define node layouts within a VC.

2) pattern 2 – High Availability Mode

"Pattern 2" backs up primary shards across nodes, independent of racks, for node-specific redundancy. This configuration tolerates up to half of the nodes in the cluster failing, suitable for deployments where odd and even nodes are separated across different cabinets.

No Rack

• P parameter: Number of primary shards per node.
• d parameter: Number of backups for each primary shard.

Viewing VC Distribution

This command allows you to view the distribution information table for a specified VC, primarily displaying the distribution table ID, status, and primary and standby segment placement rules.

• Each time a new distribution table is created, the distribution table ID automatically increments by "1".
• A maximum of two distribution tables can exist simultaneously within each VC.
• Newly added data will automatically be stored based on the distribution table with the "new" status.

Syntax:

gcadmin showdistribution [node | f] [vc vc_name]

Parameters:

• node: Displays each node’s segments.
• f: Displays in XML format.
• vc vc_name: Shows the distribution information table for the specified VC.

Example:

[gbase@181 ~]$ gcadmin showdistribution vc hr_vc
                                Distribution ID: 4 | State: new | Total segment num: 2

             Primary Segment Node IP                   Segment ID                 Duplicate Segment Node IP
========================================================================================================================
|                  172.16.9.181                  |         1          |                  172.16.9.182                  |
------------------------------------------------------------------------------------------------------------------------
|                  172.16.9.182                  |         2          |                  172.16.9.181                  |
========================================================================================================================

[gbase@181 ~]$ gcadmin showdistribution node vc hr_vc
                               Distribution ID: 4 | State: new | Total segment num: 2

============================================================================================
|  nodes   |             172.16.9.181              |             172.16.9.182              |
--------------------------------------------------------------------------------------------
| primary  |                  1                    |                  2                    |
| segments |                                       |                                       |
--------------------------------------------------------------------------------------------
|duplicate |                  2                    |                  1                    |
|segments 1|                                       |                                       |
============================================================================================

Exporting VC Distribution

To save the distribution information of a specified ID in an XML file. The generated file can be edited to modify segment information, then used to manually recreate a new distribution.

Syntax:

gcadmin getdistribution <ID> <distribution_info.xml> [vc vc_name]

Parameters:

• ID: Distribution ID to retrieve.
• distribution_info.xml: Filename to save the distribution information.

Example:

[gbase@181 ~]$ gcadmin getdistribution 4 distribution.xml vc hr_vc

get segments information
write segments information to file [distribution.xml]
gcadmin getdistribution information successful

Deleting VC Distribution

This command removes a specified distribution ID from the cluster. If no ID is specified, it deletes the "old" distribution by default. If there is only one distribution in the cluster, it deletes that distribution.

Syntax:

gcadmin rmdistribution [ID] [vc vc_name]

If the distribution is currently in use, it cannot be deleted. First, run the refreshnodedatamap drop <ID> command to remove it. Additionally, clear any fevent logs if present before deletion. Ensure all GCluster node services are operating normally before deletion to avoid irreversible fevent logs.

Additional Commands

1) Get Help

   gcadmin --help

2) Check Version

   gcadmin -V

---

With GBase 8a’s robust management tools, GBase Database (GBase数据库) efficiently supports big data analytics, empowering businesses to handle large-scale data with ease.