MariaDB & K8s: 如何在 K8s 中复制 MariaDB - MariaDB.org

在上一篇博客中，我们了解了如何在 K8s 中创建 StatefulSet MariaDB 应用程序。此外，我们在这篇博客中学习了 MariaDB 中的复制工作原理。现在，我们将尝试创建一个复制型的 StatefulSet 应用程序。作为撰写本博客的良好参考，我感谢Kubernetes 文档以及阿里云的示例。

配置复制

要复制 MariaDB 应用程序，我们将创建一个 StatefulSet，它将包含一个init 容器和一个应用程序容器。这两个容器都将基于 MariaDB 镜像。

init 容器将在应用程序容器之前运行，我们将使用该容器处理所有与复制相关的设置，主要是为特定容器类型（主库或副本）将包含特定文件的卷挂载到特定目录中。在此过程中使用的文件将从配置映射（我们需要挂载一个额外的卷）中获取，该配置映射将包含主库和副本所需的配置文件，这些文件将被挂载到 /etc/mysql/conf.d/（全局配置文件目录），以及包含 SQL 语句的 SQL 文件，这些文件将被挂载到 docker-entrypoint-initdb.d 中，这些语句将在容器首次启动时执行。

应用程序 MariaDB 容器将使用这些卷，这些卷包含针对复制进行了微调的配置，以及一个用于数据目录的持久卷。

配置文件如下所示

# ConfigMap holding information about configuration files for primary/secondary and dockerinit
apiVersion: v1
kind: ConfigMap
metadata:
  name: mariadb-configmap
data:

  primary.cnf: |
    [mariadb]
    log-bin                         # enable binary logging
    log-basename=my-mariadb         # used to be independent of hostname changes (otherwise is in datadir/mysql)

  replica.cnf: |
    [mariadb]
    log-basename=my-mariadb         # used to be independent of hostname changes (otherwise is in datadir/mysql)

  primary.sql: |
    CREATE USER 'repluser'@'%' IDENTIFIED BY 'replsecret';
    GRANT REPLICATION REPLICA ON *.* TO 'repluser'@'%';
    CREATE DATABASE primary_db;

  secondary.sql: |
    # We have to know name of sts (`mariadb-sts`) and 
    # service `mariadb-service` in advance as an FQDN.
    # No need to use master_port
    CHANGE MASTER TO 
    MASTER_HOST='mariadb-sts-0.mariadb-service.default.svc.cluster.local',
    MASTER_USER='repluser',
    MASTER_PASSWORD='replsecret',
    MASTER_CONNECT_RETRY=10;

# Secret holds information about root password
---
apiVersion: v1
kind: Secret
metadata:
    name: mariadb-secret
type: Opaque
data:
  mariadb-root-password: c2VjcmV0 # echo -n 'secret'|base64

# Headless service
---
apiVersion: v1
kind: Service
metadata:
  name: mariadb-service
  labels:
    app: mariadb
spec:
  ports:
  - port: 3306
    name: mariadb-port
  clusterIP: None
  selector:
    app: mariadb

# Statefulset
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mariadb-sts
spec:
  serviceName: "mariadb-service"
  replicas: 3
  selector:
    matchLabels:
      app: mariadb
  template:
    metadata:
      labels:
        app: mariadb
    spec:
      initContainers:
      - name: init-mariadb
        image: mariadb
        imagePullPolicy: Always
        command:
        - bash
        - "-c"
        - |
          set -ex
          echo 'Starting init-mariadb';
          # Check config map to directory that already exists 
          # (but must be used as a volume for main container)
          ls /mnt/config-map
          # Statefulset has sticky identity, number should be last
          [[ `hostname` =~ -([0-9]+)$ ]] || exit 1
          ordinal=${BASH_REMATCH[1]}
          # Copy appropriate conf.d files from config-map to 
          # mariadb-config volume (emptyDir) depending on pod number
          if [[ $ordinal -eq 0 ]]; then
            # This file holds SQL for connecting to primary
            cp /mnt/config-map/primary.cnf /etc/mysql/conf.d/server-id.cnf
            # Create the users needed for replication on primary on a volume
            # initdb (emptyDir)
            cp /mnt/config-map/primary.sql /docker-entrypoint-initdb.d
          else
            # This file holds SQL for connecting to secondary
            cp /mnt/config-map/replica.cnf /etc/mysql/conf.d/server-id.cnf
            # On replicas use secondary configuration on initdb volume
            cp /mnt/config-map/secondary.sql /docker-entrypoint-initdb.d
          fi
          # Add an offset to avoid reserved server-id=0 value.
          echo server-id=$((3000 + $ordinal)) >> etc/mysql/conf.d/server-id.cnf
          ls /etc/mysql/conf.d/
          cat /etc/mysql/conf.d/server-id.cnf
        volumeMounts:
          - name: mariadb-config-map
            mountPath: /mnt/config-map
          - name: mariadb-config
            mountPath: /etc/mysql/conf.d/
          - name: initdb
            mountPath: /docker-entrypoint-initdb.d
      restartPolicy: Always
      containers:
      - name: mariadb
        image: mariadb
        ports:
        - containerPort: 3306
          name: mariadb-port
        env:
        # Using Secrets
        - name: MARIADB_ROOT_PASSWORD
          valueFrom:
            secretKeyRef:
              name: mariadb-secret
              key: mariadb-root-password
        - name: MYSQL_INITDB_SKIP_TZINFO
          value: "1"
        # Mount volume from persistent volume claim
        volumeMounts:
        - name: datadir
          mountPath: /var/lib/mysql/
        - name: mariadb-config
          mountPath: /etc/mysql/conf.d/
        - name: initdb
          mountPath: /docker-entrypoint-initdb.d
      volumes:
      - name: mariadb-config-map
        configMap:
          name: mariadb-configmap
          #defaultMode: 0544
      - name: mariadb-config
        emptyDir: {}
      - name: initdb
        emptyDir: {}

  volumeClaimTemplates:
  - metadata:
      name: datadir
    spec:
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 300M

测试复制

应用配置文件并观察 Pod 的创建

$ kubectl apply -f mariadb-sts-replication.yaml
configmap/mariadb-configmap created
secret/mariadb-secret created
service/mariadb-service created
statefulset.apps/mariadb-sts created

$ kubectl get pod -w
NAME            READY   STATUS     RESTARTS   AGE
mariadb-sts-0   1/1     Running    0          14s
mariadb-sts-1   1/1     Running    0          8s
mariadb-sts-2   0/1     Init:0/1   0          2s
mariadb-sts-2   0/1     PodInitializing   0          4s
mariadb-sts-2   1/1     Running           0          6s

要调试特定的 Pod/容器，请使用以下命令

$ kubectl describe pod mariadb-sts-0
$ kubectl logs mariadb-sts-0 -c init-mariadb

在主库上创建数据

$ kubectl exec -it mariadb-sts-0 -- mariadb -uroot -psecret
Defaulted container "mariadb" out of: mariadb, init-mariadb (init)
MariaDB [primary_db]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| primary_db         |
| sys                |
+--------------------+
5 rows in set (0.000 sec)
MariaDB [primary_db]> create table my_table (t int); insert into my_table values (5),(15),(25);
Query OK, 0 rows affected (0.031 sec)

Query OK, 3 rows affected (0.004 sec)
Records: 3  Duplicates: 0  Warnings: 0

检查副本上的数据

$ kubectl exec -it mariadb-sts-2 -- mariadb -uroot -psecret
MariaDB [(none)]> show databases;
+--------------------+
| Database           |
+--------------------+
| information_schema |
| mysql              |
| performance_schema |
| primary_db         |
| sys                |
+--------------------+
MariaDB [(none)]> use primary_db;
Database changed
MariaDB [primary_db]> show tables;
+----------------------+
| Tables_in_primary_db |
+----------------------+
| my_table             |
+----------------------+
1 row in set (0.000 sec)

MariaDB [primary_db]> select * from my_table;
+------+
| t    |
+------+
|    5 |
|   15 |
|   25 |
+------+
3 rows in set (0.000 sec)

扩容

$ kubectl scale sts mariadb-sts --replicas=4
statefulset.apps/mariadb-sts scaled

$ kubectl get pod -w
NAME            READY   STATUS    RESTARTS   AGE
mariadb-sts-0   1/1     Running   0          2m52s
mariadb-sts-1   1/1     Running   0          2m46s
mariadb-sts-2   1/1     Running   0          2m40s
mariadb-sts-3   0/1     Pending   0          0s
mariadb-sts-3   0/1     Pending   0          0s
mariadb-sts-3   0/1     Pending   0          2s
mariadb-sts-3   0/1     Init:0/1   0          2s
mariadb-sts-3   0/1     PodInitializing   0          5s
mariadb-sts-3   1/1     Running           0          7s

检查新创建的副本

$ kubectl exec -it mariadb-sts-3 -- mariadb -uroot -psecret
MariaDB [(none)]> use primary_db;
MariaDB [primary_db]> show tables;
+----------------------+
| Tables_in_primary_db |
+----------------------+
| my_table             |
+----------------------+
1 row in set (0.000 sec)
MariaDB [primary_db]> select * from my_table;
+------+
| t    |
+------+
|    5 |
|   15 |
|   25 |
+------+
3 rows in set (0.000 sec)

尝试在主库上插入新数据

MariaDB [primary_db]> insert into my_table values (40),(45);

检查副本 mariadb-sts-3

MariaDB [primary_db]> select * from my_table;
+------+
| t    |
+------+
|    5 |
|   15 |
|   25 |
|   40 |
|   45 |
+------+
5 rows in set (0.000 sec)

我们还可以检查 PVC

$ kubectl get pvc
NAME                    STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
datadir-mariadb-sts-0   Bound    pvc-c1676027-6c75-473b-9b46-3d9a7d370fdc   300M       RWO            standard       31m
datadir-mariadb-sts-1   Bound    pvc-7f969265-3d8f-4677-950b-271ea670321e   300M       RWO            standard       30m
datadir-mariadb-sts-2   Bound    pvc-d116e494-5078-46ec-abcb-864fa8ae6b59   300M       RWO            standard       30m
datadir-mariadb-sts-3   Bound    pvc-00aca08a-7a5e-459a-8ae1-a854c4171a27   300M       RWO            standard       28m

缩容

$ kubectl scale sts mariadb-sts --replicas=2
statefulset.apps/mariadb-sts scaled

$ kubectl get pod -w
mariadb-sts-3   1/1     Terminating       0          78s
mariadb-sts-3   0/1     Terminating       0          79s
mariadb-sts-3   0/1     Terminating       0          79s
mariadb-sts-3   0/1     Terminating       0          79s
mariadb-sts-2   1/1     Terminating       0          4m4s
mariadb-sts-2   0/1     Terminating       0          4m5s
mariadb-sts-2   0/1     Terminating       0          4m5s
mariadb-sts-2   0/1     Terminating       0          4m5s

请注意，缩容不会删除 PVC。