1.概念

etcd 是由GO语言编写的分布式的、可靠的键值存储系统，主要用于分布式系统中关键数据的存储和服务发现。

2.核心概念

节点（Node）

每个运行 etcd 的实例被称为一个节点。一个或多个节点可以组成一个集群。

集群（Cluster）

由多个节点组成的集合，这些节点共同工作以提供一致的数据存储服务。通过 Raft 共识算法确保集群中各节点间数据的一致性。

键值对（Key-Value Pair）

etcd 存储的基本单位是键值对，其中键和值都是字节数组。键用于唯一标识存储的数据项，而值则包含实际的数据内容。

3.etcd集群准备

节点 IP地址 操作系统版本 etcd版本
etcd-node1 192.168.100.5 Ubuntu 24.04.2 LTS v3.6.4
etcd-node2 192.168.100.6 Ubuntu 24.04.2 LTS v3.6.4
etcd-node3 192.168.100.7 Ubuntu 24.04.2 LTS v3.6.4

3.1 配置IP地址

配置etcd-node1节点IP

sudo cat /etc/netplan/ens32-cloud-init.yaml 
network:version: 2ethernets:ens32:dhcp4: falseaddresses:- "192.168.100.5/24"routes:- to: defaultvia: 192.168.100.254nameservers:addresses:- 114.114.114.114
sudo netplan apply

配置etcd-node2节点IP

sudo cat /etc/netplan/ens32-cloud-init.yaml 
network:version: 2ethernets:ens32:dhcp4: falseaddresses:- "192.168.100.6/24"routes:- to: defaultvia: 192.168.100.254nameservers:addresses:- 114.114.114.114
sudo netplan apply

配置etcd-node3节点IP

sudo cat /etc/netplan/ens32-cloud-init.yaml 
network:version: 2ethernets:ens32:dhcp4: falseaddresses:- "192.168.100.7/24"routes:- to: defaultvia: 192.168.100.254nameservers:addresses:- 114.114.114.114
sudo netplan apply

3.2 配置主机名

配置etcd-node1节点主机名

sudo hostnamectl set-hostname etcd-node1

配置etcd-node2节点主机名

sudo hostnamectl set-hostname etcd-node2

配置etcd-node3节点主机名

sudo hostnamectl set-hostname etcd-node3

3.3 配置主机名与IP解析

3个节点均需要执行

sudo cat >> /etc/hosts <<EOF
192.168.100.5 etcd-node1
192.168.100.6 etcd-node2
192.168.100.7 etcd-node3
EOF

3.4 关闭防火墙

3个节点均需要执行

sudo ufw stop
sudo ufw status

3.5 时钟同步

3个节点均需要执行

sudo apt install chrony
sudo sed -i '/pool.*ubuntu\.pool\.ntp\.org/s/^/# /' /etc/chrony/chrony.conf
sudo sed -i 's/^pool ntp\.ubuntu\.com.*$/server ntp.aliyun.com iburst/' /etc/chrony/chrony.conf
sudo systemctl restart chrony
sudo chronyc sources

3.6 配置节点互信

3个节点均需要执行

sudo ssh-keygen
sudo cp /root/.ssh/id_rsa.pub /root/.ssh/authorized_keys

在etcd-node1节点上执行

sudo for i in 5 6 7
> do
> scp -r /root/.ssh 192.168.100.$i:/root/
> done

4.etcd集群部署

4.1 下载etcd

3个节点均需要执行

sudo wget https://github.com/etcd-io/etcd/releases/download/v3.6.4/etcd-v3.6.4-linux-amd64.tar.gz

4.2解压etcd

3个节点均需要执行

sudo tar xzvf etcd-v3.6.4-linux-amd64.tar.gz -C /usr/local
sudo ln -s /usr/local/etcd-v3.6.4-linux-amd64/ /usr/local/etcd

4.3复制解压文件至系统标准可执行文件路径中

3个节点均需要执行

sudo cp /usr/local/etcd/etcd* /usr/local/bin/

4.4创建etcd用户

3个节点均需要执行

sudo useradd --system --shell /bin/false --home-dir /var/lib/etcd etcd

4.5创建数据目录

3个节点均需要执行

sudo mkdir -p /var/lib/etcd 
sudo mkdir /var/lib/etcd/default.etcd
sudo chown -R etcd:etcd /var/lib/etcd /usr/local/etcd

4.6 创建etcd配置文件

配置etcd-node1节点配置文件

sudo cat > /usr/local/etcd/etcd.conf <<EOF
ETCD_NAME="etcd-node1"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.100.5:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.100.5:2379,http://127.0.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.5:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.5:2379"
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.100.5:2380,etcd-node2=https://192.168.100.6:2380,etcd-node3=https://192.168.100.7:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

配置etcd-node2节点配置文件

sudo cat > /usr/local/etcd/etcd.conf <<EOF
ETCD_NAME="etcd-node2"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.100.6:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.100.6:2379,http://127.0.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.6:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.6:2379"
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.100.5:2380,etcd-node2=https://192.168.100.6:2380,etcd-node3=https://192.168.100.7:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

配置etcd-node3节点配置文件

sudo cat > /usr/local/etcd/etcd.conf <<EOF
ETCD_NAME="etcd-node3"
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="https://192.168.100.7:2380"
ETCD_LISTEN_CLIENT_URLS="https://192.168.100.7:2379,http://127.0.0.1:2379"#[Clustering]
ETCD_INITIAL_ADVERTISE_PEER_URLS="https://192.168.100.7:2380"
ETCD_ADVERTISE_CLIENT_URLS="https://192.168.100.7:2379"
ETCD_INITIAL_CLUSTER="etcd-node1=https://192.168.100.5:2380,etcd-node2=https://192.168.100.6:2380,etcd-node3=https://192.168.100.7:2380"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_INITIAL_CLUSTER_STATE="new"
EOF

相关参数说明

参数 说明
ETCD_NAME 当前etcd节点名称
ETCD_DATA_DIR 数据存储目录
ETCD_LISTEN_CLIENT_URLS 当前节点通过该地址监听客户端发送的信息
ETCD_LISTEN_PEER_URLS 当前节点通过该地址监听集群其他节点发送的信息
ETCD_INITIAL_ADVERTISE_PEER_URLS 集群的其他节点通过该地址与当前节点通信
ETCD_ADVERTISE_CLIENT_URLS 客户端通过该地址与当前节点通信
ETCD_INITIAL_CLUSTER 当前集群的所有节点信息，当前节点根据此信息与其他节点取得联系
ETCD_INITIAL_CLUSTER_TOKEN 用于区分不同的集群，同一集群的所有节点配置相同的值
ETCD_INITIAL_CLUSTER_STATE 本次是否为新建集群，取值为 new 或者 existing

4.7下载cfssl 证书生成工具

在etcd-node1节点上下载cfssl

wget https://github.com/cloudflare/cfssl/releases/download/v1.6.5/cfssl_1.6.5_linux_amd64

在etcd-node1节点上下载cfssljson

wget https://github.com/cloudflare/cfssl/releases/download/v1.6.5/cfssljson_1.6.5_linux_amd64

4.8授权并移至系统标准可执行文件路径中

在etcd-node1节点上执行

 chmod +x cfssl_1.6.5_linux_amd64chmod +x cfssljson_1.6.5_linux_amd64mv cfssl_1.6.5_linux_amd64 /usr/local/bin/cfssljsonmv cfssljson_1.6.5_linux_amd64 /usr/local/bin/cfssljson

4.9创建CA证书

在etcd-node1节点上配置CA证书策略

mkdir /usr/local/etcd/ssl
cat > /usr/local/etcd/ssl/ca-config.json <<EOF
{"signing": {"default": {"expiry": "87600h"},"profiles": {"etcd-server": {"usages": ["signing","key encipherment","client auth","server auth"],"expiry": "87600h"}}}
}
EOF

在etcd-node1节点上配置CA证书请求文件

cat > /usr/local/etcd/ssl/ca-csr.json <<EOF 
{"CN": "My etcd CA","key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "LANZHOU","O": "LZ","ST": "LANZHOU","OU": "CN"}],"ca": {"expiry": "87600h"}
}
EOF

在etcd-node1节点上生成CA证书

cfssl gencert -initca ca-csr.json | cfssljson -bare ca -

4.10创建etcd证书

在etcd-node1节点上配置etcd请求文件

cat > /usr/local/etcd/ssl/etcd-server.json  <<EOF
{"CN": "etcd","hosts": ["127.0.0.1","192.168.100.5","192.168.100.6","192.168.100.7"],"key": {"algo": "rsa","size": 2048},"names": [{"C": "CN","L": "LANZHOU","ST": "LANZHOU","OU": "CN"}]
}
EOF

在etcd-node1节点上生成 Etcd 证书

cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=etcd-server etcd-server.json | cfssljson -bare etcd-server

说明：

参数 说明
-ca-key 指定CA证书机构的私钥
-config 指定CA证书策略
-profile 指定使用CA证书策略
etcd-server.pem 证书/公钥
etcd-server-key.pem 私钥

4.7创建 systemd 服务

3个节点均需要执行

sudo cat > /usr/lib/systemd/system/etcd.service <<EOF
[Unit]
Description=Etcd Server
After=network.target
After=network-online.target
Wants=network-online.target[Service]
Type=notify
EnvironmentFile=-/usr/local/etcd/etcd.conf
WorkingDirectory=/var/lib/etcd/
ExecStart=/usr/local/bin/etcd \--cert-file=/usr/local/etcd/ssl/etcd-server.pem \--key-file=/usr/local/etcd/ssl/etcd-server-key.pem \--trusted-ca-file=/usr/local/etcd/ssl/ca.pem \--peer-cert-file=/usr/local/etcd/ssl/etcd-server.pem \--peer-key-file=/usr/local/etcd/ssl/etcd-server-key.pem \--peer-trusted-ca-file=/usr/local/etcd/ssl/ca.pem \--peer-client-cert-auth \--client-cert-auth
Restart=on-failure
RestartSec=5
LimitNOFILE=65536[Install]
WantedBy=multi-user.target
EOF

4.8 启动etcd

3个节点均需要执行

sudo systemctl daemon-reload
sudo systemctl enable etcd
sudo systemctl start etcd
sudo systemctl status etcd

4.9查看集群成员

任意节点执行

sudo etcdctl member list

4.10查看集群节点健康状态

ETCDCTL_API=3 /usr/local/bin/etcdctl --write-out=table --cacert=/usr/local/etcd/ssl/ca.pem --cert=/usr/local/etcd/ssl/etcd-server.pem --key=/usr/local/etcd/ssl/etcd-server-key.pem --endpoints=https://192.168.100.5:2379,https://192.168.100.6:2379,https://192.168.100.7:2379 endpoint health