Kuberntes部署MetalLB负载均衡器

MetalLB简介

MetalLB是一个为基础 Kubernetes集群提供负载均衡实现的工具，使用标准路由协议。

Kubernetes在基础集群中不提供网络负载均衡器（类型为LoadBalancer的服务）的实现。Kubernetes提供的网络负载平衡器实现都是调用各种IaaS平台（如GCP、AWS、Azure等）的接口代码。如果您没有运行在受支持的IaaS平台上（如GCP、AWS、Azure等），则创建时LoadBalancers将无限期处于“挂起”状态。

在基础集群中，操作员只有两个接口来将用户流量引入他们的集群，“NodePort”和“externalIPs”服务。

这两个选项在生产使用中都有显著的缺点，这使得基础集群成为 Kubernetes 生态系统中的二等公民。

MetalLB旨在通过提供与标准网络设备集成的网络负载均衡器实现来解决这种不平衡，以便基础群集上的external services尽可能“正常工作”。

1
2
3
4
5
6
7
root@master:~# kubectl get nodes -o wide
NAME           STATUS   ROLES           AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE           KERNEL-VERSION     CONTAINER-RUNTIME
k8s-master01   Ready    control-plane   18h   v1.30.2   192.168.1.31   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18
k8s-master02   Ready    control-plane   18h   v1.30.2   192.168.1.32   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18
k8s-master03   Ready    control-plane   18h   v1.30.2   192.168.1.33   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18
k8s-node01     Ready    <none>          18h   v1.30.2   192.168.1.34   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18
k8s-node02     Ready    <none>          18h   v1.30.2   192.168.1.35   <none>        Ubuntu 24.04 LTS   6.8.0-35-generic   containerd://1.7.18

修改kube-system

如果您正在使用IPVS模式下的kube-proxy，则自Kubernetes v1.14.2起，您必须启用严格的ARP模式。

请注意，如果您使用kube-router作为服务代理，则不需要此操作，因为它默认启用了 strict ARP。

您可以通过编辑当前集群中的kube-proxy配置来实现此操作：

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# kubeadm 部署方式 修改kube-system
kubectl get configmap kube-proxy -n kube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | \
kubectl apply -f - -n kube-system


# 二进制 部署方式 修改kube-system
cat > /etc/kubernetes/kube-proxy.yaml << EOF
apiVersion: kubeproxy.config.k8s.io/v1alpha1
bindAddress: 0.0.0.0
clientConnection:
  acceptContentTypes: ""
  burst: 10
  contentType: application/vnd.kubernetes.protobuf
  kubeconfig: /etc/kubernetes/kube-proxy.kubeconfig
  qps: 5
clusterCIDR: 172.16.0.0/12,fc00:2222::/112
configSyncPeriod: 15m0s
conntrack:
  max: null
  maxPerCore: 32768
  min: 131072
  tcpCloseWaitTimeout: 1h0m0s
  tcpEstablishedTimeout: 24h0m0s
enableProfiling: false
healthzBindAddress: 0.0.0.0:10256
hostnameOverride: ""
iptables:
  masqueradeAll: false
  masqueradeBit: 14
  minSyncPeriod: 0s
  syncPeriod: 30s
ipvs:
  strictARP: true
  masqueradeAll: true
  minSyncPeriod: 5s
  scheduler: "rr"
  syncPeriod: 30s
kind: KubeProxyConfiguration
metricsBindAddress: 127.0.0.1:10249
mode: "ipvs"
nodePortAddresses: null
oomScoreAdj: -999
portRange: ""
udpIdleTimeout: 250ms
EOF

systemctl restart kube-proxy
systemctl status kube-proxy

部署metallb

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# 下载应用包
wget https://mirrors.chenby.cn/https://github.com/metallb/metallb/archive/refs/tags/v0.14.5.tar.gz
tar -zxvf v0.14.5.tar.gz
cd metallb-0.14.5/config/manifests

# 修改镜像地址
sed -i "s#quay.io#quay.chenby.cn#g" metallb-native.yaml 
cat metallb-native.yaml | grep image
        image: quay.chenby.cn/metallb/controller:v0.14.5
        image: quay.chenby.cn/metallb/speaker:v0.14.5
        
# 执行部署
kubectl apply -f metallb-native.yaml


root@k8s-master01:~# kubectl -n metallb-system get all 
NAME                              READY   STATUS    RESTARTS   AGE
pod/controller-6975f6bf7b-nm2d6   1/1     Running   0          23m
pod/speaker-4jtb4                 1/1     Running   0          23m
pod/speaker-fpd6q                 1/1     Running   0          23m
pod/speaker-mmfxq                 1/1     Running   0          23m
pod/speaker-rxs2b                 1/1     Running   0          23m
pod/speaker-sfxvb                 1/1     Running   0          23m

NAME                              TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)   AGE
service/metallb-webhook-service   ClusterIP   10.96.95.84   <none>        443/TCP   23m

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/speaker   5         5         5       5            5           kubernetes.io/os=linux   23m

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/controller   1/1     1            1           23m

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/controller-6975f6bf7b   1         1         1       23m
root@k8s-master01:~#

创建池

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# 新版本metallb使用了CR（Custom Resources），这里我们通过IPAddressPool的CR，进行地址池的定义。
# 如果实例中不设置IPAddressPool选择器L2Advertisement；那么L2Advertisement默认为该实例所有的IPAddressPool相关联。

cat > metallb-config-ipaddresspool.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.70-192.168.1.79
EOF

# 进行L2关联地址池的绑定。

cat > metallb-config-L2Advertisement.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system
spec:
  ipAddressPools:
  - first-pool
EOF

# 执行部署
kubectl apply -f metallb-config-ipaddresspool.yaml
kubectl apply -f metallb-config-L2Advertisement.yaml

测试

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
# 测试
cat > metallb-nginx.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: default
spec:
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx
        image: nginx:1
        ports:
        - name: http
          containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  ports:
  - name: http
    port: 80
    protocol: TCP
    targetPort: 80
  selector:
    app: nginx
  type: LoadBalancer
EOF

# 执行部署
kubectl apply -f metallb-nginx.yaml

查看

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
# 查看
# EXTERNAL-IP已经获取到IP地址
root@k8s-master01:~# kubectl get all
NAME                                READY   STATUS    RESTARTS      AGE
pod/nginx-648c475cfb-5pvvv          1/1     Running   0             4m24s

NAME                   TYPE           CLUSTER-IP        EXTERNAL-IP    PORT(S)        AGE
service/nginx          LoadBalancer   10.96.197.147     192.168.1.51   80:30752/TCP   4m24s

NAME                           READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/nginx          1/1     1            1           4m24s

NAME                                      DESIRED   CURRENT   READY   AGE
replicaset.apps/nginx-648c475cfb          1         1         1       4m24s
root@k8s-master01:~#

访问

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# 访问
root@k8s-master01:~# curl 192.168.1.51
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
html { color-scheme: light dark; }
body { width: 35em; margin: 0 auto;
font-family: Tahoma, Verdana, Arial, sans-serif; }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>
root@k8s-master01:~#

关于

https://www.oiox.cn/

https://www.oiox.cn/index.php/start-page.html

CSDN、GitHub、51CTO、知乎、开源中国、思否、博客园、掘金、简书、华为云、阿里云、腾讯云、哔哩哔哩、今日头条、新浪微博、个人博客

全网可搜《小陈运维》

文章主要发布于微信公众号