Kubernetes核心-Ingress-metallb

使用一种能感知协议配置的机制来解析 URI、主机名称、路径等 Web 概念， 让你的 HTTP（或 HTTPS）网络服务可被访问。 Ingress 概念允许你通过 Kubernetes API 定义的规则将流量映射到不同后端。

部署metallb

MetalLB 是Kubernetes 的一个裸机环境下的负载均衡器，它为LoadBalancer 类型的Service 提供IP 地址分配和对外流量广播，从而使裸机Kubernetes 集群也能像云环境一样通过外部IP 访问内部服务

下载部署

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# 我可以连接国际网络，就直接部署了
kubectl apply -f https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml

# 网络不通畅可以使用手动修改镜像地址

# 下载yaml配置
wget https://raw.githubusercontent.com/metallb/metallb/v0.15.2/config/manifests/metallb-native.yaml

# 修改镜像地址
# 自行找代理
sed -i "s#quay.io#quay.chenby.cn#g" metallb-native.yaml 
cat metallb-native.yaml | grep image
        image: quay.chenby.cn/metallb/controller:v0.14.5
        image: quay.chenby.cn/metallb/speaker:v0.14.5
        
# 执行部署
kubectl apply -f metallb-native.yaml

查看运行情况

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
root@k8s-master01:~# kubectl -n metallb-system get all 
NAME                              READY   STATUS    RESTARTS   AGE
pod/controller-6599cd9c46-rr54w   1/1     Running   0          78s
pod/speaker-55j5t                 1/1     Running   0          78s
pod/speaker-bcr4j                 1/1     Running   0          78s
pod/speaker-p7vgz                 1/1     Running   0          78s
pod/speaker-pzvkd                 1/1     Running   0          78s
pod/speaker-vcjvr                 1/1     Running   0          78s

NAME                              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/metallb-webhook-service   ClusterIP   10.106.20.159   <none>        443/TCP   78s

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/speaker   5         5         5       5            5           kubernetes.io/os=linux   78s

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/controller   1/1     1            1           78s

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/controller-6599cd9c46   1         1         1       78s

配置VIP的资源池

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
# 新版本metallb使用了CR（Custom Resources），这里我们通过IPAddressPool的CR，进行地址池的定义。
# 如果实例中不设置IPAddressPool选择器L2Advertisement；那么L2Advertisement默认为该实例所有的IPAddressPool相关联。

cat > metallb-config-ipaddresspool.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: first-pool
  namespace: metallb-system
spec:
  addresses:
  - 192.168.1.71-192.168.1.75
EOF

# 进行L2关联地址池的绑定。

cat > metallb-config-L2Advertisement.yaml << EOF
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: example
  namespace: metallb-system
spec:
  ipAddressPools:
  - first-pool
EOF

# 执行部署
kubectl apply -f metallb-config-ipaddresspool.yaml
kubectl apply -f metallb-config-L2Advertisement.yaml

Ingress安装

执行部署

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
# 添加仓库
helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
helm repo update

# 拉取仓库
helm pull ingress-nginx/ingress-nginx
tar xvf ingress-nginx-4.12.3.tgz

# 查看镜像地址，若你的环境无法下载这个镜像，那么就需要你自行找镜像 或者更换镜像地址
[root@k8s-master01 ingress-nginx]# cat  values.yaml | grep image | grep -v \#
  image:
  image:
    image: ingress-nginx/controller
      image:
        image: ingress-nginx/kube-webhook-certgen
  image:
    image: defaultbackend-amd64
imagePullSecrets: []
[root@k8s-master01 ingress-nginx]# 

# 我这里的环境可以直接拉取 我直接进行了安装
helm install ingress-nginx ingress-nginx/ingress-nginx \
  --namespace ingress --create-namespace

# 查看完成安装
[root@k8s-master01 ~]# kubectl get all -n ingress
NAME                                            READY   STATUS    RESTARTS   AGE
pod/ingress-nginx-controller-6996967cbb-fplzt   1/1     Running   0          56s

NAME                                         TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                      AGE
service/ingress-nginx-controller             LoadBalancer   10.109.63.223    192.168.1.71   80:32046/TCP,443:31512/TCP   56s
service/ingress-nginx-controller-admission   ClusterIP      10.110.227.191   <none>         443/TCP                      56s

NAME                                       READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/ingress-nginx-controller   1/1     1            1           56s

NAME                                                  DESIRED   CURRENT   READY   AGE
replicaset.apps/ingress-nginx-controller-6996967cbb   1         1         1       56s
[root@k8s-master01 ~]#

创建测试镜像

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16

cat > Dockerfile  << EOF
FROM nginx
RUN echo 'nginx-v1' > /usr/share/nginx/html/index.html
EOF
docker build -t registry.cn-hangzhou.aliyuncs.com/chenby/cby:nginx-v1 .

cat > Dockerfile  << EOF
FROM nginx
RUN echo 'nginx-v2' > /usr/share/nginx/html/index.html
EOF
docker build -t registry.cn-hangzhou.aliyuncs.com/chenby/cby:nginx-v2 .


docker push registry.cn-hangzhou.aliyuncs.com/chenby/cby:nginx-v1
docker push registry.cn-hangzhou.aliyuncs.com/chenby/cby:nginx-v2

创建测试应用

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
cat > ingress-demo-app.yaml << EOF
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cby-nginx-v1
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cby-nginx-v1
  template:
    metadata:
      labels:
        app: cby-nginx-v1
    spec:
      containers:
      - name: cby-nginx-v1
        image: registry.cn-hangzhou.aliyuncs.com/chenby/cby:nginx-v1
        ports:
        - containerPort: 9000
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: cby-nginx-v2
  name: cby-nginx-v2
spec:
  replicas: 2
  selector:
    matchLabels:
      app: cby-nginx-v2
  template:
    metadata:
      labels:
        app: cby-nginx-v2
    spec:
      containers:
      - image: registry.cn-hangzhou.aliyuncs.com/chenby/cby:nginx-v2
        name: nginx
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: cby-nginx-v2
  name: cby-nginx-v2
spec:
  selector:
    app: cby-nginx-v2
  ports:
  - port: 8000
    protocol: TCP
    targetPort: 80
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: cby-nginx-v1
  name: cby-nginx-v1
spec:
  selector:
    app: cby-nginx-v1
  ports:
  - port: 8000
    protocol: TCP
    targetPort: 80
EOF
# 创建路由
cat >> ingress-demo-app-ingress.yaml <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
  - host: "nginx-v1.chenby.cn"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: cby-nginx-v1
            port:
              number: 8000
  - host: "nginx-v2.chenby.cn"
    http:
      paths:
      - pathType: Prefix
        path: "/"  
        backend:
          service:
            name: cby-nginx-v2
            port:
              number: 8000
EOF

# 等创建完成后在执行：
kubectl  apply -f ingress-demo-app.yaml 
kubectl  apply -f ingress-demo-app-ingress.yaml 

# 查看ING
kubectl  get ingress
NAME                 CLASS   HOSTS                                   ADDRESS        PORTS   AGE
ingress-host-bar     nginx   nginx-v1.chenby.cn,nginx-v2.chenby.cn   192.168.1.71   80      69m


测试访问

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
# 写入hosts
cat >> /etc/hosts <<EOF
192.168.1.71 nginx-v1.chenby.cn
192.168.1.71 nginx-v2.chenby.cn
EOF


# 访问 v1 正常
[root@localhost ~]# curl nginx-v1.chenby.cn
nginx-v1
[root@localhost ~]# 

# # 访问 v2 正常
[root@localhost ~]# curl nginx-v2.chenby.cn
nginx-v2
[root@localhost ~]#

测试路径重写

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
# 删除刚建的ingress

kubectl  delete -f ingress-demo-app-ingress.yaml 

# 写入新的ingress
cat >> ingress-demo-app-ingress-router.yaml <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress  
metadata:
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$2
  name: ingress-host-bar
spec:
  ingressClassName: nginx
  rules:
  - host: "nginx-v1.chenby.cn"
    http:
      paths:
      - pathType: Prefix
        path: "/"
        backend:
          service:
            name: cby-nginx-v1
            port:
              number: 8000
  - host: "nginx-v2.chenby.cn"
    http:
      paths:
      - pathType: Prefix
        path: "/nginx(/|$)(.*)" 
        pathType: ImplementationSpecific
        backend:
          service:
            name: cby-nginx-v2  
            port:
              number: 8000
EOF

# 执行部署
kubectl  apply -f ingress-demo-app-ingress-router.yaml 


# 访问 v1 返回正常 
[root@localhost ~]# curl nginx-v1.chenby.cn
nginx-v1
[root@localhost ~]#

# 访问 v2 访问异常
[root@localhost ~]# curl nginx-v2.chenby.cn 
<html>
<head><title>404 Not Found</title></head>
<body>
<center><h1>404 Not Found</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@localhost ~]#

# 访问时加上URI 访问正常
[root@localhost ~]# curl nginx-v2.chenby.cn/nginx
nginx-v2
[root@localhost ~]#

测试流量限制

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66

# 删除刚建的ingress

kubectl  delete -f ingress-demo-app-ingress-router.yaml

# 写入新的ingress
cat >> ingress-demo-app-ingress-limit.yaml <<EOF
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-limit-rate
  annotations:
    nginx.ingress.kubernetes.io/limit-rps: "1"
spec:
  ingressClassName: nginx
  rules:
  - host: "nginx-v1.chenby.cn"
    http:
      paths:
      - pathType: Exact
        path: "/"
        backend:
          service:
            name: cby-nginx-v1
            port:
              number: 8000
  - host: "nginx-v2.chenby.cn"
    http:
      paths:
      - pathType: Exact
        path: "/"  
        backend:
          service:
            name: cby-nginx-v2
            port:
              number: 8000
EOF

# 执行部署
kubectl  apply -f ingress-demo-app-ingress-limit.yaml 


# 访问过快 会返回 503
[root@localhost ~]# for i in {1..8}; do curl nginx-v1.chenby.cn; done
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
nginx-v1
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>
<html>
<head><title>503 Service Temporarily Unavailable</title></head>
<body>
<center><h1>503 Service Temporarily Unavailable</h1></center>
<hr><center>nginx</center>
</body>
</html>
[root@localhost ~]#

过滤查看ingress端口

上面有安装metallb组件，所有ingress有自动获取到地址，我这里就不修改nodeport了，各位如果没有metallb组件 就需要修改 nodeport

1
2
3
4
5
6
7
8
# 修改为nodeport
kubectl edit svc -n ingress-nginx   ingress-nginx-controller
type: NodePort

[root@hello ~/yaml]# kubectl  get svc -A | grep ingress
ingress-nginx      ingress-nginx-controller               LoadBalancer   10.110.161.30    192.168.1.71   80:32480/TCP,443:30195/TCP   12m
ingress-nginx      ingress-nginx-controller-admission     ClusterIP      10.105.211.217   <none>         443/TCP                      12m
[root@hello ~/yaml]#

关于

https://www.oiox.cn/

https://www.oiox.cn/index.php/start-page.html

CSDN、GitHub、知乎、开源中国、思否、掘金、简书、华为云、阿里云、腾讯云、哔哩哔哩、今日头条、新浪微博、个人博客

全网可搜《小陈运维》

文章主要发布于微信公众号：《Linux运维交流社区》