ingress的使用

最新推荐文章于 2025-05-12 23:45:33 发布

Hurry6

最新推荐文章于 2025-05-12 23:45:33 发布

阅读量1.7k

点赞数 44

CC 4.0 BY-SA版权

文章标签： kubernetes 云原生 ingress

本文链接：https://blog.csdn.net/Hurry6/article/details/147192684

Service对集群之外暴露服务的主要方式有两种：NotdePort和 LoadBalancer，但是这两种方式，都有一定的缺点：

NodePort方式的缺点是会占用很多集群机器的端口，那么当集群服务变多的时候，这个缺点就愈发明显
LB方式的缺点是每个service需要一个LB，浪费、麻烦，并且需要kubernetes之外设备的支持

基于这种现状，kubernetes提供了Ingress资源对象，Ingress只需要一个NodePort或者一个LB就可以满足暴露多个Service的需求。工作机制大致如下图表示：

实际上，Ingress相当于一个7层的负载均衡器，是kubernetes对反向代理的一个抽象，它的工作原理类似于Nginx，可以理解成在Ingress里建立诸多映射规则，Ingress Controller通过监听这些配置规则并转化成Nginx的反向代理配置 , 然后对外部提供服务。在这里有两个核心概念：

ingress：kubernetes中的一个对象，作用是定义请求如何转发到service的规则
ingress controller：具体实现反向代理及负载均衡的程序，对ingress定义的规则进行解析，根据配置的规则来实现请求转发，实现方式有很多，比如Nginx, Contour, Haproxy等等

Ingress（以Nginx为例）的工作原理如下：

1. 用户编写Ingress规则，说明哪个域名对应kubernetes集群中的哪个Service
2. Ingress控制器动态感知Ingress服务规则的变化，然后生成一段对应的Nginx反向代理配置
3. Ingress控制器会将生成的Nginx配置写入到一个运行着的Nginx服务中，并动态更新
4. 到此为止，其实真正在工作的就是一个Nginx了，内部配置了用户定义的请求转发规则

环境准备

下载ingress部署的yaml文件

wget https://github.com/kubernetes/ingressnginx/blob/main/deploy/static/provider/cloud/deploy.yaml

修改镜像地址为阿里云地址，总三处

registry.k8s.io/ingress-nginx/controller:v1.12.1

改为 registry.cnhangzhou.aliyuncs.com/google_containers/nginx-ingresscontroller:v1.12.1

k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.1.1 改为（有两处需要修改） registry.cn-hangzhou.aliyuncs.com/google_containers/kube-webhook-certgen:v1.5.2

修改type: ClusterIP 为 type: NodePort (可选)

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.12.1
  name: ingress-nginx-controller
  namespace: ingress-nginx
  annotations:   #添加注解
    lb.kubesphere.io/v1alpha1: openelb
    protocol.openelb.kubesphere.io/v1alpha1: layer2
    eip.openelb.kubesphere.io/v1alpha2: eip-pool
spec:
  externalTrafficPolicy: Local
  ipFamilies:
  - IPv4
  ipFamilyPolicy: SingleStack
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https
    port: 443
    protocol: TCP
    targetPort: https
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  type: LoadBalancer
---  
apiVersion: v1  
kind: Service
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.12.1
  name: ingress-nginx-controller-admission
  namespace: ingress-nginx
spec:
  ports:
  - appProtocol: http
    name: http
    port: 80
    protocol: TCP
    targetPort: http
  - appProtocol: https
    name: https-webhook
    port: 443
    targetPort: webhook
  selector:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
  #type: ClusterIP 
  type: NodePort   #添加修改

部署ingress-nginx

[root@k8s-master01 dev]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created


[root@k8s-master01 dev]# kubectl get svc -n ingress-nginx
NAME                                 TYPE           CLUSTER-IP    EXTERNAL-IP       PORT(S)                      AGE
ingress-nginx-controller             LoadBalancer   10.96.0.243   192.168.234.201   80:31527/TCP,443:30862/TCP   51s
ingress-nginx-controller-admission   NodePort       10.96.3.118   <none>            80:30689/TCP,443:32072/TCP   51s

[root@k8s-master01 dev]# kubectl get pod  -n ingress-nginx
NAME                                       READY   STATUS      RESTARTS   AGE
ingress-nginx-admission-create-8bqd4       0/1     Completed   0          6m31s
ingress-nginx-admission-patch-vxx2h        0/1     Completed   3          6m31s
ingress-nginx-controller-75cdf6c95-wxgk6   1/1     Running     0          6m31s

Ingress暴露服务的方式

方式一：Deployment+LoadBalancer模式的Service

如果要把ingress部署在公有云，那用这种方式比较合适。用Deployment部署ingresscontroller，创建一个type为 LoadBalancer的 service关联这组pod。大部分公有云，都会为 LoadBalancer的 service自动创建一个负载均衡器，通常还绑定了公网地址。只要把域名解析指向该地址，就实现了集群服务的对外暴露

方式二：DaemonSet+HostNetwork+nodeselector

用DaemonSet结合nodeselector来部署ingress-controller到特定的node 上，然后使用 HostNetwork直接把该pod与宿主机node的网络打通，直接使用宿主机的80/433端口就能访问服务。这时，ingress-controller所在的node机器就很类似传统架构的边缘节点，比如机房入口的 nginx服务器。该方式整个请求链路最简单，性能相对NodePort模式更好。缺点是由于直接利用宿主机节点的网络和端口，一个node只能部署一个ingress-controller pod。比较适合大并发的生产环境使用

优点

该方式整个请求链路最简单，性能相对NodePort模式更好

缺点

由于直接利用宿主机节点的网络和端口，一个node只能部署一个 ingress-controller pod

方式三：Deployment+NodePort模式的Service

同样用deployment模式部署ingres-controller，并创建对应的服务，但是type为NodePort。这样，ingress就会暴露在集群节点ip的特定端口上。由于nodeport暴露的端口是随机端口，一般会在前面再搭建一套负载均衡器来转发请求。该方式一般用于宿主机是相对固定的环境ip地址不变的场景。NodePort方式暴露ingress虽然简单方便，但是NodePort多了一层NAT，在请求量级很大时可能对性能会有一定影响

实战1：

#指定nginx-ingress-controller运行在node2节点
[root@k8s-master01 dev]# kubectl label nodes k8s-worker02 ingress=true
node/k8s-worker02 labeled

[root@k8s-master01 dev]# kubectl  get  nodes --show-labels
NAME           STATUS   ROLES           AGE     VERSION   LABELS
k8s-master01   Ready    control-plane   3h59m   v1.30.5   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-master01,kubernetes.io/os=linux,node-role.kubernetes.io/control-plane=,node.kubernetes.io/exclude-from-external-load-balancers=
k8s-worker01   Ready    <none>          3h58m   v1.30.5   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-worker01,kubernetes.io/os=linux
k8s-worker02   Ready    <none>          3h58m   v1.30.5   beta.kubernetes.io/arch=amd64,beta.kubernetes.io/os=linux,ingress=true,kubernetes.io/arch=amd64,kubernetes.io/hostname=k8s-worker02,kubernetes.io/os=linux


#修改Deployment为Daemonset,指定节点运行，并开启hostNetwork
[root@k8s-master01 dev]# kubectl delete -f deploy.yaml
namespace "ingress-nginx" deleted
serviceaccount "ingress-nginx" deleted
serviceaccount "ingress-nginx-admission" deleted
role.rbac.authorization.k8s.io "ingress-nginx" deleted
role.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
clusterrole.rbac.authorization.k8s.io "ingress-nginx" deleted
clusterrole.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
rolebinding.rbac.authorization.k8s.io "ingress-nginx" deleted
rolebinding.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
clusterrolebinding.rbac.authorization.k8s.io "ingress-nginx" deleted
clusterrolebinding.rbac.authorization.k8s.io "ingress-nginx-admission" deleted
configmap "ingress-nginx-controller" deleted
service "ingress-nginx-controller" deleted
service "ingress-nginx-controller-admission" deleted
deployment.apps "ingress-nginx-controller" deleted
job.batch "ingress-nginx-admission-create" deleted
job.batch "ingress-nginx-admission-patch" deleted
ingressclass.networking.k8s.io "nginx" deleted
validatingwebhookconfiguration.admissionregistration.k8s.io "ingress-nginx-admission" deleted


[root@k8s-master01 dev]# vim deploy.yaml


# 改控制器类型为DaemSet
...
apiversion: apps/vl
kind: DaemonSet   #修改kind
nodeSelector:
        # kubernetes.io/os: linux
          ingress: "true" #选择节点运行
      hostNetwork:  true

---
把这四行注释
# strategy:
#   rollingUpdate:
#     maxUnavailable: 1
#   type: RollingUpdate



#部署
[root@k8s-master01 dev]# kubectl apply -f deploy.yaml
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
daemonset.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created

[root@k8s-master01 dev]# kubectl get pod -n ingress-nginx -o wide
NAME                                   READY   STATUS      RESTARTS   AGE     IP               NODE           NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-r7xv2   0/1     Completed   0          4m26s   100.119.84.84    k8s-worker01   <none>           <none>
ingress-nginx-admission-patch-qxwlt    0/1     Completed   0          4m26s   100.119.84.83    k8s-worker01   <none>           <none>
ingress-nginx-controller-98z4x         1/1     Running     0          4m26s   192.168.234.14   k8s-worker02   <none>           <none>



#到节点2查看
[root@k8s-worker02 ~]# netstat  -lnput | grep nginx
tcp        0      0 0.0.0.0:8181            0.0.0.0:*               LISTEN      214692/nginx: maste
tcp        0      0 0.0.0.0:443             0.0.0.0:*               LISTEN      214692/nginx: maste
tcp        0      0 127.0.0.1:10245         0.0.0.0:*               LISTEN      214657/nginx-ingres
tcp        0      0 127.0.0.1:10246         0.0.0.0:*               LISTEN      214692/nginx: maste
tcp        0      0 127.0.0.1:10247         0.0.0.0:*               LISTEN      214692/nginx: maste
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      214692/nginx: maste
tcp6       0      0 :::8181                 :::*                    LISTEN      214692/nginx: maste
tcp6       0      0 :::8443                 :::*                    LISTEN      214657/nginx-ingres
tcp6       0      0 :::443                  :::*                    LISTEN      214692/nginx: maste
tcp6       0      0 :::10254                :::*                    LISTEN      214657/nginx-ingres
tcp6       0      0 :::80                   :::*                    LISTEN      214692/nginx: maste


#创建ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-ingress
  namespace: dev
spec:
  ingressClassName: nginx
  rules:
  - host: www.itopenlab.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: svc-lb
            port:
              number: 80

[root@k8s-master01 dev]# kubectl apply -f ingress1.yaml
ingress.networking.k8s.io/nginx-ingress created

[root@k8s-master01 dev]# kubectl get svc,ingress,pod -n dev
NAME             TYPE           CLUSTER-IP    EXTERNAL-IP       PORT(S)        AGE
service/svc-lb   LoadBalancer   10.96.3.126   192.168.234.200   80:32376/TCP   133m

NAME                                      CLASS   HOSTS               ADDRESS           PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.itopenlab.com   192.168.234.201   80      9m21s

NAME                                    READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-7cffcbf558-7z6hs   1/1     Running   0          4h10m
pod/nginx-deployment-7cffcbf558-9fpc2   1/1     Running   0          4h10m
pod/nginx-deployment-7cffcbf558-hjlqv   1/1     Running   0          4h10m

在节点2上
[root@k8s-worker02 ~]# vim /etc/hosts
192.168.234.201 www.itopenlab.com

[root@k8s-worker02 ~]# curl www.itopenlab.com
100.119.84.73 ,web1 test page
[root@k8s-worker02 ~]# curl www.itopenlab.com
100.73.45.71 ,web2 test page
[root@k8s-worker02 ~]# curl www.itopenlab.com
100.73.45.72 ,web3 test page

实战2：

创建tomcat-service.yaml

# --------------------- Deployment ---------------------
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deployment
  namespace: dev
spec:
  replicas: 2
  selector:
    matchLabels:
      app: tomcat-pod
  template:
    metadata:
      labels:
        app: tomcat-pod
    spec:
      containers:
        - name: tomcat
          image: tomcat:8.5-jre10-slim
          ports:
            - containerPort: 8080  # 确保 Tomcat 容器实际监听此端口
---
# --------------------- Service ---------------------
apiVersion: v1
kind: Service
metadata:
  name: tomcat-service
  namespace: dev
  annotations:
    lb.kubesphere.io/v1alpha1: openelb
    protocol.openelb.kubesphere.io/v1alpha2: layer2  # ✅ 版本升级至 v1alpha2
    eip.openelb.kubesphere.io/v1alpha2: eip-pool
spec:
  selector:
    app: tomcat-pod  # 必须与 Deployment 中的 Pod 标签匹配
  type: LoadBalancer
  ports:
    - port: 80       # 服务对外暴露的端口
      targetPort: 8080  # 必须与容器端口一致


[root@k8s-master01 dev]# vim tomcat-service.yaml
[root@k8s-master01 dev]# kubectl create  -f tomcat-service.yaml
deployment.apps/tomcat-deployment created
service/tomcat-service created
[root@k8s-master01 dev]# kubectl get svc -n dev
NAME             TYPE           CLUSTER-IP    EXTERNAL-IP       PORT(S)          AGE
svc-lb           LoadBalancer   10.96.3.126   192.168.234.200   80:32376/TCP     163m
tomcat-service   LoadBalancer   10.96.1.2     192.168.234.202   8080:31378/TCP   10s

再往ingress1.yaml修改

apiVersion: networking.k8s.io/v1
  2 kind: Ingress
  3 metadata:
  4   name: nginx-ingress
  5   namespace: dev
  6 spec:
  7   ingressClassName: nginx
  8   rules:
  9   - host: www.itopenlab.com
 10     http:
 11       paths:
 12       - path: /
 13         pathType: Prefix
 14         backend:
 15           service:
 16             name: svc-lb
 17             port:
 18               number: 80
 19
 20
 21   - host: tomcat.itopenlab.com
 22     http:
 23       paths:
 24       - path: /
 25         pathType: Prefix
 26         backend:
 27           service:
 28             name: tomcat-service
 29             port:
 30               number: 80



[root@k8s-master01 dev]# kubectl apply -f ingress1.yaml
ingress.networking.k8s.io/nginx-ingress configured


[root@k8s-master01 dev]# kubectl get svc,ing,pod -n dev
NAME                     TYPE           CLUSTER-IP    EXTERNAL-IP       PORT(S)        AGE
service/svc-lb           LoadBalancer   10.96.3.126   192.168.234.200   80:32376/TCP   3h9m
service/tomcat-service   LoadBalancer   10.96.0.170   192.168.234.202   80:32050/TCP   9m2s

NAME                                      CLASS   HOSTS                                    ADDRESS           PORTS   AGE
ingress.networking.k8s.io/nginx-ingress   nginx   www.itopenlab.com,tomcat.itopenlab.com   192.168.234.201   80      65m

NAME                                     READY   STATUS    RESTARTS   AGE
pod/nginx-deployment-7cffcbf558-7z6hs    1/1     Running   0          5h6m
pod/nginx-deployment-7cffcbf558-9fpc2    1/1     Running   0          5h6m
pod/nginx-deployment-7cffcbf558-hjlqv    1/1     Running   0          5h6m
pod/tomcat-deployment-8674d9bc4c-7fqtk   1/1     Running   0          9m2s
pod/tomcat-deployment-8674d9bc4c-8p62n   1/1     Running   0          9m2s




# 接下来,在本地电脑上配置host文件（C:\Windows\System32\drivers\etc\hosts）,解析上面的两个域名到具体的svc上
192.168.234.200  www.itopenlab.com
192.168.234.202  tomcat.itopenlab.com

分别访问tomcat.itopenlab.com 和 nginx.itopenlab.com 查看效果