k8s学习笔记5-部署和应用ingress-nginx-controller(v1.3.0)

时间：2023-09-28 04:37:02 18s弯头连接器

k8s学习笔记5-部署和应用ingress-nginx-controller-1.3.0

一.介绍和原理
- A.Ingress 是什么
- B.Ingress的组成
二.部署
- A.下载部署文件和镜像
- B.node节点打标签
- C.修改部署文件
- D.部署并查看
三.应用
四.参考文档

一.介绍和原理

A.Ingress 是什么

Ingress 从集群外到集群内公开服务 HTTP 和 HTTPS 路由。 流量路由由 Ingress 规则控制的资源定义。 下面是一个将所有流量发送到相同的流量 Service 的简单 Ingress 示例：

1.ingress：
ingress是一个API对象，通过yaml配置文件，ingress对象的作用是定义如何转发请求service配置模板可以理解为规则。
ingress通过http或https暴露集群内部service，给service提供外部URI、负载均衡、SSL/TLS基于域名的能力和反向代理。ingress要依靠ingress-controiler具体实现上述功能。
2.ingress-controller：
ingress-controller是实现反向代理和负载平衡的具体程序ingress根据配置规则则，并根据配置规则转发请求。
ingress-controller并不是k8s实际上，自带组件ingess-controller用户可以选择不同的总称ingress-controller实现，目前，由k8s维护的ingress-controller只有google云的GCE与ingress-nginx两个，还有很多其他的第三方维护ingress-controller，详见官方文件。
但不管是哪一种ingress-controller，实现的机制相似，但具体配置不同。

一般来说，ingress-controller一种形式pod，里面跑着demon程序和反向代理程序。daemon根据集群的变化，负责不断监控集群的变化ingress对象生成配置并的配置应用于反向代理，如ingress-nginx是动态生成nginx配置，动态更新upstreanm，必要时reload新配置的程序应用。为了方便，后面的例子都以k8s官方维护的ingress-nginx为例。
3.Ingress的工作原理

1.ingress-controller通过和 kubernetes APIServer互动，动态感知集群ingress规则变化； 2.然后读取它。根据自定义规则，规则是指哪个域名对应哪个域名service，生成一段nginx配置； 3.再写到nginx-ingress-controller的pod里，这个ingres-controller的pod一个在里面运行Nginx服务， 生成的控制器nginx置写入/etc/nginx.conf文件中； 4.然后reload使配置生效。为了实现域名区分配置和动态更新。

客户端通常会将请求发送到负载平衡器，从负载平衡器转发到负载平衡器ingress控制所在的node减少节点的目的Node节点的负载压力

ingress根据控制器ingress相应的资源对象配置转发请求service

service请求将根据端点转发到相关点Pod

二.部署

A.下载部署文件和镜像

1.下载ingress-nginx部署文件(如果文件无法下载，请参考链接https://blog.csdn.net/weixin_43656190/article/details/117604612)

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.3.0/deploy/static/provider/cloud/deploy.yaml

2.会导致镜像问题yaml文件中的镜像无法下载，解决方案：首先docker hub搜索并下载后，将其推送到私人harbor在镜像服务器上
下载以下两个镜像：

root@k8s-master1:~/ingress# cat deploy.yaml |grep imag         image: registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5         imagePullPolicy: IfNotPresent         image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660         imagePullPolicy: IfNotPresent         image: registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660         imagePullPolicy: IfNotPresent

在docker hub上搜索

root@k8s-master1:~/ingress# docker search ingress-nginx --no-trunc  NAME                                           DESCRIPTION                                      STARS     OFFICIAL   AUTOMATED willdockerhub/ingress-nginx-controller         sync from  k8s.gcr.io/ingress-nginx/controller   20                    pollyduan/ingress-nginx-controller             k8s.gcr.io/ingress-nginx/controller              9                     liangjw/ingress-nginx-controller               k8s.gcr.io/ingress-nginx/controller              7                     wangshun1024/ingress-nginx-controller                                                           3                     dyrnq/ingress-nginx-controller                 k8s.gcr.io/ingress-nginx/controller              2                     acicn/ingress-nginx-controller                                                                 2                    
wallarm/ingress-nginx                                                                           0                    
13916077927/ingress-nginx-controller           sync from k8s.gcr.io/ingress-nginx/controller    0                    
rancher/ingress-nginx                                                                           0                    
v5cn/ingress-nginx-controller                  k8s.gcr.io/ingress-nginx/controller:v0.46.0      0                    
platform9/ingress-nginx                                                                         0                    
cangyin/ingress-nginx-controller                                                                0                    
forging2012/ingress-nginx-controller                                                            0                    
sundewang/ingress-nginx-controller                                                              0                    
boy530/ingress-nginx-controller                                                                 0                    
blendlabs/ingress-nginx-catchall               A blend branded nginx catchall.                  0                    
cjk2atmb/ingress-nginx-controller                                                               0                    
opsdockerimage/ingress-nginx-controller                                                         0                    
acicn/ingress-nginx-kube-webhook-certgen                                                        0                    
rudl/ingress-nginx                             Nginx ingress controller                         0                    
opsdockerimage/ingress-nginx-nginx                                                              0                    
lchdzh/ingress-nginx-controller                                                                 0                    
liy36/ingress-nginx-controller                                                                  0                    
opsdockerimage/ingress-nginx-e2e-test-runner                                                    0                    
chainstack/ingress-nginx                                                                        0                    
root@k8s-master1:~/ingress# docker search kube-webhook-certgen --no-trunc 
NAME                                 DESCRIPTION                                                                               STARS     OFFICIAL   AUTOMATED
jettech/kube-webhook-certgen         Kubernetes webhook certificate generator and patcher                                      14                   
liangjw/kube-webhook-certgen         k8s.gcr.io/ingress-nginx/kube-webhook-certgen                                             8                    
dyrnq/kube-webhook-certgen           k8s.gcr.io/ingress-nginx/kube-webhook-certgen                                             5                    
wangshun1024/kube-webhook-certgen                                                                                              3                    
wonderflow/kube-webhook-certgen                                                                                                1                    
lianyuxue1020/kube-webhook-certgen   new pull lianyuxue1020/kube-webhook-certgen:v1.1.1                                        1                    
newrelic/kube-webhook-certgen                                                                                                  0                    
catalystcloud/kube-webhook-certgen                                                                                             0                    
spdplx2021/kube-webhook-certgen                                                                                                0                    
congruity360/kube-webhook-certgen                                                                                              0                    
pangser/kube-webhook-certgen                                                                                                   0                    
oamdev/kube-webhook-certgen                                                                                                    0                    
sundewang/kube-webhook-certgen                                                                                                 0                    
hzde0128/kube-webhook-certgen                                                                                                  0                    
kubespheredev/kube-webhook-certgen                                                                                             0                    
caas4/kube-webhook-certgen                                                                                                     0                    
rpkatz/kube-webhook-certgen                                                                                                    0                    
xiaoyan1121/kube-webhook-certgen     k8s.gcr.io/ingress-nginx/kube-webhook-certgen                                             0                    
serenade11/kube-webhook-certgen                                                                                                0                    
invidian/kube-webhook-certgen                                                                                                  0                    
serialt/kube-webhook-certgen                                                                                                   0                    
zhangdiandong/kube-webhook-certgen   kube-webhook-certgen:v1.0                                                                 0                    
fnape/kube-webhook-certgen           k8s.gcr.io/ingress-nginx/kube-webhook-certgen   Mirror accessible from CN                 0                    
giantswarm/kube-webhook-certgen                                                                                                0                    
madforg/kube-webhook-certgen         k8s ingress-nginx 实验所需的gcr.io镜像 ( k8s.gcr.io/ingress-nginx/kube-webhook-certgen:v1.0 )    0                    
root@k8s-master1:~/ingress#

3.下载，重命名，上传，替换镜像

#1.下载镜像
root@k8s-master1:~/ingress# docker pull dyrnq/ingress-nginx-controller:v1.3.0
root@k8s-master1:~/ingress# docker pull lianyuxue1020/kube-webhook-certgen:v1.1.1
#2.将镜像重命名
root@k8s-master1:~/ingress# docker tag dyrnq/ingress-nginx-controller:v1.3.0 registry.harbor.com/library/ingress-nginx-controller:v1.3.0
root@k8s-master1:~/ingress# docker tag lianyuxue1020/kube-webhook-certgen:v1.1.1 registry.harbor.com/library/kube-webhook-certgen:v1.1.1
#3.将镜像push到我们之前搭建的harbor仓库
root@k8s-master1:~/ingress# docker push registry.harbor.com/library/ingress-nginx-controller:v1.3.0 
The push refers to repository [registry.harbor.com/library/ingress-nginx-controller]
11d5b7949407: Pushed 
a2ab1d98af90: Pushed 
5b42063defba: Pushed 
3ee663ef34d0: Pushed 
bf8b0761126a: Pushed 
158aa99c71d8: Pushed 
05856d1bb285: Pushed 
f1b0a27f62c1: Pushed 
5f70bf18a086: Pushed 
7f366fb88640: Pushed 
12ca2606c07b: Pushed 
ca8e478eeb76: Pushed 
b45bedc7f782: Pushed 
24302eb7d908: Pushed 
v1.3.0: digest: sha256:067673df26a65ec5c2d5b30f25db869bad4d7d391fc81882250134577e581ef0 size: 3256
root@k8s-master1:~/ingress# docker push registry.harbor.com/library/kube-webhook-certgen:v1.1.1 
The push refers to repository [registry.harbor.com/library/kube-webhook-certgen]
ce7a3c1169b6: Pushed 
c0d270ab7e0d: Pushed 
v1.1.1: digest: sha256:78351fc9d9b5f835e0809921c029208faeb7fbb6dc2d3b0d1db0a6584195cfed size: 739
#4.将ingress-nginx部署文件中的镜像地址替换成harbor仓库的地址
root@k8s-master1:~/ingress# sed -i s#registry.k8s.io/ingress-nginx/controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323e#registry.harbor.com/library/ingress-nginx-controller:v1.3.0#g deploy.yaml 
root@k8s-master1:~/ingress# sed -i s#registry.k8s.io/ingress-nginx/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660#registry.harbor.com/library/kube-webhook-certgen:v1.1.1#g deploy.yaml 
#5.查看镜像地址是否替换成功
root@k8s-master1:~/ingress# cat deploy.yaml |grep image
        image: registry.harbor.com/library/ingress-nginx-controller:v1.3.0@sha256:d1707ca76d3b044ab8a28277a2466a02100ee9f58a86af1535a3edf9323ea1b5
        imagePullPolicy: IfNotPresent
        image: registry.harbor.com/library/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
        imagePullPolicy: IfNotPresent
        image: registry.harbor.com/library/kube-webhook-certgen:v1.1.1@sha256:64d8c73dca984af206adf9d6d7e46aa550362b1d7a01f3a0a91b20cc67868660
        imagePullPolicy: IfNotPresent
root@k8s-master1:~/ingress#

B.node节点打标签

可以通过nodeSelector确定将ingress-nginx-controller部署在那些机器上

root@k8s-master1:~/ingress# kubectl label nodes k8s-node1 hasIngress=true
node/k8s-node1 labeled
root@k8s-master1:~/ingress# kubectl label nodes k8s-node2 hasIngress=true
node/k8s-node2 labeled
root@k8s-master1:~/ingress#

C.修改部署文件

修改三个地方
1.HostNetwork
是为了打通Cluster和node的网络，让Cluster直接监听node的端口，一般是80和443，不用再通过随机绑定的nodePort来访问集群服务

以前访问：http://www.xwl.com:30080/web1/contextpath
现在访问：http://www.xwl.com/web1/contextpath
可以看到不再需要随机绑定的nodePort了，因为hostNetwork直接监听集群的80端口。

2.nodeSelector
主要是为了能够选择将ingress-nginx-controller部署到那个机器上
3.DaemonSet
因为Deployment可能会把多个pod调度到同一个node，那就失去高可用的意义了。而DaemonSet在一个节点上只会有一个Pod，符合我们的要求。
修改deploy.yaml

......
apiVersion: apps/v1
#kind: Deployment
kind: DaemonSet  #这里把Deployment改成DaemonSet
metadata:
  labels:
    app.kubernetes.io/component: controller
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/part-of: ingress-nginx
    app.kubernetes.io/version: 1.3.0
  name: ingress-nginx-controller
  namespace: ingress-nginx
spec:
  minReadySeconds: 0
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app.kubernetes.io/component: controller
      app.kubernetes.io/instance: ingress-nginx
      app.kubernetes.io/name: ingress-nginx
  template:
    metadata:
      labels:
        app.kubernetes.io/component: controller
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/name: ingress-nginx
    spec:
      hostNetwork: true  #这里加一句
      containers:
      - args:
        - /nginx-ingress-controller
        - --publish-service=$(POD_NAMESPACE)/ingress-nginx-controller
        - --election-id=ingress-controller-leader
        - --controller-class=k8s.io/ingress-nginx
        - --ingress-class=nginx
        - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
        - --validating-webhook=:8443
        - --validating-webhook-certificate=/usr/local/certificates/cert
        - --validating-webhook-key=/usr/local/certificates/key
        env:
......
      nodeSelector:
        kubernetes.io/os: linux
        hasIngress: "true"  # 这里个面具标签选择node
......

D.部署并查看

部署ingress-nginx,部署后，需要等待一会，大约1分钟这样，貌似部署成功，其实没有部署成功

root@k8s-master1:~/ingress# kubectl apply -f deploy.yaml 
namespace/ingress-nginx created
serviceaccount/ingress-nginx created
serviceaccount/ingress-nginx-admission created
role.rbac.authorization.k8s.io/ingress-nginx created
role.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrole.rbac.authorization.k8s.io/ingress-nginx created
clusterrole.rbac.authorization.k8s.io/ingress-nginx-admission created
rolebinding.rbac.authorization.k8s.io/ingress-nginx created
rolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx created
clusterrolebinding.rbac.authorization.k8s.io/ingress-nginx-admission created
configmap/ingress-nginx-controller created
service/ingress-nginx-controller created
service/ingress-nginx-controller-admission created
deployment.apps/ingress-nginx-controller created
job.batch/ingress-nginx-admission-create created
job.batch/ingress-nginx-admission-patch created
ingressclass.networking.k8s.io/nginx created
validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created
root@k8s-master1:~/ingress# kubectl get pods -n ingress-nginx -o wide 
NAME                                   READY   STATUS      RESTARTS   AGE     IP                NODE        NOMINATED NODE   READINESS GATES
ingress-nginx-admission-create-vc554   0/1     Completed   0          3m18s   10.244.2.44       k8s-node2   <none>           <none>
ingress-nginx-admission-patch-swbrw    0/1     Completed   0          3m18s   10.244.2.45       k8s-node2   <none>           <none>
ingress-nginx-controller-c8bxg         1/1     Running     0          3m18s   192.168.100.243   k8s-node1   <none>           <none>
ingress-nginx-controller-ph6vn         1/1     Running     0          3m18s   192.168.100.244   k8s-node2   <none>           <none>
root@k8s-master1:~/ingress#

三.应用

1、创建tomcat-nginx.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx-deployment
  namespace: dev
spec:
  replicas: 3
  selector:
    matchLabels:
      app: nginx-pod
  template:
    metadata:
      labels:
        app: nginx-pod
    spec:
      containers:
      - name: nginx
        image: nginx:1.17.1
        ports:
        - containerPort: 80
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: tomcat-deployment
  namespace: dev
spec:
  replicas: 3
  selector:
    matchLabels:
      app: tomcat-pod
  template:
    metadata:
      labels:
        app: tomcat-pod
    spec:
      containers:
      - name: tomcat
        image: tomcat:8.5-jre10-slim
        ports:
        - containerPort: 8080

---
apiVersion: v1
kind: Service
metadata:
  name: nginx-service
  namespace: dev
spec:
  ports:
    - port: 80
      name: nginx
  clusterIP: None
  selector:
    app: nginx-pod
---
apiVersion: v1
kind: Service
metadata:
  name: tomcat-service
  namespace: dev
spec:
  ports:
    - port: 8080
      name: tomcat
  clusterIP: None
  selector:
    app: tomcat-pod

2.创建ingress-http.yaml

kind: Ingress
metadata:
  name: ingress-http
  namespace: dev
spec:
  ingressClassName: nginx
  rules:
  - host: nginx.xwl.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-service
            port:
              number: 80
  - host: tomcat.xwl.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: tomcat-service
            port:
              number: 8080

3.修改任意一个主机的/etc/hosts文件，添加如下4个域名解析

192.168.100.243 nginx.xwl.com
192.168.100.244 nginx.xwl.com
192.168.100.243 tomcat.xwl.com
192.168.100.244 tomcat.xwl.com

4.验证是否能够正常登录
登录nginx.xwl.com

登录tomcat.xwl.com

四.参考文档

1.ingress官方文档
https://kubernetes.io/zh-cn/docs/concepts/services-networking/ingress/
2.ingress-nginx官方文档
https://kubernetes.github.io/ingress-nginx/deploy/#quick-start
3.hostNetwork与ingress-nginx高可用
https://blog.csdn.net/weixin_41535316/article/details/121873795
4.k8s部署ingress-nginx的方法步骤
https://blog.csdn.net/a13568hki/article/details/124452407
5.K8S对外服务之Ingress （使用https的时候，可以参考这篇文档）
https://blog.51cto.com/u_15382481/5205194
6.k8s ingress原理
https://blog.csdn.net/weixin_46108954/article/details/105717399

锐单商城拥有海量元器件数据手册、IC替代型号，打造电子元器件IC百科大全！

k8s学习笔记5-部署和应用ingress-nginx-controller(v1.3.0)

k8s学习笔记5-部署和应用ingress-nginx-controller-1.3.0

相关文章