K3s 配置 Envoy Gateway:结合 Cloudflare Tunnel 与 Tailscale 实现内外网服务隔离
配置概览
- 私有服务:通过 Envoy Gateway 暴露为
LoadBalancer类型的 Service,配合集群内 K3s 自带的 ServiceLB(原 Klipper LoadBalancer),借助 Tailscale 节点网络进行内网访问。
- 公开服务:通过 Envoy Gateway 暴露为
ClusterIP类型的 Service,流量仅在集群内部流转,前端由部署在集群内的cloudflared容器以 Cloudflare Tunnel 方式反向代理出去。
[GatewayClass: envoy-gateway] (控制面控制器) │ ├──> [Gateway: envoy-gateway-tailscale] ──> Service类型: LoadBalancer (Tailscale 内网接入) │ └──> [Gateway: envoy-gateway-cloudflare] ──> Service类型: ClusterIP (Cloudflare Tunnel 接入)
cert-manager 完成 Let's Encrypt TLS 证书自动签发,并利用 external-dns 根据网关类型自动同步 DNS 记录(Tailscale 自动建 A 记录,Cloudflare Tunnel 自动建 CNAME 记录)。K3s 配置安装
配置文件 /etc/rancher/k3s/config.yaml
write-kubeconfig-mode: "0600" # token: "" # 强随机字符串 (openssl rand -hex 16) token-file: /etc/rancher/k3s/token bind-address: "0.0.0.0" node-ip: "<Tailscale-IP>" node-external-ip: "<Tailscale-IP>" tls-san: - "127.0.0.1" - "localhost" - "<Tailscale-IP>" - "<HostName>" - "<Tailscale-MagicDNS>" - "<个人域名>" disable: - traefik kube-proxy-arg: - "proxy-mode=nftables" # 基础的 kubelet 参数就放在这,官方建议是使用专门的 kubelet 配置文件 kubelet-arg: - "max-pods=110" - "image-gc-high-threshold=80" - "image-gc-low-threshold=60" - "container-log-max-size=10Mi" - "container-log-max-files=3" - "eviction-hard=memory.available<300Mi,nodefs.available<2Gi" - "serialize-image-pulls=false" # 需调整内核参数 protect-kernel-defaults: true secrets-encryption: true
token-file 生成
mkdir -p /etc/rancher/k3s TOKEN_FILE="/etc/rancher/k3s/token" if [[ ! -f "${TOKEN_FILE}" ]]; then if command -v openssl >/dev/null 2>&1; then openssl rand -hex 16 > "${TOKEN_FILE}" else head -c 32 /dev/urandom \ | base64 \ | tr -dc 'a-zA-Z0-9' \ | head -c 32 > "${TOKEN_FILE}" fi chmod 600 "${TOKEN_FILE}" else log "检测到已有 token,跳过生成" fi
内核参数配置
cat >/etc/sysctl.d/99-k3s-kubelet.conf <<EOF # Kubernetes network forwarding net.ipv4.ip_forward = 1 net.ipv6.conf.all.forwarding = 1 # Bridge netfilter net.bridge.bridge-nf-call-iptables = 1 net.bridge.bridge-nf-call-ip6tables = 1 # Protect kernel defaults requirements vm.overcommit_memory = 1 kernel.panic = 10 kernel.panic_on_oops = 1 EOF sysctl --system >/dev/null

Envoy 网关部署
直接用 Helm 安装 Envoy Gateway,可配置项:
Envoy GatewayGateway Helm Chart
Gateway Helm Chart
The Helm chart for Envoy Gateway Homepage: https://gateway.envoyproxy.io/ Maintainers Name Email Url envoy-gateway-steering-committee https://github.com/envoyproxy/gateway/blob/main/GOVERNANCE.md envoy-gateway-maintainers https://github.com/envoyproxy/gateway/blob/main/CODEOWNERS Source Code https://github.com/envoyproxy/gateway Requirements Repository Name Version crds 0.0.0 Values Key Type Default Description certgen object {"job":{"affinity":{},"annotations":{},"args":[],"nodeSelector":{},"pod":{"annotations":{},"labels":{},"securityContext":{"fsGroup":65532,"runAsGroup":65532,"runAsNonRoot":true,"runAsUser":65532,"seccompProfile":{"type":"RuntimeDefault"}}},"resources":{},"securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true,"runAsGroup":65532,"runAsNonRoot":true,"runAsUser":65532,"seccompProfile":{"type":"RuntimeDefault"}},"tolerations":[],"ttlSecondsAfterFinished":30},"rbac":{"annotations":{},"labels":{}}} Certgen is used to generate the certificates required by EnvoyGateway. If you want to construct a custom certificate, you can generate a custom certificate through Cert-Manager before installing EnvoyGateway. Certgen will not overwrite the custom certificate. Please do not manually modify values.yaml to disable certgen, it may cause EnvoyGateway OIDC,OAuth2,etc. to not work as expected. commonLabels object {} Labels to apply to all resources config.envoyGateway object {"extensionApis":{},"gateway":{"controllerName":"gateway.envoyproxy.io/gatewayclass-controller"},"logging":{"level":{"default":"info"}},"provider":{"type":"Kubernetes"}} EnvoyGateway configuration. Visit https://gateway.envoyproxy.io/docs/api/extension_types/#envoygateway to view all options. crds.enabled bool true Install Envoy Gateway CRDs, Gateway API CRDs, and Gateway API safe upgrade policy resources. Set to false when these resources are managed separately. createNamespace bool false deployment.annotations object {} deployment.envoyGateway.extraEnv list [] Additional environment variables for the envoy-gateway container. deployment.envoyGateway.image.repository string "" deployment.envoyGateway.image.tag string "" deployment.envoyGateway.imagePullPolicy string "" deployment.envoyGateway.imagePullSecrets list [] deployment.envoyGateway.livenessProbe.httpGet.path string "/healthz" deployment.envoyGateway.livenessProbe.httpGet.port int 8081 deployment.envoyGateway.livenessProbe.periodSeconds int 20 deployment.envoyGateway.livenessProbe.successThreshold int 1 deployment.envoyGateway.livenessProbe.timeoutSeconds int 1 deployment.envoyGateway.readinessProbe.httpGet.path string "/readyz" deployment.envoyGateway.readinessProbe.httpGet.port int 8081 deployment.envoyGateway.readinessProbe.periodSeconds int 10 deployment.envoyGateway.readinessProbe.successThreshold int 1 deployment.envoyGateway.readinessProbe.timeoutSeconds int 1 deployment.envoyGateway.resources.limits.memory string "1024Mi" deployment.envoyGateway.resources.requests.cpu string "100m" deployment.envoyGateway.resources.requests.memory string "256Mi" deployment.envoyGateway.securityContext.allowPrivilegeEscalation bool false deployment.envoyGateway.securityContext.capabilities.drop[0] string "ALL" deployment.envoyGateway.securityContext.privileged bool false deployment.envoyGateway.securityContext.readOnlyRootFilesystem bool true deployment.envoyGateway.securityContext.runAsGroup int 65532 deployment.envoyGateway.securityContext.runAsNonRoot bool true deployment.envoyGateway.securityContext.runAsUser int 65532 deployment.envoyGateway.securityContext.seccompProfile.type string "RuntimeDefault" deployment.envoyGateway.startupProbe.failureThreshold int 30 deployment.envoyGateway.startupProbe.httpGet.path string "/healthz" deployment.envoyGateway.startupProbe.httpGet.port int 8081 deployment.envoyGateway.startupProbe.periodSeconds int 1 deployment.envoyGateway.startupProbe.successThreshold int 1 deployment.envoyGateway.startupProbe.timeoutSeconds int 1 deployment.envoyGateway.strategy object {} Volume source for the Wasm module cache mounted at /var/lib/eg/wasm. Defaults to an emptyDir when left empty. Example: persist the Wasm module cache across controller restarts by backing it with a PersistentVolumeClaim: wasmCacheVolume: persistentVolumeClaim: claimName: envoy-gateway-wasm-cache deployment.envoyGateway.wasmCacheVolume object {} deployment.pod.affinity object {} deployment.pod.annotations.“prometheus.io/port” string "19001" deployment.pod.annotations.“prometheus.io/scrape” string "true" deployment.pod.extraVolumeMounts list [] deployment.pod.extraVolumes list [] deployment.pod.labels object {} deployment.pod.nodeSelector object {} deployment.pod.securityContext.fsGroup int 65532 deployment.pod.securityContext.runAsGroup int 65532 deployment.pod.securityContext.runAsNonRoot bool true deployment.pod.securityContext.runAsUser int 65532 deployment.pod.securityContext.seccompProfile.type string "RuntimeDefault" deployment.pod.tolerations list [] deployment.pod.topologySpreadConstraints list [] deployment.ports[0].name string "grpc" deployment.ports[0].port int 18000 deployment.ports[0].targetPort int 18000 deployment.ports[1].name string "ratelimit" deployment.ports[1].port int 18001 deployment.ports[1].targetPort int 18001 deployment.ports[2].name string "wasm" deployment.ports[2].port int 18002 deployment.ports[2].targetPort int 18002 deployment.ports[3].name string "metrics" deployment.ports[3].port int 19001 deployment.ports[3].targetPort int 19001 deployment.priorityClassName string nil deployment.replicas int 1 global.imagePullSecrets list [] Global override for image pull secrets global.imageRegistry string "" Global override for image registry global.images.envoyGateway.image string nil Full image for the Envoy Gateway control plane Deployment installed by this chart. global.images.envoyGateway.pullPolicy string nil Image pull policy for the Envoy Gateway control plane Deployment. Default behavior: latest images will be Always else IfNotPresent. global.images.envoyGateway.pullSecrets list [] Pull secrets for the Envoy Gateway control plane Deployment. global.images.envoyProxy.image string "" Full image for the managed Envoy Proxy data plane. This updates the generated envoyProxy config and does not change the envoy-gateway control plane Deployment image. If not specified, the default image built into envoy-gateway is used. global.images.envoyProxy.pullPolicy string "" Image pull policy for the managed Envoy Proxy data plane. Default behavior: IfNotPresent. global.images.envoyProxy.pullSecrets list [] Pull secrets for the managed Envoy Proxy data plane. global.images.ratelimit.image string "docker.io/envoyproxy/ratelimit:master" global.images.ratelimit.pullPolicy string "IfNotPresent" global.images.ratelimit.pullSecrets list [] hpa.behavior object {} hpa.enabled bool false hpa.maxReplicas int 1 hpa.metrics list [] hpa.minReplicas int 1 kubernetesClusterDomain string "cluster.local" namespaceOverride string "" Override the namespace for resources deployed by the chart. Defaults to the release namespace. podDisruptionBudget.minAvailable int 0 service.annotations object {} service.trafficDistribution string "" service.type string "ClusterIP" Service type. Can be set to LoadBalancer with specific IP, e.g.: type: LoadBalancer loadBalancerIP: 10.236.90.20 topologyInjector.annotations object {} topologyInjector.enabled bool true

Gateway Helm Chart
The Helm chart for Envoy Gateway Homepage: https://gateway.envoyproxy.io/ Maintainers Name Email Url envoy-gateway-steering-committee https://github.com/envoyproxy/gateway/blob/main/GOVERNANCE.md envoy-gateway-maintainers https://github.com/envoyproxy/gateway/blob/main/CODEOWNERS Source Code https://github.com/envoyproxy/gateway Requirements Repository Name Version crds 0.0.0 Values Key Type Default Description certgen object {"job":{"affinity":{},"annotations":{},"args":[],"nodeSelector":{},"pod":{"annotations":{},"labels":{},"securityContext":{"fsGroup":65532,"runAsGroup":65532,"runAsNonRoot":true,"runAsUser":65532,"seccompProfile":{"type":"RuntimeDefault"}}},"resources":{},"securityContext":{"allowPrivilegeEscalation":false,"capabilities":{"drop":["ALL"]},"privileged":false,"readOnlyRootFilesystem":true,"runAsGroup":65532,"runAsNonRoot":true,"runAsUser":65532,"seccompProfile":{"type":"RuntimeDefault"}},"tolerations":[],"ttlSecondsAfterFinished":30},"rbac":{"annotations":{},"labels":{}}} Certgen is used to generate the certificates required by EnvoyGateway. If you want to construct a custom certificate, you can generate a custom certificate through Cert-Manager before installing EnvoyGateway. Certgen will not overwrite the custom certificate. Please do not manually modify values.yaml to disable certgen, it may cause EnvoyGateway OIDC,OAuth2,etc. to not work as expected. commonLabels object {} Labels to apply to all resources config.envoyGateway object {"extensionApis":{},"gateway":{"controllerName":"gateway.envoyproxy.io/gatewayclass-controller"},"logging":{"level":{"default":"info"}},"provider":{"type":"Kubernetes"}} EnvoyGateway configuration. Visit https://gateway.envoyproxy.io/docs/api/extension_types/#envoygateway to view all options. crds.enabled bool true Install Envoy Gateway CRDs, Gateway API CRDs, and Gateway API safe upgrade policy resources. Set to false when these resources are managed separately. createNamespace bool false deployment.annotations object {} deployment.envoyGateway.extraEnv list [] Additional environment variables for the envoy-gateway container. deployment.envoyGateway.image.repository string "" deployment.envoyGateway.image.tag string "" deployment.envoyGateway.imagePullPolicy string "" deployment.envoyGateway.imagePullSecrets list [] deployment.envoyGateway.livenessProbe.httpGet.path string "/healthz" deployment.envoyGateway.livenessProbe.httpGet.port int 8081 deployment.envoyGateway.livenessProbe.periodSeconds int 20 deployment.envoyGateway.livenessProbe.successThreshold int 1 deployment.envoyGateway.livenessProbe.timeoutSeconds int 1 deployment.envoyGateway.readinessProbe.httpGet.path string "/readyz" deployment.envoyGateway.readinessProbe.httpGet.port int 8081 deployment.envoyGateway.readinessProbe.periodSeconds int 10 deployment.envoyGateway.readinessProbe.successThreshold int 1 deployment.envoyGateway.readinessProbe.timeoutSeconds int 1 deployment.envoyGateway.resources.limits.memory string "1024Mi" deployment.envoyGateway.resources.requests.cpu string "100m" deployment.envoyGateway.resources.requests.memory string "256Mi" deployment.envoyGateway.securityContext.allowPrivilegeEscalation bool false deployment.envoyGateway.securityContext.capabilities.drop[0] string "ALL" deployment.envoyGateway.securityContext.privileged bool false deployment.envoyGateway.securityContext.readOnlyRootFilesystem bool true deployment.envoyGateway.securityContext.runAsGroup int 65532 deployment.envoyGateway.securityContext.runAsNonRoot bool true deployment.envoyGateway.securityContext.runAsUser int 65532 deployment.envoyGateway.securityContext.seccompProfile.type string "RuntimeDefault" deployment.envoyGateway.startupProbe.failureThreshold int 30 deployment.envoyGateway.startupProbe.httpGet.path string "/healthz" deployment.envoyGateway.startupProbe.httpGet.port int 8081 deployment.envoyGateway.startupProbe.periodSeconds int 1 deployment.envoyGateway.startupProbe.successThreshold int 1 deployment.envoyGateway.startupProbe.timeoutSeconds int 1 deployment.envoyGateway.strategy object {} Volume source for the Wasm module cache mounted at /var/lib/eg/wasm. Defaults to an emptyDir when left empty. Example: persist the Wasm module cache across controller restarts by backing it with a PersistentVolumeClaim: wasmCacheVolume: persistentVolumeClaim: claimName: envoy-gateway-wasm-cache deployment.envoyGateway.wasmCacheVolume object {} deployment.pod.affinity object {} deployment.pod.annotations.“prometheus.io/port” string "19001" deployment.pod.annotations.“prometheus.io/scrape” string "true" deployment.pod.extraVolumeMounts list [] deployment.pod.extraVolumes list [] deployment.pod.labels object {} deployment.pod.nodeSelector object {} deployment.pod.securityContext.fsGroup int 65532 deployment.pod.securityContext.runAsGroup int 65532 deployment.pod.securityContext.runAsNonRoot bool true deployment.pod.securityContext.runAsUser int 65532 deployment.pod.securityContext.seccompProfile.type string "RuntimeDefault" deployment.pod.tolerations list [] deployment.pod.topologySpreadConstraints list [] deployment.ports[0].name string "grpc" deployment.ports[0].port int 18000 deployment.ports[0].targetPort int 18000 deployment.ports[1].name string "ratelimit" deployment.ports[1].port int 18001 deployment.ports[1].targetPort int 18001 deployment.ports[2].name string "wasm" deployment.ports[2].port int 18002 deployment.ports[2].targetPort int 18002 deployment.ports[3].name string "metrics" deployment.ports[3].port int 19001 deployment.ports[3].targetPort int 19001 deployment.priorityClassName string nil deployment.replicas int 1 global.imagePullSecrets list [] Global override for image pull secrets global.imageRegistry string "" Global override for image registry global.images.envoyGateway.image string nil Full image for the Envoy Gateway control plane Deployment installed by this chart. global.images.envoyGateway.pullPolicy string nil Image pull policy for the Envoy Gateway control plane Deployment. Default behavior: latest images will be Always else IfNotPresent. global.images.envoyGateway.pullSecrets list [] Pull secrets for the Envoy Gateway control plane Deployment. global.images.envoyProxy.image string "" Full image for the managed Envoy Proxy data plane. This updates the generated envoyProxy config and does not change the envoy-gateway control plane Deployment image. If not specified, the default image built into envoy-gateway is used. global.images.envoyProxy.pullPolicy string "" Image pull policy for the managed Envoy Proxy data plane. Default behavior: IfNotPresent. global.images.envoyProxy.pullSecrets list [] Pull secrets for the managed Envoy Proxy data plane. global.images.ratelimit.image string "docker.io/envoyproxy/ratelimit:master" global.images.ratelimit.pullPolicy string "IfNotPresent" global.images.ratelimit.pullSecrets list [] hpa.behavior object {} hpa.enabled bool false hpa.maxReplicas int 1 hpa.metrics list [] hpa.minReplicas int 1 kubernetesClusterDomain string "cluster.local" namespaceOverride string "" Override the namespace for resources deployed by the chart. Defaults to the release namespace. podDisruptionBudget.minAvailable int 0 service.annotations object {} service.trafficDistribution string "" service.type string "ClusterIP" Service type. Can be set to LoadBalancer with specific IP, e.g.: type: LoadBalancer loadBalancerIP: 10.236.90.20 topologyInjector.annotations object {} topologyInjector.enabled bool true
# 安装 helm install eg oci://docker.io/envoyproxy/gateway-helm --version v1.9.1 -n envoy-gateway-system --create-namespace --set config.envoyGateway.logging.format=json # 等待 Envoy Gateway 启动就绪 kubectl wait --timeout=5m -n envoy-gateway-system deployment/envoy-gateway --for=condition=Available

创建 Gateway Class
apiVersion: gateway.networking.k8s.io/v1 kind: GatewayClass metadata: name: eg spec: controllerName: gateway.envoyproxy.io/gatewayclass-controller
创建 EnvoyProxy 配置,覆写 Envoy Gateway Controller 之后生成的 Service 名称,
apiVersion: gateway.envoyproxy.io/v1alpha1 kind: EnvoyProxy metadata: name: cloudflare-proxy-config namespace: envoy-gateway-system spec: provider: type: Kubernetes kubernetes: envoyService: type: ClusterIP name: envoy-svc-fixed
创建 Gateway,其中 Cloudflare Tunnel 对应引入上面的 Envoy 自定义覆写配置,Tailscale 不用
apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: cloudflare-gateway namespace: envoy-gateway-system spec: gatewayClassName: eg listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: All infrastructure: parametersRef: group: gateway.envoyproxy.io kind: EnvoyProxy name: cloudflare-proxy-config --- apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: tailscale-gateway namespace: envoy-gateway-system spec: gatewayClassName: eg listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: All
- Tailscale Gateway:Service 为默认的
LoadBalancer。K3s 的 ServiceLB 检测到后,会在节点启动 Pod 进行端口转发与流量透传。
- Cloudflare Gateway:引入自定义
EnvoyProxy配置,指定 Service 为ClusterIP,仅限 K3s 内网访问。
Tailscale 链路测试
Deployment + Service
apiVersion: apps/v1 kind: Deployment metadata: name: httpbin namespace: default spec: replicas: 1 selector: matchLabels: app: httpbin template: metadata: labels: app: httpbin spec: containers: - name: httpbin image: kennethreitz/httpbin ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: httpbin namespace: default spec: selector: app: httpbin hostnames: - httpbin.okhk.net ports: - port: 80 targetPort: 80
HTTPRoute
apiVersion: gateway.networking.k8s.io/v1 kind: HTTPRoute metadata: name: httpbin namespace: default spec: parentRefs: - name: tailscale-gateway namespace: envoy-gateway-system sectionName: http rules: - matches: - path: type: PathPrefix value: / backendRefs: - name: httpbin port: 80
本地可以通过修改 hosts 或在内网 DNS 中配置域名指向 Tailscale 节点 IP,即可成功访问。

Cloudflare Tunnel 接入
Namespace
apiVersion: v1 kind: Namespace metadata: name: cloudflare
Secret
apiVersion: v1 kind: Secret metadata: name: cf-tunnel-secret namespace: cloudflare type: Opaque stringData: tunnel-token: "<Tunnel Token>"
Deployment
apiVersion: apps/v1 kind: Deployment metadata: name: cloudflared namespace: cloudflare labels: app.kubernetes.io/name: cloudflared spec: replicas: 1 revisionHistoryLimit: 2 selector: matchLabels: app.kubernetes.io/name: cloudflared template: metadata: labels: app.kubernetes.io/name: cloudflared spec: containers: - name: cloudflared image: cloudflare/cloudflared:2026.9.3 imagePullPolicy: IfNotPresent args: - tunnel - --output - json - --loglevel - debug - --metrics - 0.0.0.0:2000 - --no-autoupdate - run env: - name: TUNNEL_TOKEN valueFrom: secretKeyRef: name: cf-tunnel-secret key: tunnel-token ports: - name: metrics containerPort: 2000 protocol: TCP resources: requests: cpu: 10m memory: 32Mi limits: cpu: 100m memory: 128Mi livenessProbe: httpGet: path: /ready port: metrics initialDelaySeconds: 15 periodSeconds: 30 timeoutSeconds: 5 failureThreshold: 3 readinessProbe: httpGet: path: /ready port: metrics initialDelaySeconds: 5 periodSeconds: 10 timeoutSeconds: 3 failureThreshold: 3 securityContext: allowPrivilegeEscalation: false readOnlyRootFilesystem: true runAsNonRoot: true runAsUser: 65532 runAsGroup: 65532 capabilities: drop: - ALL seccompProfile: type: RuntimeDefault volumeMounts: - name: tmp mountPath: /tmp volumes: - name: tmp emptyDir: {}
部署完成后,Cloudflare Dashboard 显示隧道连接已建立。配置一条 Published application routes ,Service 指向 K3s 中 Envoy Gateway Service。也就是:envoy-svc-fixed.envoy-gateway-system。此时公网流量经过 Cloudflare CDN 边缘节点,由 Tunnel 穿透注入集群,再传递给 Envoy Gateway 进行路由分发。



cert-manager 自动签发 Let's Encrypt 证书
Helm 部署
helm install \ cert-manager oci://quay.io/jetstack/charts/cert-manager \ --version v1.21.2 \ --namespace cert-manager \ --create-namespace \ --set crds.enabled=true
Cloudflare API Key 创建
Zone - DNS - Edit
◦ Zone - Zone - Read
• Zone Resources:选择包含您域名的 Zone(或 Include - All zones) 将 Token 存储为 Kubernetes Secret,后续要使用 ClusterIssuer ,必须存储在 cert-manager 相同 namespace 下
kubectl create secret generic cloudflare-api-token-secret \ --namespace cert-manager \ --from-literal=api-token=<CLOUDFLARE_API_TOKEN>
配置 cert-manager ClusterIssuer 开启 DNS-01 Challenge 认证
apiVersion: cert-manager.io/v1 kind: ClusterIssuer metadata: name: letsencrypt-cloudflare-dns spec: acme: # 邮箱,用于接收 LetsEncrypt 证书到期提醒 email: your-email@example.com server: https://acme-v02.api.letsencrypt.org/directory privateKeySecretRef: name: letsencrypt-cloudflare-account-key solvers: - dns01: cloudflare: apiTokenSecretRef: name: cloudflare-api-token-secret key: api-token
申请证书(Envoy Gateway Namespace 下)
apiVersion: cert-manager.io/v1 kind: Certificate metadata: name: okhk-tls namespace: envoy-gateway-system spec: secretName: okhk-tls-secret issuerRef: name: letsencrypt-cloudflare-dns kind: ClusterIssuer dnsNames: - "*.okhk.net"
在 Envoy Gateway 的 Listener 中引用挂载 TLS 证书
apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: envoy-gateway namespace: envoy-gateway-system spec: gatewayClassName: eg listeners: - name: http protocol: HTTP port: 80 allowedRoutes: namespaces: from: All - name: https protocol: HTTPS port: 443 hostname: "*.okhk.net" tls: mode: Terminate certificateRefs: - kind: Secret name: okhk-tls-secret allowedRoutes: namespaces: from: All infrastructure: parametersRef: group: gateway.envoyproxy.io kind: EnvoyProxy name: custom-proxy-config
external-dns 配置
为了避免手动维护 DNS 记录,通过部署 external-dns 实现 HTTPRoute 与 DNS 的自动同步:
helm repo add external-dns https://kubernetes-sigs.github.io/external-dns/ helm repo update helm install external-dns external-dns/external-dns \ -namespace external-dns \ --create-namespace \ -f external-dns-values.yaml
domainFilters: - okhk.net env: - name: CF_API_TOKEN valueFrom: secretKeyRef: key: apiKey name: external-dns extraArgs: - --cloudflare-dns-records-per-page=5000 - --cloudflare-record-comment=provisioned by external-dns - --events - --min-event-sync-interval=5s - --batch-change-size=200 - --batch-change-interval=1s - --metrics-address=:7979 - --events-emit=RecordError interval: 1m logFormat: json logLevel: debug policy: upsert provider: name: cloudflare registry: txt secretConfiguration: data: apiKey: enabled: true mountPath: /etc/secrets sources: - gateway-httproute txtOwnerId: edns txtPrefix: _externaldns.
- Tailscale 路由:
external-dns默认解析 Gateway 的外部 IP 并生成 A 记录,正好契合 Tailscale LoadBalancer 的 IP 地址。
- Cloudflare Tunnel 路由:在 Gateway 资源上添加注解
external-dns.kubernetes.io/target指向 Tunnel CNAME,并在HTTPRoute上添加注解以自动开启 Cloudflare Proxied:
Gateway 中配置 CNAME 记录指向目标:external-dns.kubernetes.io/target
apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: annotations: external-dns.kubernetes.io/target: <Tunnel-ID>.cfargotunnel.com name: cloudflare-gateway namespace: envoy-gateway-system spec: gatewayClassName: eg infrastructure: parametersRef: group: gateway.envoyproxy.io kind: EnvoyProxy name: cf-proxy-config listeners: - allowedRoutes: namespaces: from: All name: http port: 80 protocol: HTTP - allowedRoutes: namespaces: from: All hostname: "*.okhk.net" name: https port: 443 protocol: HTTPS tls: certificateRefs: - kind: Secret name: okhk-tls-secret mode: Terminate --- apiVersion: gateway.networking.k8s.io/v1 kind: Gateway metadata: name: tailscale-gateway namespace: envoy-gateway-system spec: gatewayClassName: eg listeners: - allowedRoutes: namespaces: from: All hostname: "*.okhk.net" name: https port: 443 protocol: HTTPS tls: certificateRefs: - kind: Secret name: okhk-tls-secret mode: Terminate
HTTPRoute 中配置开启 Cloudflare 代理: external-dns.kubernetes.io/cloudflare-proxied
metadata: annotations: external-dns.kubernetes.io/cloudflare-proxied: "true"
10.42.0.28 和 10.42.0.29 分别是两个 Gateway 实例的 Pod IP,可见确实是两个 Gateway 分别转发的。

HTTPRoute 并绑定到不同的 Gateway,即可无缝完成私有服务与公开服务的隔离发布。