【K8s】【笔记】----第一章Kubernetes 介绍【K8s】【笔记】----第二章Kubernetes 集群环境搭建【K8s】【笔记】----第三章Kubernetes 资源管理【K8s】【笔记】----第四章Kubernetes 实战入门【K8s】【笔记】----第五章Kubernetes Pod详解【K8s】【笔记】----第六章Kubernetes Pod控制器详解【K8s】【笔记】----第七章Kubernetes Service详解【K8s】【笔记】----第八章Kubernetes 数据存储【K8s】【笔记】----第九章Kubernetes 安全认证文章目录2. kubernetes集群环境搭建 ️2.1 前置知识点2.2 kubeadm 部署方式介绍2.3 安装要求2.4 最终目标2.5 准备环境2.6 系统初始化2.6.1 设置系统主机名以及 Host 文件的相互解析2.6.2 安装依赖文件所有节点都要操作2.6.3 设置防火墙为 Iptables 并设置空规则所有节点都要操作2.6.4 关闭 SELINUX所有节点都要操作2.6.5 调整内核参数对于 K8S所有节点都要操作2.6.6 调整系统时区所有节点都要操作2.6.7 设置 rsyslogd 和 systemd journald所有节点都要操作2.6.8 kube-proxy开启ipvs的前置条件所有节点都要操作2.6.9 安装 Docker 软件所有节点都要操作2.6.10 安装 Kubeadm 所有节点都要操作2.7 部署Kubernetes Master2.7.1 初始化主节点主节点操作2.7.2 加入主节点以及其余工作节点2.7.3 部署网络2.8 测试kubernetes 集群2.8.1 部署nginx 测试2. kubernetes集群环境搭建 ️2.1 前置知识点目前生产部署Kubernetes 集群主要有两种方式kubeadm⚡Kubeadm 是一个K8s 部署工具提供kubeadm init 和kubeadm join用于快速部署Kubernetes 集群。官方地址https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm/二进制包从github 下载发行版的二进制包手动部署每个组件组成Kubernetes 集群。Kubeadm 降低部署门槛但屏蔽了很多细节遇到问题很难排查。如果想更容易可控推荐使用二进制包部署Kubernetes 集群虽然手动部署麻烦点期间可以学习很多工作原理也利于后期维护。2.2 kubeadm 部署方式介绍kubeadm 是官方社区推出的一个用于快速部署kubernetes 集群的工具这个工具能通过两条指令完成一个kubernetes 集群的部署创建一个Master 节点kubeadm init将Node 节点加入到当前集群中$ kubeadm join Master 节点的IP 和端口2.3 安装要求在开始之前部署Kubernetes 集群机器需要满足以下几个条件一台或多台机器操作系统CentOS7.x-86_x64硬件配置2GB 或更多RAM2 个CPU 或更多CPU硬盘30GB 或更多集群中所有机器之间网络互通可以访问外网需要拉取镜像禁止swap 分区2.4 最终目标在所有节点上安装Docker 和kubeadm部署Kubernetes Master部署容器网络插件部署Kubernetes Node将节点加入Kubernetes 集群中部署Dashboard Web 页面可视化查看Kubernetes 资源2.5 准备环境角色IP地址组件k8s-master01192.168.5.3dockerkubectlkubeadmkubeletk8s-node01192.168.5.4dockerkubectlkubeadmkubeletk8s-node02192.168.5.5dockerkubectlkubeadmkubelet2.6 系统初始化2.6.1 设置系统主机名以及 Host 文件的相互解析hostnamectl set-hostname k8s-master01bashhostnamectl set-hostname k8s-node01bashhostnamectl set-hostname k8s-node02bashcatEOF/etc/hosts192.168.5.3 k8s-master01192.168.5.4 k8s-node01192.168.5.5 k8s-node02 EOFscp/etc/hosts root192.168.5.4:/etc/hostsscp/etc/hosts root192.168.5.5:/etc/hosts2.6.2 安装依赖文件所有节点都要操作yuminstall-yconntrack ntpdate ntp ipvsadm ipset jq iptablescurlsysstat libseccompwgetvimnet-toolsgit2.6.3 设置防火墙为 Iptables 并设置空规则所有节点都要操作systemctl stop firewalldsystemctl disable firewalld yum-yinstalliptables-servicessystemctl start iptablessystemctlenableiptablesiptables-Fserviceiptables save2.6.4 关闭 SELINUX所有节点都要操作swapoff-ased-i/ swap / s/^\(.*\)$/#\1/g/etc/fstab setenforce0sed-is/^SELINUX.*/SELINUXdisabled//etc/selinux/config2.6.5 调整内核参数对于 K8S所有节点都要操作modprobe br_netfiltercatEOFkubernetes.conf net.bridge.bridge-nf-call-iptables1net.bridge.bridge-nf-call-ip6tables1net.ipv4.ip_forward1net.ipv4.tcp_tw_recycle0vm.swappiness0# 禁止使用 swap 空间只有当系统 OOM 时才允许使用它vm.overcommit_memory1# 不检查物理内存是否够用vm.panic_on_oom0# 开启 OOMfs.inotify.max_user_instances8192fs.inotify.max_user_watches1048576fs.file-max52706963fs.nr_open52706963net.ipv6.conf.all.disable_ipv61net.netfilter.nf_conntrack_max2310720EOFcpkubernetes.conf /etc/sysctl.d/kubernetes.confsysctl-p/etc/sysctl.d/kubernetes.conf2.6.6 调整系统时区所有节点都要操作# 设置系统时区为 中国/上海timedatectl set-timezone Asia/Shanghai# 将当前的 UTC 时间写入硬件时钟timedatectl set-local-rtc0# 重启依赖于系统时间的服务systemctl restart rsyslog systemctl restart crond2.6.7 设置 rsyslogd 和 systemd journald所有节点都要操作# 持久化保存日志的目录mkdir/var/log/journalmkdir/etc/systemd/journald.conf.dcat/etc/systemd/journald.conf.d/99-prophet.confEOF [Journal] # 持久化保存到磁盘 Storagepersistent # 压缩历史日志 Compressyes SyncIntervalSec5m RateLimitInterval30s RateLimitBurst1000 # 最大占用空间 10G SystemMaxUse10G # 单日志文件最大 200M SystemMaxFileSize200M # 日志保存时间 2 周 MaxRetentionSec2week # 不将日志转发到 syslog ForwardToSyslogno EOFsystemctl restart systemd-journald2.6.8 kube-proxy开启ipvs的前置条件所有节点都要操作catEOF/etc/sysconfig/modules/ipvs.modules#!/bin/bashmodprobe -- ip_vs modprobe -- ip_vs_rr modprobe -- ip_vs_wrr modprobe -- ip_vs_sh modprobe -- nf_conntrack_ipv4 EOFchmod755/etc/sysconfig/modules/ipvs.modulesbash/etc/sysconfig/modules/ipvs.moduleslsmod|grep-eip_vs-enf_conntrack_ipv42.6.9 安装 Docker 软件所有节点都要操作yuminstall-yyum-utils device-mapper-persistent-data lvm2 yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo yuminstall-ydocker-ce## 创建 /etc/docker 目录mkdir/etc/dockercat/etc/docker/daemon.jsonEOF { exec-opts: [native.cgroupdriversystemd], log-driver: json-file, log-opts: { max-size: 100m } } EOFmkdir-p/etc/systemd/system/docker.service.d# 重启docker服务systemctl daemon-reloadsystemctl restartdockersystemctlenabledocker上传文件到/etc/yum.repos.d/目录下也可以 代替yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo命令docker-ce.repo[docker-ce-stable]nameDocker CE Stable -$basearchbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/stableenabled1gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-stable-debuginfo]nameDocker CE Stable - Debuginfo$basearchbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/stableenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-stable-source]nameDocker CE Stable - Sourcesbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/stableenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-test]nameDocker CE Test -$basearchbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/testenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-test-debuginfo]nameDocker CE Test - Debuginfo$basearchbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/testenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-test-source]nameDocker CE Test - Sourcesbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/testenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-nightly]nameDocker CE Nightly -$basearchbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/$basearch/nightlyenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-nightly-debuginfo]nameDocker CE Nightly - Debuginfo$basearchbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/debug-$basearch/nightlyenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg[docker-ce-nightly-source]nameDocker CE Nightly - Sourcesbaseurlhttps://mirrors.aliyun.com/docker-ce/linux/centos/$releasever/source/nightlyenabled0gpgcheck1gpgkeyhttps://mirrors.aliyun.com/docker-ce/linux/centos/gpg2.6.10 安装 Kubeadm 所有节点都要操作catEOF/etc/yum.repos.d/kubernetes.repo[kubernetes] nameKubernetes baseurlhttp://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 enabled1 gpgcheck0 repo_gpgcheck0 gpgkeyhttp://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg EOFyuminstall-ykubelet kubeadm kubectlsystemctlenablekubelet2.7 部署Kubernetes Master2.7.1 初始化主节点主节点操作kubeadm init --apiserver-advertise-address192.168.5.3 --image-repository registry.aliyuncs.com/google_containers --kubernetes-version v1.21.1 --service-cidr10.96.0.0/12 --pod-network-cidr10.244.0.0/16mkdir-p$HOME/.kubesudocp-i/etc/kubernetes/admin.conf$HOME/.kube/configsudochown$(id-u):$(id-g)$HOME/.kube/config2.7.2 加入主节点以及其余工作节点kubeadmjoin192.168.5.3:6443--tokenh0uelc.l46qp29nxscke7f7\--discovery-token-ca-cert-hash sha256:abc807778e24bff73362ceeb783cc7f6feec96f20b4fd707c3f8e8312294e28f2.7.3 部署网络kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml下边是文件--- apiVersion: policy/v1beta1 kind: PodSecurityPolicy metadata: name: psp.flannel.unprivileged annotations: seccomp.security.alpha.kubernetes.io/allowedProfileNames: docker/default seccomp.security.alpha.kubernetes.io/defaultProfileName: docker/default apparmor.security.beta.kubernetes.io/allowedProfileNames: runtime/default apparmor.security.beta.kubernetes.io/defaultProfileName: runtime/default spec: privileged:falsevolumes: - configMap - secret - emptyDir - hostPath allowedHostPaths: - pathPrefix:/etc/cni/net.d- pathPrefix:/etc/kube-flannel- pathPrefix:/run/flannelreadOnlyRootFilesystem:false# Users and groupsrunAsUser: rule: RunAsAny supplementalGroups: rule: RunAsAny fsGroup: rule: RunAsAny# Privilege EscalationallowPrivilegeEscalation:falsedefaultAllowPrivilegeEscalation:false# CapabilitiesallowedCapabilities:[NET_ADMIN,NET_RAW]defaultAddCapabilities:[]requiredDropCapabilities:[]# Host namespaceshostPID:falsehostIPC:falsehostNetwork:truehostPorts: - min:0max:65535# SELinuxseLinux:# SELinux is unused in CaaSPrule:RunAsAny--- kind: ClusterRole apiVersion: rbac.authorization.k8s.io/v1 metadata: name: flannel rules: - apiGroups:[extensions]resources:[podsecuritypolicies]verbs:[use]resourceNames:[psp.flannel.unprivileged]- apiGroups: -resources: - pods verbs: - get - apiGroups: -resources: - nodes verbs: - list -watch- apiGroups: -resources: - nodes/status verbs: - patch --- kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: flannel roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: flannel subjects: - kind: ServiceAccount name: flannel namespace: kube-system --- apiVersion: v1 kind: ServiceAccount metadata: name: flannel namespace: kube-system --- kind: ConfigMap apiVersion: v1 metadata: name: kube-flannel-cfg namespace: kube-system labels: tier:nodeapp: flannel data: cni-conf.json:|{name:cbr0,cniVersion:0.3.1,plugins:[{type:flannel,delegate:{hairpinMode:true,isDefaultGateway:true}},{type:portmap,capabilities:{portMappings:true}}]}net-conf.json:|{Network:10.244.0.0/16,Backend:{Type:vxlan}}--- apiVersion: apps/v1 kind: DaemonSet metadata: name: kube-flannel-ds namespace: kube-system labels: tier:nodeapp: flannel spec: selector: matchLabels: app: flannel template: metadata: labels: tier:nodeapp: flannel spec: affinity: nodeAffinity: requiredDuringSchedulingIgnoredDuringExecution: nodeSelectorTerms: - matchExpressions: - key: kubernetes.io/os operator: In values: - linux hostNetwork:truepriorityClassName: system-node-critical tolerations: - operator: Exists effect: NoSchedule serviceAccountName: flannel initContainers: - name: install-cni image: quay.io/coreos/flannel:v0.14.0 command: -cpargs: --f- /etc/kube-flannel/cni-conf.json - /etc/cni/net.d/10-flannel.conflist volumeMounts: - name: cni mountPath: /etc/cni/net.d - name: flannel-cfg mountPath: /etc/kube-flannel/ containers: - name: kube-flannel image: quay.io/coreos/flannel:v0.14.0 command: - /opt/bin/flanneld args: - --ip-masq - --kube-subnet-mgr resources: requests: cpu:100mmemory:50Milimits: cpu:100mmemory:50MisecurityContext: privileged:falsecapabilities: add:[NET_ADMIN,NET_RAW]env: - name: POD_NAME valueFrom: fieldRef: fieldPath: metadata.name - name: POD_NAMESPACE valueFrom: fieldRef: fieldPath: metadata.namespace volumeMounts: - name: run mountPath: /run/flannel - name: flannel-cfg mountPath: /etc/kube-flannel/ volumes: - name: run hostPath: path: /run/flannel - name: cni hostPath: path: /etc/cni/net.d - name: flannel-cfg configMap: name: kube-flannel-cfg2.8 测试kubernetes 集群2.8.1 部署nginx 测试kubectl create deployment nginx--imagenginx kubectl expose deployment nginx--port80--typeNodePort kubectl get pod,svc