Kubernetes

Kubernetes 分布式部署（2）容器环境配置

By jiutingqiu

1、部署Docker

下载docker静态包
https://download.docker.com

一般的我们会去这个路径下下载
https://download.docker.com/linux/static/stable/

root@k8s-master-u2404-4-20-101:~# mkdir docker_intall
root@k8s-master-u2404-4-20-101:~# cd docker_intall/
root@k8s-master-u2404-4-20-101:~/docker_intall# wget https://download.docker.com/linux/static/stable/aarch64/docker-26.1.0.tgz

root@k8s-master-u2404-4-20-101:~/docker_intall# tar xf docker-26.1.0.tgz 
root@k8s-master-u2404-4-20-101:~/docker_intall# cp -a docker/* /usr/bin/

接下来配置 containerd 和 docker 的 systemd

这里简单说一下

docker 原先是一个完整的容器引擎，后来将containerd（运行时）拆分出来,docker负责调用containerd

docker -> containerd -> runc

Containerd Systemd

下面我注释的部分是可以给普通用户使用

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /etc/systemd/system/containerd.service
[Unit]
Description=containerd container runtime
Documentation=https://containerd.io
After=network.target local-fs.target

[Service]
#非root使用
#ExecStartPre=/bin/mkdir -m 0755 -p /run/containerd
#ExecStartPost=/bin/chmod 666 /run/containerd/containerd.sock
ExecStartPre=-/sbin/modprobe overlay
ExecStart=/usr/bin/containerd
Type=notify
Delegate=yes
KillMode=process
Restart=always
RestartSec=5
LimitNPROC=infinity
LimitCORE=infinity
LimitNOFILE=1048576
TasksMax=infinity
OOMScoreAdjust=-999

[Install]
WantedBy=multi-user.target

root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl enable --now containerd.service
Created symlink /etc/systemd/system/multi-user.target.wants/containerd.service → /etc/systemd/system/containerd.service.
root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl status containerd
● containerd.service - containerd container runtime
     Loaded: loaded (/etc/systemd/system/containerd.service; enabled; preset: enabled)
     Active: active (running) since Sat 2025-07-19 20:40:02 CST; 9s ago
       Docs: https://containerd.io
    Process: 3079 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 3083 (containerd)
      Tasks: 7
     Memory: 9.8M (peak: 10.3M)
        CPU: 185ms
     CGroup: /system.slice/containerd.service
             └─3083 /usr/bin/containerd

Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421595912+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421791266+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421821684+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421836486+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421844763+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421847699+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421985111+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.421992532+08:00" level=i>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 systemd[1]: Started containerd.service - containerd container runtim>
Jul 19 20:40:02 k8s-master-u2404-4-20-101 containerd[3083]: time="2025-07-19T20:40:02.423193853+08:00" level=i>
lines 1-22/22 (END)

Docker Systemd 和 Docker Socket

前面说到docker现在主要调用containerd，所以需要一种通信方式，通信方式有很多比如信号量，socket，tcp。。。这里主要是用socket

故而我们需要一个socket

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /etc/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket containerd.service

[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP 
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
OOMScoreAdjust=-500

[Install]
WantedBy=multi-user.target

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /etc/systemd/system/docker.socket 
[Unit]
Description=Docker Socket for the API

[Socket]
ListenStream=/var/run/docker.sock
SocketMode=0660
SocketUser=root
#加入docker组就可以
SocketGroup=docker

[Install]
WantedBy=sockets.target

root@k8s-master-u2404-4-20-101:~/docker_intall# groupadd docker
root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl enable --now docker.socket  && systemctl enable --now docker.service
Created symlink /etc/systemd/system/sockets.target.wants/docker.socket → /etc/systemd/system/docker.socket.
Created symlink /etc/systemd/system/multi-user.target.wants/docker.service → /etc/systemd/system/docker.service.

root@k8s-master-u2404-4-20-101:~/docker_intall# docker info
Client:
 Version:    26.1.0
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 26.1.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 926c9586fe4a6236699318391cd44976a98e31f1
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-64-generic
 Operating System: Ubuntu 24.04.2 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 3.811GiB
 Name: k8s-master-u2404-4-20-101
 ID: 3793990f-39e8-4f88-bdb9-41d326b28c48
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

2、配置 Docker 代理

这里主要有两种

daemon.json 配置代理

docker.service 配置代理

daemon.json 配置代理

文件路径：/etc/docker/daemon.json

root@k8s-master-u2404-4-20-101:~/docker_intall# mkdir /etc/docker
root@k8s-master-u2404-4-20-101:~/docker_intall# vim /etc/docker/daemon.json 
{
  "exec-opts": ["native.cgroupdriver=systemd"],
  "registry-mirrors": [
    "https://docker.mirrors.ustc.edu.cn",
    "http://hub-mirror.c.163.com"
  ],
  "max-concurrent-downloads": 10,
  "log-driver": "json-file",
  "log-level": "warn",
  "log-opts": {
    "max-size": "10m",
    "max-file": "3"
  },
  "data-root": "/var/lib/docker"
}

#----------------------------或者-------------------------------
#把127.0.0.1:7890换成你的代理地址
{
  "proxies": {
    "default": {
      "httpProxy": "http://127.0.0.1:7890",
      "httpsProxy": "http://127.0.0.1:7890",
      "noProxy": "*.test.example.com,.example2.com,127.0.0.0/8"
    }
  }
}
}

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /etc/docker/daemon.json
root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl restart docker
root@k8s-master-u2404-4-20-101:~/docker_intall# docker info
Client:
 Version:    26.1.0
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 26.1.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 926c9586fe4a6236699318391cd44976a98e31f1
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-64-generic
 Operating System: Ubuntu 24.04.2 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 3.811GiB
 Name: k8s-master-u2404-4-20-101
 ID: 3793990f-39e8-4f88-bdb9-41d326b28c48
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Registry Mirrors: #此处
  https://docker.mirrors.ustc.edu.cn/ 
  http://hub-mirror.c.163.com/
 Live Restore Enabled: false
 Product License: Community Engine

docker.service配置代理

文件路径：/etc/systemd/system/docker.service

不过这种方式的我一般不这么用，一般情况下我会写在/etc/systemd/system/docker.service.d/http-proxy.conf

vim /etc/systemd/system/docker.service.d/http-proxy.conf #添加 [Service] Environment="HTTP_PROXY=http://127.0.0.1:7897" Environment="HTTPS_PROXY=http://127.0.0.1:7897" Environment="NO_PROXY=localhost,127.0.0.1,.mycompany.com"

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /etc/systemd/system/docker.service
root@k8s-master-u2404-4-20-101:~/docker_intall# cat /etc/systemd/system/docker.service
[Unit]
Description=Docker Application Container Engine
Documentation=https://docs.docker.com
After=network-online.target firewalld.service containerd.service
Wants=network-online.target
Requires=docker.socket containerd.service

[Service]
Type=notify
ExecStart=/usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
ExecReload=/bin/kill -s HUP 
TimeoutSec=0
RestartSec=2
Restart=always
StartLimitBurst=3
StartLimitInterval=60s
LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity
TasksMax=infinity
Delegate=yes
KillMode=process
OOMScoreAdjust=-500
Environment="HTTP_PROXY=http://127.0.0.1:7897"
Environment="HTTPS_PROXY=http://127.0.0.1:7897"
Environment="NO_PROXY=localhost,127.0.0.1,.mycompany.com"

[Install]
WantedBy=multi-user.target

root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl daemon-reload 
root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl restart docker
root@k8s-master-u2404-4-20-101:~/docker_intall# docker info
Client:
 Version:    26.1.0
 Context:    default
 Debug Mode: false

Server:
 Containers: 0
  Running: 0
  Paused: 0
  Stopped: 0
 Images: 0
 Server Version: 26.1.0
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 926c9586fe4a6236699318391cd44976a98e31f1
 runc version: v1.1.12-0-g51d5e94
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.8.0-64-generic
 Operating System: Ubuntu 24.04.2 LTS
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 3.811GiB
 Name: k8s-master-u2404-4-20-101
 ID: 3793990f-39e8-4f88-bdb9-41d326b28c48
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 HTTP Proxy: http://127.0.0.1:7897
 HTTPS Proxy: http://127.0.0.1:7897
 No Proxy: localhost,127.0.0.1,.mycompany.com
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false
 Product License: Community Engine

3、配置 cri-docker

cri-dockerd 是一个用于在 Kubernetes 中继续使用 Docker 作为容器运行时的工具。 由于 Kubernetes 自 1.20 版本开始逐步弃用对 Dockershim（Kubernetes 内部与 Docker 通信的中间层）的支持，到 1.24 版本完全移除 Dockershim，这意味着 Docker 不再直接作为 Kubernetes 的容器运行时。 cri-dockerd 项目诞生的目的是弥补这一缺口，使用户能够继续在 Kubernetes 中使用 Docker 作为容器运行时。它充当了 Docker 和 Kubernetes CRI 之间的桥梁。

既然是中间层，那么肯定就得有通信的方式，这里用的也是socket

下载cri-docker

https://github.com/Mirantis/cri-dockerd/

root@k8s-master-u2404-4-20-101:~/docker_intall# wget https://github.com/Mirantis/cri-dockerd/releases/download/v0.3.13/cri-dockerd-0.3.13.arm64.tgz
root@k8s-master-u2404-4-20-101:~/docker_intall# tar xf cri-dockerd-0.3.13.arm64.tgz 
root@k8s-master-u2404-4-20-101:~/docker_intall# cp -a cri-dockerd/cri-dockerd /usr/bin/
root@k8s-master-u2404-4-20-101:~/docker_intall# chmod +x /usr/bin/cri-dockerd

cri-docker启动文件

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /usr/lib/systemd/system/cri-docker.service
[Unit]
Description=CRI Interface for Docker Application Container Engine
Documentation=https://docs.mirantis.com
After=network-online.target firewalld.service docker.service
Wants=network-online.target
Requires=cri-docker.socket

[Service]
Type=notify
ExecStart=/usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.com/google_containers/pause:3.9
ExecReload=/bin/kill -s HUP 
TimeoutSec=0
RestartSec=2
Restart=always

StartLimitBurst=3

StartLimitInterval=60s

LimitNOFILE=infinity
LimitNPROC=infinity
LimitCORE=infinity

TasksMax=infinity
Delegate=yes
KillMode=process

[Install]
WantedBy=multi-user.target

root@k8s-master-u2404-4-20-101:~/docker_intall# vim /usr/lib/systemd/system/cri-docker.socket
[Unit]
Description=CRI Docker Socket for the API
PartOf=cri-docker.service

[Socket]
ListenStream=%t/cri-dockerd.sock
SocketMode=0660
SocketUser=root
SocketGroup=docker

[Install]
WantedBy=sockets.target

root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl daemon-reload ; systemctl enable cri-docker --now
Created symlink /etc/systemd/system/multi-user.target.wants/cri-docker.service → /usr/lib/systemd/system/cri-docker.service.

root@k8s-master-u2404-4-20-101:~/docker_intall# systemctl status cri-docker
● cri-docker.service - CRI Interface for Docker Application Container Engine
     Loaded: loaded (/usr/lib/systemd/system/cri-docker.service; enabled; preset: enabled)
     Active: active (running) since Sat 2025-07-19 21:37:01 CST; 2min 41s ago
TriggeredBy: ● cri-docker.socket
       Docs: https://docs.mirantis.com
   Main PID: 4678 (cri-dockerd)
      Tasks: 8
     Memory: 8.5M (peak: 9.4M)
        CPU: 100ms
     CGroup: /system.slice/cri-docker.service
             └─4678 /usr/bin/cri-dockerd --network-plugin=cni --pod-infra-container-image=registry.aliyuncs.co>

Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 cri-dockerd[4678]: time="2025-07-19T21:37:01+08:00" level=info msg=">
Jul 19 21:37:01 k8s-master-u2404-4-20-101 systemd[1]: Started cri-docker.service - CRI Interface for Docker Ap>
~

4、碎碎念

关于docker

docker单拿出来，目前最好的用途是做交付工具，这里最常用的技巧应该就是

多阶段构建docker镜像，对镜像进行瘦身

简单来说，就是我先启动一个镜像然后在里面完成编译，保存这个镜像，然后我在启一个镜像 COPY这个镜像里的产物

这样就丢掉了所在运行时不需要的东西，例如命令，缓存，本体镜像等

关于cri-docker和docker

目前kubernetes已经弃用了docker，但是可以直接支持containerd和CRI-O

所以我们使用docker的话还得使用cri-docker作为中间层

By jiutingqiu

Related Post

Kubernetes

Kubernetes 分布式部署（4）ETCD 部署配置

jiutingqiu
Kubernetes

Kubernetes 分布式部署（3）证书配置

jiutingqiu
Kubernetes

Kubernetes 分布式部署（1）Linux基础配置

jiutingqiu

发表回复

您的电子邮箱地址不会被公开。 必填项已用*标注

You Missed

Kubernetes

Kubernetes 分布式部署（4）ETCD 部署配置

Kubernetes

Kubernetes 分布式部署（3）证书配置

Kubernetes

Kubernetes 分布式部署（2）容器环境配置

Kubernetes

Kubernetes 分布式部署（1）Linux基础配置