Bound Kubernetes Pod to Specified CPU

This is my second paper on medium, I will start writing blod every week about the technoloy using in my daily job.

For some cpu intense task, we can bound pod to specify cpu to avoid scramble for cpu.

Steps

1. drain the node

kubectl drain <NODE_NAME>

2. stop kubelet service

systemctl stop kubelet

if you are using k3s, you need to stop the k3s service instead.

systemctl stop k3s

3. change the kubelet config

We should change the cpu-manager-policy from none to static, you can add it to kubelet service as args

--cpu-manager-policy="static"

if you are using k3s, you can use the following args to specify kubelet config path

--kubelet-arg=config=/etc/rancher/k3s/kubelet.config

in the kubelet.config, you should have following content, change the cpuManagerPolicy to static.

You can add topologyManagerPolicy to single node to get better performance, make sure that get all cpu resource from single node, while not cross it. but it may cause schedule failed when require more resource than any one node can provided.

apiVersion: kubelet.config.k8s.io/v1beta1
kind: KubeletConfiguration
maxPods: 300
cpuManagerPolicy: static
topologyManagerPolicy: single-numa-node
systemReserved:
  cpu: "1"
  memory: "1Gi"
  ephemeral-storage: "2Gi"

You can get detail kubelet config iterms from the Kubelet Configuration (v1beta1) | Kubernetes

4. delete the old cpu manager status file

rm var/lib/kubelet/cpu_manager_state

5. restart kubelet service

systemctl start kubelet

if you are using k3s, you should restart k3s service instead

systemctl restart k3s

6. uncordon the node to make it schedulable

kubectl uncordon <NODE_NAME>

7. config pod QoS

Only a Pod that gets assigned a QoS class of Guaranteed can be scheduled to exclusive cpu pool, that means you should set pod requests same with limits in resources filed.

resources:
  limits:
    cpu: "1"
    ephemeral-storage: 10Gi
    memory: 1Gi
  requests:
    cpu: "1"
    ephemeral-storage: 10Gi
    memory: 1Gi

Check the process and cpu

We have 8 cpus each has 0–7 number, and 1 cpu is reserved in systemReserved config.

We set a Guaranteed pod request 1 cpu, the pod should be bound to a specified cpu. Our process name is ocp-driver-platform.

Get the pid of the process, and get the cpu set to the process by

taskset -pc <pid>

Get the cpu manager status by

cat/var/lib/kubelet/cpu_manager_state

Get the following data

cat /var/lib/kubelet/cpu_manager_state

You can get the ocp-driver-platform is bound to number 4 cpu.

Conclusion

if you want to bound a pod to a specified cpu to decrease the effect of the process to any other pod, you can make it by the above steps.

If anything wrong in the article, please let me know. Thanks for reading.