kubectl Cheatsheet
Working with Pods
How to force restart a pod
$ kubectl get pod PODNAME -n NAMESPACE -o yaml | kubectl replace --force -f -
How to get a list of pending pods
$ kubectl get pods --field-selector=status.phase=Pending
How to Delete multiple pods?
Delete multiple pods by label:
$ kubectl delete pods -l app=my-app -n default
Delete multiple pods by name:
$ kubectl get pods -n $NAMESPACE --no-headers=true | awk '/pattern/{print $1}'| xargs kubectl delete -n $NAMESPACE pod
$ kubectl get pods -n $NAMESPACE | grep $PATTERN | awk '{print $2}' | xargs kubectl delete pod -n $NAMESPACE
Delete all completed / failed pods
$ kubectl --kubeconfig KUBECONFIG delete pods -A --field-selector status.phase=Succeeded
$ kubectl --kubeconfig KUBECONFIG delete pods -A --field-selector status.phase=Failed
Force delete all pods in a namespace:
$ kubectl delete pod --all --grace-period=0 --force --namespace foo-system
Force delete all terminating pods
$ kubectl get pods -A | grep Terminating | awk '{print $2 " -n=" $1}' | xargs kubectl delete pod --grace-period=0 --force
How to restrict pods to only run on the control-plane nodes?
$ kubectl patch -n kubevirt kubevirt kubevirt --type merge --patch '{"spec": {"infra": {"nodePlacement": {"nodeSelector": {"node-role.kubernetes.io/control-plane": ""}}}}}'
To restrict the virt-handler
pods to only run on nodes with the region=primary
label:
$ kubectl patch -n kubevirt kubevirt kubevirt --type merge --patch '{"spec": {"workloads": {"nodePlacement": {"nodeSelector": {"region": "primary"}}}}}'
Clusters
# Get Clusters.
$ kubectl config get-clusters
# Get Cluster Info
$ kubectl cluster-info
Kubernetes control plane is running at https://127.0.0.1:36397
CoreDNS is running at https://127.0.0.1:36397/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
$ kubectl cluster-info dump
Specify output columns
$ kubectl get services -A -o=custom-columns=NAME:.metadata.name,Namespace:.metadata.namespace
API Resources
To see which Kubernetes resources are and aren't in a namespace:
# In a namespace
$ kubectl api-resources --namespaced=true
# Not in a namespace
$ kubectl api-resources --namespaced=false
Check resources
# Get a list of Services:
$ kubectl get services
# Check the service accounts:
$ kubectl -n kube-system get sa
# Get pods on a specific node.
$ kubectl get pods --all-namespaces -o wide --field-selector spec.nodeName=$NODE
# Get num of running pods.
$ kubectl get pods -A --field-selector status.phase=Running | wc -l
If there are multiple resources with the same name (e.g. Cluster
), add the apigroup to it:
$ kubectl get clusters.cluster.x-k8s.io
How to list all resources in a namespace
$ kubectl api-resources --verbs=list --namespaced -o name | xargs -n 1 kubectl get --show-kind --ignore-not-found -n NAMESPACE
How to delete all objects of a certain kind in a namespace?
$ kubectl get KIND -n NAMESPACE -o name | xargs -I{} kubectl delete {} -n NAMESPACE
Check resource consumption
$ kubectl top node
$ kubectl top pod -A
Who Am I and What Can I Do?
Who Am I? Use whoami
command to check username and groups:
$ kubectl auth whoami
ATTRIBUTE VALUE
Username kubernetes-admin
Groups [system:masters system:authenticated]
This should match the cert:
$ cat /path/to/kubeconfig | yq '.users[0].user.client-certificate-data' | base64 -d | openssl x509 -text -noout | grep "Subject:"
Subject: O = system:masters, CN = kubernetes-admin
Check config
# Show current-context
$ kubectl config current-context
# Check details of the Config
$ kubectl config view
# use a different context
$ kubectl config use-context CONTEXT_NAME
What can i do?
# List all
$ kubectl auth can-i --list
# Check to see if I can do everything in my current namespace ("*" means all)
$ kubectl auth can-i '*' '*'
# Check to see if I can create pods in any namespace
$ kubectl auth can-i create pods --all-namespaces
# Check to see if I can list deployments in my current namespace
$ kubectl auth can-i list deployments.extensions
kubectl patch
3 types:
--type=strategy
--type=merge
--type=json
--type=strategy
The default. Not supported for Custom Resources.
$ kubectl patch serviceaccount NAME -n NAMESPACE -p '{"imagePullSecrets": [{"name": "IMAGE_PULL_SECRET_NAME"}]}'
From the go code:
exec.Command("kubectl", "patch", "serviceaccount",
"NAME",
"-n", "NAMESPACE",
"-p", `'{"imagePullSecrets": [{"name": "IMAGE_PULL_SECRET_NAME"}]}'`).Run()
--type=json
$ kubectl patch KIND NAME -n NAMESPACE --type=json -p="[{'op': 'remove', 'path': '/metadata/finalizers'}]"
op
can be: add
, replace
, remove
.
Note that when setting an object to empty: replace
with {}
does not work ,use remove
.
Read more: https://jsonpatch.com/
Search string in resources
# use grep, but hard to see which pod it is.
$ kubectl get pod -A -o yaml | grep "something"
# use jq, get pod name.
$ kubectl get pod -A -o json | jq -r '.items[] | select(tostring | contains("something")) | .metadata.name'
How to check Node Status
e.g. check ephemeral storage
$ kubectl get --raw "/api/v1/nodes/$NODE_NAME/proxy/stats/summary"
# equivalent to
$ curl http://$HOST:$PORT/api/v1/nodes/$NODE_NAME/proxy/stats/summary
# and
$ kubectl get --raw "/api/v1/nodes/$NODE_NAME/proxy/metrics/resource"
$ kubectl get --raw "/api/v1/nodes/$NODE_NAME/proxy/metrics/cadvisor"
Working with PV
Check capacities:
$ kubectl describe pv
$ kubectl describe pvc
The PV's Status
should be "Bound"
if it has been successfully allocated to the application.
Check remaining disk space:
$ kubectl -n NAMESPACE exec POD_NAME -- df -ah
More examples:
# How to get PVs of a namespace
$ kubectl get pv -o json | jq -r '.items[] | select(.spec.claimRef.namespace == "NAMESPACE") | .metadata.name'
# How to change the reclaim policies of the persistent volumes to Retain.
$ kubectl patch pv/${NAME} -p "{'spec':{'persistentVolumeReclaimPolicy':'Retain'}}"
# How to remove claimRef
$ kubectl patch pv/${NAME} --type json -p '[{"op":"remove","path":"/spec/claimRef"}]'
Working with Secret
# Get and decode secret
$ kubectl get secret SECRET_NAME -n NAMESPACE --template="{{index .data \"ca.crt\" | base64decode}}" > https.crt
# cert is stored in certificate-authority-data in kubeconfig
$ curl $(kubectl config view --minify --output 'jsonpath={..cluster.server}')
# curl: (60) SSL certificate problem: unable to get local issuer certificate
# get cert
$ kubectl config view --minify --raw --output 'jsonpath={..cluster.certificate-authority-data}' | base64 -d > /tmp/kubectl-cacert
$ curl --cacert /tmp/kubectl-cacert $(kubectl config view --minify --output 'jsonpath={..cluster.server}')
# should get 403
How to Get ClusterRoleBinding of a specific subject kind / name?
$ kubectl get clusterrolebindings -o json | jq -r '.items[] | select(.subjects[0].kind=="Group") | select(.subjects[0].name=="system:masters")'
How to Show init containers and normal containers.
$ kubectl get -A pod -o="custom-columns=NAME:.metadata.name,INIT-CONTAINERS:.spec.initContainers[*].name,CONTAINERS:.spec.containers[*].name"
How to apply a YAML?
Apply a file:
$ kubectl apply -f ./foo.yaml
Apply from commandline with raw text:
$ kubectl apply -f - <<EOF
apiVersion: v1
kind: Namespace
metadata:
name: example-namespace
EOF
How to force delete a CR (by deleting finalizers)?
Sometimes the CR deletion is blocked by finalizers, so the object will be stuck in Terminating state. To delete finalizers:
$ kubectl patch KIND NAME -n NAMESPACE --type=json -p="[{'op': 'remove', 'path': '/metadata/finalizers'}]"
How to check the x509 certificate?
# Check the cert in a Secret
$ kubectl get secret -n foo-system foo-serving-cert -o json | jq -r '.data."ca.crt"' | base64 -d | openssl x509 -text | less
# Check the cert in a CertificateRequest
$ kubectl get certificaterequest -n foo-system foo-serving-cert-p8795 -o json | jq -r '.status.ca' | base64 -d | openssl x509 -text | less
How to Renew a Certificate?
Certificates are stored by cert-manager
inside a Secret
, deleting this Secret
triggers a certificate renewal.
Note: Delete the Secret
holding the certificate, not the Certificate
itself.
# Get the name of the Secret:
SECRET_NAME=$(kubectl -n foo-system get Certificate foo-serving-cert -o jsonpath='{.spec.secretName}')
# Delete the Secret to trigger certificate renewal.
$ kubectl --kubeconfig ${KUBECONFIG:?} -n gpc-system delete Secret ${SECRET_NAME}
How to deal with kinds with the same name?
If you have multiple types named Cluster
, you can specify the one with KIND.VERSION.GROUP
.
For example, to delete the Cluster
in v1
of foo.example.com
group:
$ kubectl delete clusters.v1.foo.example.com NAME -n NAMESPACE
How to add/modify and remove annotation?
# add or modify an annotation
$ kubectl annotate KIND NAME -n NAMESPACE foo.example.com/paused=true
# remove an annotation, add `-` at the end
$ kubectl annotate KIND NAME -n NAMESPACE foo.example.com/paused-
How to test certificate from commandline?
$ kubectl get secret SECRET_NAME -n cert-manager -ojsonpath='{.data.ca\.crt}' | base64 --decode > trust.crt
$ openssl s_client -connect some.domain.example.com:443 -CAfile trust.crt
How to save full logs?
Sometimes the logs may be rotated out. To save the full log for debugging, dump the logs:
while true ; do kubectl logs -l name=label-name -n foo-system --tail -1 > $(date +"%Y-%m-%d-%H-%M-%S")-log; sleep 30 ; done
How to remove unnecessary fields when dumping the manifest?
$ kubectl get KIND NAME -n NAMESPACE -o json | \
jq "del(.status, .metadata.annotations, .metadata.creationTimestamp,
.metadata.finalizers, .metadata.generation,
.metadata.resourceVersion, .metadata.uid)" > md.json
How to find all possible clusters in a kubeconfig?
Your KUBECONFIG
may have multiple contexts:
$ kubectl config view -o jsonpath='{"Cluster name\tServer\n"}{range .clusters[*]}{.name}{"\t"}{.cluster.server}{"\n"}{end}'
How to update ConfigMaps?
# see what changes would be made, returns nonzero returncode if different
$ kubectl get configmap kube-proxy -n kube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | \
kubectl diff -f - -n kube-system
# actually apply the changes, returns nonzero returncode on errors only
$ kubectl get configmap kube-proxy -n kube-system -o yaml | \
sed -e "s/strictARP: false/strictARP: true/" | \
kubectl apply -f - -n kube-system
How to update status?
The normal kubectl patch
cannot change status
. Add --subresource=status
:
$ kubectl patch KIND NAME -n NAMESPACE --subresource=status --type=json -p="[{'op': 'replace', 'path': '/status/myConditions/0/status', 'value': 'True'}]"
Note that if /status/myConditions
is a list, you can select the first one by /status/myConditions/0/status
.
Or use --type=merge
:
$ kubectl patch KIND NAME -n NAMESPACE --type=merge --subresource=status -p '{
"status": {
"conditions": [{
"type": "...",
"status": "True",
"reason": "...",
"message": "...",
"lastTransitionTime": "1900-10-10T00:00:00Z"
}]
}
}'
How to get raw info?
# Check API priority and fairness
$ kubectl get --raw /debug/api_priority_and_fairness/dump_priority_levels
$ kubectl get --raw /debug/api_priority_and_fairness/dump_queues
$ kubectl get --raw /debug/api_priority_and_fairness/dump_requests
# Check status
$ kubectl get --raw='/readyz?verbose'
Plugins
Add the tree plugin to visualize
$ kubectl krew install tree
Full list of plugins: https://github.com/kubernetes-sigs/krew-index/tree/master/plugins
Well-known ports
6443
: API Server (orhaproxy
)2379
/2380
:etcd
10250
:kubelet
10256
:kube-proxy
10257
:kube-controller-manager
10259
:kube-scheduler
30000
-32767
: NodePort Services
Troubleshooting
Unable to use a TTY - input is not a terminal or the right kind of file
If you see this error when running kubectl exec -it
, try to remove -t
.
Deprecation
kubectl kustomize
subcommand, --kustomize
flag are being deprecated. kustomize
will be added in krew index. https://github.com/kubernetes/enhancements/blob/master/keps/sig-cli/4706-deprecate-and-remove-kustomize/README.md