K8s: Crafting Custom Kubectl Shortcuts

“Unleashing the Full Potential of kubectl: Elevating Visibility and Efficiency with Custom Command Output and Shortcuts”

my kubectl

When using the kubectl command-line tool, the default output format for most operations is human-readable text. However, you can also specify different output like json or jsonpaths are helpful. In this article I want to showcase the cool capabilities of custom-columns and how useful it will be during our day to day operation with kubernetes with some practical examples.

1. Get the list of pods along with the node where it is running with container status and

➜ kubectl get pods -n nginx -o custom-columns="POD:.metadata.name,\
NODE:.spec.nodeName,POD_STATUS:.status.phase,\
CONTAINER_STATUS:.status.containerStatuses[*].state.waiting.reason"
POD                      NODE                 POD_STATUS   CONTAINER_STATUS
nginx-748c667d99-g8pq9   k8s-sample-worker3   Running      <none>
nginx-748c667d99-h67cc   k8s-sample-worker2   Running      <none>
nginx-748c667d99-x7z6k   k8s-sample-worker    Running      <none>
nginx-7bbc4565cc-szhpf   k8s-sample-worker3   Running      CrashLoopBackOff

I agree this can be obtained from using “-o wide” along with pod status,

➜ kubectl get pods -o wide -n nginx
NAME                     READY   STATUS             RESTARTS         AGE   IP           NODE                 NOMINATED NODE   READINESS GATES
nginx-748c667d99-g8pq9   1/1     Running            0                13h   10.244.1.2   k8s-sample-worker3   <none>           <none>
nginx-748c667d99-h67cc   1/1     Running            0                13h   10.244.3.2   k8s-sample-worker2   <none>           <none>
nginx-748c667d99-x7z6k   1/1     Running            0                13h   10.244.2.2   k8s-sample-worker    <none>           <none>
nginx-7bbc4565cc-szhpf   0/1     CrashLoopBackOff   54 (4m58s ago)   13h   10.244.1.3   k8s-sample-worker3   <none>           <none>

But I want more information on why the pod crashed

➜ kubectl get pods -n nginx -o custom-columns="POD:.metadata.name,\
NODE:.spec.nodeName,CONTAINER_FAIL_MESSAGE:.status.containerStatuses[*].lastState.terminated.message"
POD                      NODE                 CONTAINER_FAIL_MESSAGE
nginx-748c667d99-g8pq9   k8s-sample-worker3   <none>
nginx-748c667d99-h67cc   k8s-sample-worker2   <none>
nginx-748c667d99-x7z6k   k8s-sample-worker    <none>
nginx-7bbc4565cc-szhpf   k8s-sample-worker3   failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "bla": executable file not found in $PATH: unknown

Okay, this is much more useful. So I am creating an alias as below.

➜ alias getpodfail='kubectl get pods -o custom-columns="POD:.metadata.name,NODE:.spec.nodeName,CONTAINER_FAIL_MESSAGE:.status.containerStatuses[*].lastState.terminated.message"'

➜ getpodfail -n nginx
POD                      NODE                 CONTAINER_FAIL_MESSAGE
nginx-748c667d99-g8pq9   k8s-sample-worker3   <none>
nginx-748c667d99-h67cc   k8s-sample-worker2   <none>
nginx-748c667d99-x7z6k   k8s-sample-worker    <none>
nginx-7bbc4565cc-szhpf   k8s-sample-worker3   failed to create containerd task: failed to create shim task: OCI runtime create failed: runc create failed: unable to start container process: exec: "bla": executable file not found in $PATH: unknown                   <none>

Kubectl has an option to filter rows using “field-selector”. But unfortunately it does not have capability for filtering nested loops or array objects. But we can use it to filter flat fields.

--field-selector="status.phase==Running"

2. I want to see both pod and container status. Sometimes pod will be unhealthy even though container status is ready.

➜ kubectl get pods -o custom-columns='NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status,CONTAINERS_READY:.status.containerStatuses[*].ready' -n kube-s
ystem
NAME                                               READY   CONTAINERS_READY
coredns-787d4945fb-2vfp9                           True    true
coredns-787d4945fb-x8gv6                           True    true
etcd-k8s-sample-control-plane                      True    true
kindnet-2s78p                                      True    true
kindnet-48tzd                                      False   true
kindnet-ddp6j                                      True    true
kindnet-hl4wp                                      True    true
kube-apiserver-k8s-sample-control-plane            True    true
kube-controller-manager-k8s-sample-control-plane   True    true
kube-proxy-gldxd                                   True    true
kube-proxy-k89fg                                   True    true
kube-proxy-kh5g7                                   True    true
kube-proxy-zqfz5                                   False   true
kube-scheduler-k8s-sample-control-plane            True    true

I saw some pods which was showing as not ready as above, but container was in ready state. Could be due to issues when the kind k8s setup I am using had some resizing and pods went into bad state.

3. Get the CPU and memory resource limits of pods

➜ kubectl get pods -o custom-columns="POD:.metadata.name,\
CPU_REQUEST:.spec.containers[*].resources.requests.cpu,\
MEM_REQUEST:.spec.containers[*].resources.requets.memory,\
CPU_LIMIT:.spec.containers[*].resources.limits.cpu,\
MEM_LIMIT:.spec.containers[*].resources.limits.memory"

Lets create an alias and use it as below

➜ getpodresource='kubectl get pods -o custom-columns="POD:.metadata.name,CPU_REQUEST:.spec.containers[*].resources.requests.cpu,MEM_REQUEST:.spec.containers[*].resources.requets.memory"'
➜ getpodresource -n kube-system
POD                                                CPU_REQUEST   MEM_REQUEST
coredns-787d4945fb-2vfp9                           100m          <none>
coredns-787d4945fb-x8gv6                           100m          <none>
etcd-k8s-sample-control-plane                      100m          <none>
kindnet-2s78p                                      100m          <none>
kindnet-48tzd                                      100m          <none>
kindnet-ddp6j                                      100m          <none>
kindnet-hl4wp                                      100m          <none>
kube-apiserver-k8s-sample-control-plane            250m          <none>
kube-controller-manager-k8s-sample-control-plane   200m          <none>
kube-proxy-gldxd                                   <none>        <none>
kube-proxy-k89fg                                   <none>        <none>
kube-proxy-kh5g7                                   <none>        <none>
kube-proxy-zqfz5                                   <none>        <none>
kube-scheduler-k8s-sample-control-plane            100m          <none>

4. Display ingress resources with hostname

➜ kubectl get ingress -o custom-columns="INGRESS:.metadata.name,HOST:.spec.rules[*].host" -n kube-system

5. Get me the list of hostpaths mounted by pods in worker nodes

➜ kubectl get pods -o custom-columns='NAMESPACE:.metadata.namespace,PODNAME:.metadata.name,HOSTPATHS:.spec.volumes[*].hostPath.path' -n kube-system
NAMESPACE     PODNAME                                            HOSTPATHS
kube-system   coredns-787d4945fb-2vfp9                           <none>
kube-system   coredns-787d4945fb-x8gv6                           <none>
kube-system   etcd-k8s-sample-control-plane                      /etc/kubernetes/pki/etcd,/var/lib/etcd
kube-system   kindnet-2s78p                                      /etc/cni/net.d,/run/xtables.lock,/lib/modules
kube-system   kindnet-48tzd                                      /etc/cni/net.d,/run/xtables.lock,/lib/modules
kube-system   kindnet-ddp6j                                      /etc/cni/net.d,/run/xtables.lock,/lib/modules
kube-system   kindnet-hl4wp                                      /etc/cni/net.d,/run/xtables.lock,/lib/modules
kube-system   kube-apiserver-k8s-sample-control-plane            /etc/ssl/certs,/etc/ca-certificates,/etc/kubernetes/pki,/usr/local/share/ca-certificates,/usr/share/ca-certificates
kube-system   kube-controller-manager-k8s-sample-control-plane   /etc/ssl/certs,/etc/ca-certificates,/usr/libexec/kubernetes/kubelet-plugins/volume/exec,/etc/kubernetes/pki,/etc/kubernetes/controller-manager.conf,/usr/local/share/ca-certificates,/usr/share/ca-certificates
kube-system   kube-proxy-gldxd                                   /run/xtables.lock,/lib/modules
kube-system   kube-proxy-k89fg                                   /run/xtables.lock,/lib/modules
kube-system   kube-proxy-kh5g7                                   /run/xtables.lock,/lib/modules
kube-system   kube-proxy-zqfz5                                   /run/xtables.lock,/lib/modules
kube-system   kube-scheduler-k8s-sample-control-plane            /etc/kubernetes/scheduler.conf

This can be useful to quickly get the list of mounted hostpaths. As a security best practice, only allow permitted hostpaths through a security policy.

➜  alias hostpaths="kubectl get pods -o custom-columns='NAMESPACE:.metadata.namespace,PODNAME:.metadata.name,HOSTPATHS:.spec.volumes[*].hostPath.path'"
➜  gethostpaths -n kube-system

Summary

I believe these shortcuts will be helpful when operating daily on k8s clusters. To summarize these are the aliases we discussed

➜ alias getpodfail='kubectl get pods -o custom-columns="POD:.metadata.name,NODE:.spec.nodeName,CONTAINER_FAIL_MESSAGE:.status.containerStatuses[*].lastState.terminated.message"'

➜ alias getpodcontainerstatus="kubectl get pods -o custom-columns='NAME:.metadata.name,READY:.status.conditions[?(@.type=="Ready")].status,CONTAINERS_READY:.status.containerStatuses[*].ready'"

➜ alias getingress="kubectl get ingress -o custom-columns="INGRESS:.metadata.name,HOST:.spec.rules[*].host" -n kube-system"

➜ alias gethostpaths="kubectl get pods -o custom-columns='NAMESPACE:.metadata.namespace,PODNAME:.metadata.name,HOSTPATHS:.spec.volumes[*].hostPath.path'"

➜ alias getpodresource='kubectl get pods -o custom-columns="POD:.metadata.name,CPU_REQUEST:.spec.containers[*].resources.requests.cpu,MEM_REQUEST:.spec.containers[*].resources.requets.memory"'

I hope this article was helpful. If you liked my article, do like, follow and share.
I am publishing future articles here — https://devopsforyou.com/