If you have applications running as containers in Kubernetes clusters, you may come across scenarios where you want to monitor or analyse raw network traffic. This is usually needed if your containers are making outbound API calls and you find random connection issues with your API calls. If you are using public cloud providers such as Azure, there are monitoring tools such as Network watcher which can be used to monitor traffic. However in this article I will show you a quick and easy way to monitor the pod network traffic at the TCP/IP level using pod console only.

We will be using a linux command line utility called tcpdump to capture TCP/IP network traffic. Note that API calls using http protocol actually runs on top of TCP/IP. Hence tcpdump provides a more precise view of networking issues at the TCP/IP level.

Step 1: Identify the pod name using the following command in you machine's command line. We will use the pod name to connect to the running pod. The following command assumes you already have kubectl client installed in your machine for accessing kubernetes clusters,

Step 2: Use the pod name returned by the above command to connect and get a direct command line access to the running pod's operating system. You need to replace POD_NAME with the name of the pod returned above,

Step 3: Once you are connected to the pod, run tcpdump --version in the pod to check whether you have the tcpdump command available in your running pod. If not, run one of the following commands to install tcpdump to your running container. Please note that if your container is restarted any time, you will have to run the command again to get tcpdump installed.

If Debian based linux distributions is used in the pod, run follwing command to install tcpdump,

If Alpine linux or busybox distributions is used in the pod, run following command to install tcpdump,

Step 4: Now you are all set to capture network packets from the pod. Run the following command on the pod command line to capture network traffic to a file named networkcapture.cap. The -s option with 0 ensures capture of large network packets, -vvv option ensures detailed capture and -w specifies the file name for capture.

The above command captures detailed logs for all TCP/IP transactions. If you want to limit the traffic to a destination IP or host name (when you are troubleshooting API call issues), you can run the following command,

Step 5: When you want to stop the capture, you can press Ctrl-C to break out of tcpdump. Now you have networkcapture.cap available in your pod. Using pwd command find the location of the file and note it down. Use this value to replace REMOTE_POD_PATH_INCLUDING_FILE_NAME in the following command.

Step 6: Exit from the pod console using exit command. Run the following command from your machine to download the networkcapture.cap file from pod to your local folder.

Here is how a sample command looks like,

Step 7: You can perform a detailed analysis of the tcpdump capture using the wireshark tool. Download it from here.