Categories
DevOps networking

Kubernetes Networking Explained

Kubernetes networking might seem like a service we get for free from Kubernetes. Yet, it’s important to understand the principles and ideas behind it. Keep reading to find them out.

Table Of Contents
  1. Kubernetes networking main design decisions
  2. Kubernetes networks
  3. Kubernetes pods communications
  4. CNI
  5. Kubernetes networking demo
  6. Summary

Kubernetes networking main design decisions

Firstly, let’s state important design decisions Kubernetes developers took:

  • pods may reach pods on any node using a real ip address, there’s no NAT.
  • node-bound Kubernetes components like kube-proxy and kubelet can reach all pods on a node.
  • pods and services may reach each other using DNS name

Kubernetes networks

There are different Kubernetes networks which implement the above design:

  • node network – network of physical or virtual machines spanning Kubernetes cluster.
  • pod network – pod ips may be taken from the node network. Yet, usually, the ips are takend from a pool of ips – CIDR range.
  • cluster network – ips for services of type clusterIP are assigned in this network.

Kubernetes pods communications

When we explored Jenkins Docker in Docker Agent, we saw an example of Kubernetes pod that runs more than one container. How did they communicate? Right, using localhost.

What about pod to pod communication? It Depends. If Pods run on the same node, they will reach each other using their ip addresses with the help of some network interface. The interface might be a software bridge or tunnel. It depends on the chosen network model.

Whereas, if pods run on different nodes, nodes network must provide connectivity between pods using their ips. This is provided either using by L2/L3 or an overlay network. Overlay network gives an illusion that nodes are connected using a single network regardless of underlying network infra.

CNI

CNI is Container Network Interface. It defines container networking for multiple container orchestrators e.g. Kubernetes. Kubernetes decided that its admins have to choose CNI plugin (e.g. Calico) which implements Kubernetes networking according to their needs. Such network plugin bridges between Kubernetes and container runtime (e.g. docker) along with Kubernetes node OS for the goal of managing networking resources (e.g. ip addressing)

kubelet node local network configuration defines network plugin. It may either be CNI or Kubenet. Let’s now see a demo on Kubernetes networking.

Kubernetes networking demo

Demo Prerequisites

I assume you have Kubernetes cluster. If you don’t, install on your machine minikube and kubectl.

Start minikube Kubernetes multi-node cluster using command: minikube start --nodes 3 -p multinode-demo.

Kubernetes CNI demo

 kubectl get nodes -o wide
NAME         STATUS   ROLES           AGE     VERSION   INTERNAL-IP     EXTERNAL-IP   OS-IMAGE             KERNEL-VERSION       CONTAINER-RUNTIME
munode       Ready    control-plane   9m55s   v1.25.2   192.168.103.2   <none>        Ubuntu 20.04.5 LTS   4.15.0-193-generic   docker://20.10.18
munode-m02   Ready    <none>          9m12s   v1.25.2   192.168.103.3   <none>        Ubuntu 20.04.5 LTS   4.15.0-193-generic   docker://20.10.18
munode-m03   Ready    <none>          8m48s   v1.25.2   192.168.103.4   <none>        Ubuntu 20.04.5 LTS   4.15.0-193-generic   docker://20.10.18

Let’s deploy sample Kubernetes deployment to the cluster. It’ll be the same one we used in this demo.

kubectl create deploy nginx --image=rancher/hello-world --replicas=3

You see each pod, its ip and node it’s running on:

kubectl get pods -o wide
NAME                     READY   STATUS    RESTARTS   AGE   IP           NODE         NOMINATED NODE   READINESS GATES
nginx-5844775f89-4nnwf   1/1     Running   0          14s   10.244.0.3   munode       <none>           <none>
nginx-5844775f89-9ptln   1/1     Running   0          14s   10.244.2.2   munode-m03   <none>           <none>
nginx-5844775f89-r6tkk   1/1     Running   0          14s   10.244.1.2   munode-m02   <none>           <none>

You wonder how traffic from a pod running on node munode will reach other pod running on node munode-m02.

There’s a great explanation about kindnet network plugin used by minikube here.

Pay attention that if you want to inspect node networking configuration you can run the commands in below privileged shell:

kubectl debug node/[node-name] -it --image=mcr.microsoft.com/aks/fundamental/base-ubuntu:v0.0.11

Now you can run ip addr, route, ip route and ip route get [some_pod_ip] commands. This is based on this trick. Of course, you can also connect to the node using minikube ssh. But then, you may have to install some inspection binaries.

Summary

That’s it about Kubernetes networking. As always, feel free to share.

Recommended Kubernetes courses on Pluralsight:

Sign up using this link to get exclusive discounts like 50% off your first month or 15% off an annual subscription)

Recommended Kubernetes books on Amazon: