k8s ex01: iptables and docker

In this entry, I’ll dig into some basic network principals of iptables, which is used as the base for kube-proxy service.

Anyone used linux should have used iptables in some ways, but it is confusing when it comes to containers. I will illustrate the basic flow of the packet step by step, though it’s not 100% correct diagram but I hope it helps someone to grasp how iptables works.

And this is what I’m going to explain…The docker container which runs nginx is waiting to serve web page on port 80, while the client request the page on port 8080. Still the client can get the web page, because iptables rewrite the ip packet accordingly.

Request from the client to the container

In the image below, I highlighted the table which docker modified specifically for this port forwarding only. Usually there is a catch-all rule to either forward or drop the packet in each table.

 

Response from the container to the client

 

# run nginx image, and map node port 8080 to container port 80
shogo@worker-1:~$ sudo docker container run -d -p 8080:80 nginx
e555f0f10d57e41b18d649f9f75c4bb04b3248413093c095b5294f93c7529a0e
shogo@worker-1:~$ sudo docker ps
CONTAINER ID        IMAGE               COMMAND                  CREATED             STATUS              PORTS                  NAMES
e555f0f10d57        nginx               "nginx -g 'daemon of…"   9 seconds ago       Up 8 seconds        0.0.0.0:8080->80/tcp   infallible_feynman

# check nat table
shogo@worker-1:~$ sudo iptables -nvL -t nat
Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    5   296 DOCKER     all  --  *      *       0.0.0.0/0            0.0.0.0/0            ADDRTYPE match dst-type LOCAL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
Chain OUTPUT (policy ACCEPT 5 packets, 300 bytes)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DOCKER     all  --  *      *       0.0.0.0/0           !127.0.0.0/8          ADDRTYPE match dst-type LOCAL
Chain POSTROUTING (policy ACCEPT 5 packets, 300 bytes)
 pkts bytes target     prot opt in     out     source               destination         
  601 36762 KUBE-POSTROUTING  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes postrouting rules */
    0     0 MASQUERADE  all  --  *      !docker0  172.17.0.0/16        0.0.0.0/0           
    0     0 MASQUERADE  tcp  --  *      *       172.17.0.2           172.17.0.2           tcp dpt:80
Chain DOCKER (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 RETURN     all  --  docker0 *       0.0.0.0/0            0.0.0.0/0           
    0     0 DNAT       tcp  --  !docker0 *       0.0.0.0/0            0.0.0.0/0            tcp dpt:8080 to:172.17.0.2:80
Chain KUBE-MARK-DROP (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x8000
Chain KUBE-MARK-MASQ (0 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MARK       all  --  *      *       0.0.0.0/0            0.0.0.0/0            MARK or 0x4000
Chain KUBE-POSTROUTING (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 MASQUERADE  all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes service traffic requiring SNAT */ mark match 0x4000/0x4000

# check filter table
shogo@worker-1:~$ sudo iptables -nvL
Chain INPUT (policy ACCEPT 133 packets, 10396 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 6587  104M KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
 6712  104M sshguard   all  --  *      *       0.0.0.0/0            0.0.0.0/0           
Chain FORWARD (policy DROP 0 packets, 0 bytes)
 pkts bytes target     prot opt in     out     source               destination         
   47 22230 DOCKER-USER  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
   47 22230 DOCKER-ISOLATION-STAGE-1  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
   21  2226 ACCEPT     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0            ctstate RELATED,ESTABLISHED
    3   176 DOCKER     all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
   23 19828 ACCEPT     all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
    0     0 ACCEPT     all  --  docker0 docker0  0.0.0.0/0            0.0.0.0/0           
Chain OUTPUT (policy ACCEPT 111 packets, 18192 bytes)
 pkts bytes target     prot opt in     out     source               destination         
 6950  592K KUBE-FIREWALL  all  --  *      *       0.0.0.0/0            0.0.0.0/0           
Chain DOCKER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 ACCEPT     tcp  --  !docker0 docker0  0.0.0.0/0            172.17.0.2           tcp dpt:80
Chain DOCKER-ISOLATION-STAGE-1 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   23 19828 DOCKER-ISOLATION-STAGE-2  all  --  docker0 !docker0  0.0.0.0/0            0.0.0.0/0           
   47 22230 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
Chain DOCKER-ISOLATION-STAGE-2 (1 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      docker0  0.0.0.0/0            0.0.0.0/0           
   23 19828 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           
Chain DOCKER-USER (1 references)
 pkts bytes target     prot opt in     out     source               destination         
   47 22230 RETURN     all  --  *      *       0.0.0.0/0            0.0.0.0/0           

Chain KUBE-FIREWALL (2 references)
 pkts bytes target     prot opt in     out     source               destination         
    0     0 DROP       all  --  *      *       0.0.0.0/0            0.0.0.0/0            /* kubernetes firewall for dropping marked packets */ mark match 0x8000/0x8000
Chain sshguard (1 references)
 pkts bytes target     prot opt in     out     source               destination