k8s ex02: Security – Overall view

We’re concentrating on kubernetes function so far, and have not taken much consideration on the security. I’m going to introduce security features into our kubernetes cluster in the following posts.

Existing setup

There is no authentication/authorization deployed in APIserver , hence any request can be processed(e.g. any kubectl request, any node join request) as long it knows the ip address of APIserver.

What’s worse, those communication is not encrypted and it can be easily sniffed on the wire.

# suppose you use kubectl as follows in the local machine
shogokobayashi ~ $ kubectl get nodes
NAME       STATUS     ROLES     AGE       VERSION
worker-1   NotReady   <none>    17d       v1.11.3
worker-2   NotReady   <none>    13d       v1.11.3

# the request itself and the reply is all not encrypted.
k_shogo@controller-1:~$ sudo tcpdump -i ens4 -n dst port 8080 -vvv -X
  07:54:49.734604 IP (tos 0x0, ttl 41, id 0, offset 0, flags [DF], proto TCP (6), length 281)
      x.x.x.x.33413 > 10.240.0.11.8080: Flags [P.], cksum 0x10cf (correct), seq 0:229, ack 1, win 4136, options [nop,nop,TS val 726205998 ecr 2105429232], length 229: HTTP, length: 229
  	GET /api/v1/nodes?limit=500 HTTP/1.1
  	Host: x.x.x.x:8080
  	User-Agent: kubectl/v1.11.3 (darwin/amd64) kubernetes/a452946
  	Accept: application/json;as=Table;v=v1beta1;g=meta.k8s.io, application/json
  	Accept-Encoding: gzip
  	
  	0x0000:  4500 0119 0000 4000 2906 10e1 5862 dca1  E.....@.)...Xb..
  	0x0010:  0af0 000b 8285 1f90 75b1 2479 7335 0ce8  ........u.$ys5..
  	0x0020:  8018 1028 10cf 0000 0101 080a 2b49 062e  ...(........+I..
  	0x0030:  7d7e 4cf0 4745 5420 2f61 7069 2f76 312f  }~L.GET./api/v1/
  	0x0040:  6e6f 6465 733f 6c69 6d69 743d 3530 3020  nodes?limit=500.
  	0x0050:  4854 5450 2f31 2e31 0d0a 486f 7374 3a20  HTTP/1.1..Host:.
  	0x0060:  cccc cccc cccc cccc cccc cccc cccc cccc  x.x.x.x:80
  	0x0070:  3830 0d0a 5573 6572 2d41 6765 6e74 3a20  80..User-Agent:.
  	0x0080:  6b75 6265 6374 6c2f 7631 2e31 312e 3320  kubectl/v1.11.3.
  	0x0090:  2864 6172 7769 6e2f 616d 6436 3429 206b  (darwin/amd64).k
  	0x00a0:  7562 6572 6e65 7465 732f 6134 3532 3934  ubernetes/a45294
  	0x00b0:  360d 0a41 6363 6570 743a 2061 7070 6c69  6..Accept:.appli
  	0x00c0:  6361 7469 6f6e 2f6a 736f 6e3b 6173 3d54  cation/json;as=T
  	0x00d0:  6162 6c65 3b76 3d76 3162 6574 6131 3b67  able;v=v1beta1;g
  	0x00e0:  3d6d 6574 612e 6b38 732e 696f 2c20 6170  =meta.k8s.io,.ap
  	0x00f0:  706c 6963 6174 696f 6e2f 6a73 6f6e 0d0a  plication/json..
  	0x0100:  4163 6365 7074 2d45 6e63 6f64 696e 673a  Accept-Encoding:
  	0x0110:  2067 7a69 700d 0a0d 0a                   .gzip....

shogokobayashi ~ $ tcpdump -i en0 -n src net x.x.x.x/32 -vvv -X
08:54:49.730678 IP (tos 0x0, ttl 66, id 62756, offset 0, flags [DF], proto TCP (6), length 727)
    x.x.x.x.8080 > 192.168.1.107.53856: Flags [P.], cksum 0x4774 (correct), seq 2817:3492, ack 230, win 229, options [nop,nop,TS val 2105429377 ecr 726205998], length 675: HTTP
	0x0000:  4500 02d7 f524 4000 4206 c7e4 23cf 9335  E....$@.B...#..5
	0x0010:  c0a8 016b 1f90 d260 7335 17e8 75b1 255e  ...k...`s5..u.%^
	0x0020:  8018 00e5 4774 0000 0101 080a 7d7e 4d81  ....Gt......}~M.
	0x0030:  2b49 062e 6d61 6e61 6765 642d 6174 7461  +I..managed-atta
	0x0040:  6368 2d64 6574 6163 6822 3a22 7472 7565  ch-detach":"true
	0x0050:  227d 7d7d 7d2c 7b22 6365 6c6c 7322 3a5b  "}}}},{"cells":[
	0x0060:  2277 6f72 6b65 722d 3222 2c22 4e6f 7452  "worker-2","NotR
	0x0070:  6561 6479 222c 225c 7530 3033 636e 6f6e  eady","\u003cnon
	0x0080:  655c 7530 3033 6522 2c22 3133 6422 2c22  e\u003e","13d","
	0x0090:  7631 2e31 312e 3322 2c22 5c75 3030 3363  v1.11.3","\u003c
	0x00a0:  6e6f 6e65 5c75 3030 3365 222c 2255 6275  none\u003e","Ubu
	0x00b0:  6e74 7520 3136 2e30 342e 3520 4c54 5322  ntu.16.04.5.LTS"
	0x00c0:  2c22 342e 3135 2e30 2d31 3032 312d 6763  ,"4.15.0-1021-gc
	0x00d0:  7022 2c22 646f 636b 6572 3a2f 2f31 382e  p","docker://18.
	0x00e0:  362e 3122 5d2c 226f 626a 6563 7422 3a7b  6.1"],"object":{
	0x00f0:  226b 696e 6422 3a22 5061 7274 6961 6c4f  "kind":"PartialO
	0x0100:  626a 6563 744d 6574 6164 6174 6122 2c22  bjectMetadata","
	0x0110:  6170 6956 6572 7369 6f6e 223a 226d 6574  apiVersion":"met
	0x0120:  612e 6b38 732e 696f 2f76 3162 6574 6131  a.k8s.io/v1beta1
	0x0130:  222c 226d 6574 6164 6174 6122 3a7b 226e  ","metadata":{"n
	0x0140:  616d 6522 3a22 776f 726b 6572 2d32 222c  ame":"worker-2",
	0x0150:  2273 656c 664c 696e 6b22 3a22 2f61 7069  "selfLink":"/api
	0x0160:  2f76 312f 6e6f 6465 732f 776f 726b 6572  /v1/nodes/worker
	0x0170:  2d32 222c 2275 6964 223a 2232 6138 3132  -2","uid":"2a812
	0x0180:  3563 362d 6334 3362 2d31 3165 382d 6137  5c6-c43b-11e8-a7
	0x0190:  3633 2d34 3230 3130 6166 3030 3030 6222  63-42010af0000b"
	0x01a0:  2c22 7265 736f 7572 6365 5665 7273 696f  ,"resourceVersio
	0x01b0:  6e22 3a22 3134 3039 3034 222c 2263 7265  n":"140904","cre
	0x01c0:  6174 696f 6e54 696d 6573 7461 6d70 223a  ationTimestamp":
	0x01d0:  2232 3031 382d 3039 2d32 3954 3232 3a35  "2018-09-29T22:5
	0x01e0:  383a 3233 5a22 2c22 6c61 6265 6c73 223a  8:23Z","labels":
	0x01f0:  7b22 6265 7461 2e6b 7562 6572 6e65 7465  {"beta.kubernete
	0x0200:  732e 696f 2f61 7263 6822 3a22 616d 6436  s.io/arch":"amd6
	0x0210:  3422 2c22 6265 7461 2e6b 7562 6572 6e65  4","beta.kuberne
	0x0220:  7465 732e 696f 2f6f 7322 3a22 6c69 6e75  tes.io/os":"linu
	0x0230:  7822 2c22 6b75 6265 726e 6574 6573 2e69  x","kubernetes.i
	0x0240:  6f2f 686f 7374 6e61 6d65 223a 2277 6f72  o/hostname":"wor
	0x0250:  6b65 722d 3222 7d2c 2261 6e6e 6f74 6174  ker-2"},"annotat
	0x0260:  696f 6e73 223a 7b22 6e6f 6465 2e61 6c70  ions":{"node.alp
	0x0270:  6861 2e6b 7562 6572 6e65 7465 732e 696f  ha.kubernetes.io
	0x0280:  2f74 746c 223a 2230 222c 2276 6f6c 756d  /ttl":"0","volum
	0x0290:  6573 2e6b 7562 6572 6e65 7465 732e 696f  es.kubernetes.io
	0x02a0:  2f63 6f6e 7472 6f6c 6c65 722d 6d61 6e61  /controller-mana
	0x02b0:  6765 642d 6174 7461 6368 2d64 6574 6163  ged-attach-detac
	0x02c0:  6822 3a22 7472 7565 227d 7d7d 7d5d 7d0a  h":"true"}}}}]}.
	0x02d0:  0d0a 300d 0a0d 0a                        ..0....

As the nature of kubernetes, API server is the hub of all information. So protecting security is all around API server. Let’s see how API server is structured in terms of security.

API server security structure

API server listens the request on two ports, one for HTTP and the other for HTTPS. On HTTP port, authentication and authorisation are skipped. You can find the detail here in the official document.