k8s 09: PKI infrastructure and Certificate

To have API server to accept requests via HTTPS, PKI needs to be setup.

In this demo cluster, I use controller-1 node to create CA and other certificate.

Set up CA

1. Install tools

Official document reference is here. Because of a bug in 1.2.0 of cfssl, it needs some workaround though.

[ controller-1 ]

# install go - you need to install "go" as pre-requisite of cfssl
# https://www.linode.com/docs/development/go/install-go-on-ubuntu/

# install cfssl
# https://github.com/cloudflare/cfssl
# cfssl is the main tool to digest the CSR, but the output is usually json format.
# cfssljson digest that json file and output certificate and key.
root@controller-1:~# go get -u github.com/cloudflare/cfssl/cmd/cfssl
root@controller-1:~# go get -u github.com/cloudflare/cfssl/cmd/cfssljson

2. Initialize CA

Once tools are installed, it’s time to initialize CA. It is quite straightforward.

[ controller-1 ]

root@controller-1:~/cert# cat << EOF > ca-config.json
> {
>   "signing": {
>     "default": {
>       "expiry": "8760h"
>     },
>     "profiles": {
>       "kubernetes": {
>         "usages": [
>           "signing",
>           "key encipherment",
>           "server auth",
>           "client auth"
>         ],
>         "expiry": "8760h"
>       }
>     }
>   }
> }
> EOF
root@controller-1:~/cert# cat << EOF > ca-csr.json
> {
>   "CN": "kubernetes",
>   "key": {
>     "algo": "rsa",
>     "size": 2048
>   },
>   "names":[{
>     "C": "UK",
>     "L": "London",
>     "O": "Kubernetes",
>     "OU": "CA"
>   }]
> }
> EOF
root@controller-1:~/cert# cfssl gencert -initca ca-csr.json | cfssljson -bare ca
2018/10/13 22:04:21 [INFO] generating a new CA key and certificate from CSR
2018/10/13 22:04:21 [INFO] generate received request
2018/10/13 22:04:21 [INFO] received CSR
2018/10/13 22:04:21 [INFO] generating key: rsa-2048
2018/10/13 22:04:21 [INFO] encoded CSR
2018/10/13 22:04:21 [INFO] signed certificate with serial number

Generate certificate and key for respective services

In the next few steps, we generate a few certificates and keys in order each services to work correctly. Because we’re going to use RBAC authorisation, CN needs to be specifically set for in-built values(default roles) for those certificates of client. You can find those roles and details of RBAC here.

1. API server

This step generates server certificate for API server.

root@controller-1:~/cert# cat << EOF >> apiserver-csr.json
> {
>   "CN": "kubernetes",
>   "hosts": [
>     "127.0.0.1",
>     "10.240.0.11",
>     "10.32.0.1",
>     "x.x.x.x(apiserver-public-ip)",
>     "kubernetes",
>     "kubernetes.default",
>     "kubernetes.default.svc",
>     "kubernetes.default.svc.cluster",
>     "kubernetes.default.svc.cluster.local"
>   ],
>   "key": {
>     "algo": "rsa",
>     "size": 2048
>   },
>   "names": [{
>     "C": "UK"
>     "L": "London",
>     "O": "kubernetes",
>     "OU": "k8s Demo"
>   }]
> }
> EOF
root@controller-1:~/cert# cfssl gencert -ca=ca.pem -ca-key=ca-key.pem \
> --config=ca-config.json -profile=kubernetes \
> apiserver-csr.json | cfssljson -bare apiserver
2018/10/13 22:24:08 [INFO] generate received request
2018/10/13 22:24:08 [INFO] received CSR
2018/10/13 22:24:08 [INFO] generating key: rsa-2048
2018/10/13 22:24:08 [INFO] encoded CSR
2018/10/13 22:24:08 [INFO] signed certificate with serial number

2. Kubectl

This step generates client certificate and key for kubectl

root@controller-1:~/cert# cat << EOF >> kubectl-csr.json
> {
>   "CN": "admin",
>   "key": {
>     "algo": "rsa",
>     "size": 2048
>   },
>   "names": [
>     {
>       "C": "UK",
>       "L": "London",
>       "O": "system:masters",
>       "OU": "k8s Demo"
>     }
>   ]
> }
> EOF

root@controller-1:~/cert# cfssl gencert \
>   -ca=ca.pem \
>   -ca-key=ca-key.pem \
>   -config=ca-config.json \
>   -profile=kubernetes \
>   kubectl-csr.json | cfssljson -bare kubectl
2018/10/13 22:31:40 [INFO] generate received request
2018/10/13 22:31:40 [INFO] received CSR
2018/10/13 22:31:40 [INFO] generating key: rsa-2048
2018/10/13 22:31:40 [INFO] encoded CSR
2018/10/13 22:31:40 [INFO] signed certificate with serial number

3. Kubelet

Kubelet requires special authorization mode “Node authorizer”. And it needs to specify group as well as user(in this case nodename). Hence, the certificate is different from node to node, and need to be generated separately. For detail of node authorization, refer here.

root@controller-1:~/cert# for num in 1 2; do
> cat << EOF > worker-${num}-kubelet-csr.json
> {
>   "CN": "system:node:worker-${num}",
>   "hosts": [
>     "worker-${num}",
>     "10.240.0.2${num}"
>   ],
>   "key": {
>     "algo": "rsa",
>     "size": 2048
>   },
>   "names": [
>     {
>       "C": "UK",
>       "L": "London",
>       "O": "system:nodes",
>       "OU": "k8s Demo"
>     }
>   ]
> }
> EOF
> done

root@controller-1:~/cert# for num in 1 2; do
> cfssl gencert \
>   -ca=ca.pem \
>   -ca-key=ca-key.pem \
>   -config=ca-config.json \
>   -profile=kubernetes \
>   worker-${num}-kubelet-csr.json | cfssljson -bare worker-${num}-kubelet
> done
2018/10/13 22:54:08 [INFO] generate received request
2018/10/13 22:54:08 [INFO] received CSR
2018/10/13 22:54:08 [INFO] generating key: rsa-2048
2018/10/13 22:54:08 [INFO] encoded CSR
2018/10/13 22:54:08 [INFO] signed certificate with serial number 
2018/10/13 22:54:08 [INFO] generate received request
2018/10/13 22:54:08 [INFO] received CSR
2018/10/13 22:54:08 [INFO] generating key: rsa-2048
2018/10/13 22:54:09 [INFO] encoded CSR
2018/10/13 22:54:09 [INFO] signed certificate with serial number

4. kube-proxy

This step generates client certificate and key for kube-proxy

root@controller-1:~/cert# cat << EOF > kube-proxy-csr.json
> {
>   "CN": "system:kube-proxy",
>   "key": {
>     "algo": "rsa",
>     "size": 2048
>   },
>   "names": [
>     {
>       "C": "UK",
>       "L": "London",
>       "O": "system:node-proxier",
>       "OU": "k8s Demo"
>     }
>   ]
> }
> EOF
root@controller-1:~/cert# cfssl gencert \
>   -ca=ca.pem \
>   -ca-key=ca-key.pem \
>   -config=ca-config.json \
>   -profile=kubernetes \
>   kube-proxy-csr.json | cfssljson -bare kube-proxy
2018/10/13 22:59:31 [INFO] generate received request
2018/10/13 22:59:31 [INFO] received CSR
2018/10/13 22:59:31 [INFO] generating key: rsa-2048
2018/10/13 22:59:32 [INFO] encoded CSR
2018/10/13 22:59:32 [INFO] signed certificate with serial number

5. coredns

This step generates client certificate and key for coredns

root@controller-1:~/cert# cat << EOF > coredns-csr.json
> {
>   "CN": "system:kube-dns",
>   "key": {
>     "algo": "rsa",
>     "size": 2048
>   },
>   "names": [
>     {
>       "C": "UK",
>       "L": "London",
>       "O": "system:kube-dns",
>       "OU": "k8s Demo"
>     }
>   ]
> }
> EOF
root@controller-1:~/cert# cfssl gencert \
>   -ca=ca.pem \
>   -ca-key=ca-key.pem \
>   -config=ca-config.json \
>   -profile=kubernetes \
>   coredns-csr.json | cfssljson -bare coredns
2018/10/13 23:04:50 [INFO] generate received request
2018/10/13 23:04:50 [INFO] received CSR
2018/10/13 23:04:50 [INFO] generating key: rsa-2048
2018/10/13 23:04:51 [INFO] encoded CSR
2018/10/13 23:04:51 [INFO] signed certificate with serial number