Kubernetes - 2022 sügis

Allikas: Imre kasutab arvutit
Mine navigeerimisribaleMine otsikasti

Sissejuhatus

TODO

Rancher Management Cluster on Docker

Käivitamine 

docker run -d --restart=unless-stopped \
  -p 80:80 -p 443:443 \
  --privileged \
  rancher/rancher:latest

kus

  • TODO

Jälgida käivitumist, kulub mitu minutit 

root@tf-vm-1:~# docker logs -f practical_mcnulty

Seejärel pöörduda webgui haldusliidese poole, https://192.168.110.11/

Kasulikud lisamaterjalid

Node tekitamine - proxmoxve node driver abil

Node driver - https://github.com/lnxbil/docker-machine-driver-proxmox-ve/releases/download/v4/docker-machine-driver-proxmoxve.linux-amd64

https://github.com/rancher/os/releases aadressilt saab kopeerida

https://github.com/rancher/os/releases/download/v1.5.8/rancheros-proxmoxve-autoformat.iso

Template, defaultist erinevad

  • debugDriver - linnutada
  • debugResty - linnutada
  • provisionStrategy - cdrom
  • proxmoxHost - 192.168.110.250
  • proxmoxNode - pm-kns
  • proxmoxPool - vaiki
  • proxmoxRealm - pam
  • proxmoxUserName - root
  • proxmoxUserPassword - parool
  • sshPassword - tühi (tundub, et toimib rancheos default)
  • sshPort - 22
  • sshUsername - tühi (tundub, et toimib rancheos default)
  • vmCloneFull - 2
  • vmCloneVmid - tühi
  • vmCpu -
  • vmCpuCores - 1
  • vmCpuSockets - 4
  • vmImageFile - local:iso/rancheros-proxmoxve-autoformat.iso
  • vmMemory - 4
  • vmNetBridge - vmbr0
  • vmNetFirewall - 0
  • vmNetModel - virtio
  • vmNetTag - 0
  • vmScsiController - virtio-scsi-pci
  • vmStoragePath - sn_data (tundub, et peab olema proxmox Directory tüüpi)
  • vmStorageSize - 32
  • vmStorageType - QCOW2

ja sama json kujul 

{
"annotations": {
"ownerBindingsCreated": "true"
},
"baseType": "nodeTemplate",
"cloudCredentialId": null,
"created": "2022-10-26T22:51:41Z",
"createdTS": 1666824701000,
"creatorId": "user-7rrms",
"driver": "proxmoxve",
"engineEnv": { },
"engineInstallURL": "https://releases.rancher.com/install-docker/20.10.sh",
"engineLabel": { },
"engineOpt": { },
"engineRegistryMirror": [ ],
"id": "cattle-global-nt:nt-d9w8b",
"labels": {
"cattle.io/creator": "norman"
},
"links": {
"nodePools": "…/v3/nodePools?nodeTemplateId=cattle-global-nt%3Ant-d9w8b",
"nodes": "…/v3/nodes?nodeTemplateId=cattle-global-nt%3Ant-d9w8b",
"self": "…/v3/nodeTemplates/cattle-global-nt:nt-d9w8b",
"update": "…/v3/nodeTemplates/cattle-global-nt:nt-d9w8b"
},
"logOpt": { },
"name": "tmplimre",
"principalId": "local://user-7rrms",
"proxmoxveConfig": {
"debugDriver": true,
"debugResty": true,
"provisionStrategy": "cdrom",
"proxmoxHost": "192.168.110.250",
"proxmoxNode": "pm-kns",
"proxmoxPool": "vaiki",
"proxmoxRealm": "pam",
"proxmoxUserName": "root",
"proxmoxUserPassword": "parool",
"sshPassword": "",
"sshPort": "22",
"sshUsername": "",
"vmCienabled": "",
"vmCitype": "",
"vmCloneFull": "2",
"vmCloneVmid": "",
"vmCpu": "",
"vmCpuCores": "1",
"vmCpuSockets": "2",
"vmImageFile": "local:iso/rancheros-proxmoxve-autoformat.iso",
"vmMemory": "2",
"vmNetBridge": "vmbr0",
"vmNetFirewall": "0",
"vmNetModel": "virtio",
"vmNetMtu": "",
"vmNetTag": "0",
"vmNuma": "",
"vmProtection": "",
"vmScsiAttributes": "",
"vmScsiController": "virtio-scsi-pci",
"vmStartOnboot": "",
"vmStoragePath": "sn_data",
"vmStorageSize": "12",
"vmStorageType": "QCOW2",
"vmVmidRange": ""
},
"state": "active",
"storageOpt": { },
"transitioning": "no",
"transitioningMessage": "",
"type": "nodeTemplate",
"useInternalIpAddress": true,
"uuid": "a6c62f53-9316-41bb-8242-2d641988e522"
}

Node tekitamine - custom variatsioon

Rancher webgui peal avada 

Cluster Management -> clusternimi

Ilmunud aknas õpetatakse midagi sellist 

# curl --insecure -fL https://192.168.54.101/system-agent-install.sh | sudo  sh -s - --server https://192.168.54.101 --label 'cattle.io/os=linux' \
  --token lsfjvdnf4q52cmhqnfkxw72vfmk4nwlvdsbbjczlclqjhhrvz9h7q4 --ca-checksum d19d044aac5ca8969aa80079bddb945d4e173a2e8a531d7a83536ffb72a62188 --etcd --controlplane --worker

Tulemusena ilmub uus node Rancher webgui peale välja.

Rancher hallatud klastriga suhtlemine kubectl utiliidiga

Rancher hallatud klastriga töökohaarvutist käsurealt st kubectl utiliidi abil suhtlemiseks tuleb esmalt kopeerida kubectl utiliit 

TODO

Seejärel Rancher webgui liidesest kopeerida klastri seadistus 

TODO

ja klastri poole pöördumiseks öelda 

$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/clubu1.yaml get all --all-namespaces -o wide
NAMESPACE             NAME                                               READY   STATUS      RESTARTS      AGE     IP               NODE     NOMINATED NODE   READINESS GATES
cattle-fleet-system   pod/fleet-agent-bfc5655cc-crbl6                    1/1     Running     0             10m     10.42.0.12       clubu1   <none>           <none>
cattle-system         pod/cattle-cluster-agent-674cc68d59-zgrkq          1/1     Running     1 (11m ago)   14m     10.42.0.5        clubu1   <none>           <none>
cattle-system         pod/cattle-node-agent-n56dp                        1/1     Running     0             14m     192.168.110.13   clubu1   <none>           <none>
cattle-system         pod/helm-operation-7vpbz                           0/2     Completed   0             9m36s   10.42.0.13       clubu1   <none>           <none>

...

MetalLB load balanceri ja NginX ingress kontrolleri kasutamine

Paigaldamine

$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/clc.yaml apply -f \
  https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml

Tulemusena käivitatakse mitmesugused tegevused 

imre@moraal:~/postgres-operator$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/clc.yaml get all -n metallb-system 
NAME                              READY   STATUS    RESTARTS   AGE
pod/controller-6c58495cbb-qnb8h   1/1     Running   0          2d15h
pod/speaker-lpkf9                 1/1     Running   0          2d15h

NAME                      TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
service/webhook-service   ClusterIP   10.43.195.140   <none>        443/TCP   2d15h

NAME                     DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR            AGE
daemonset.apps/speaker   1         1         1       1            1           kubernetes.io/os=linux   2d15h

NAME                         READY   UP-TO-DATE   AVAILABLE   AGE
deployment.apps/controller   1/1     1            1           2d15h

NAME                                    DESIRED   CURRENT   READY   AGE
replicaset.apps/controller-6c58495cbb   1         1         1       2d15h

Tekitada aadresside pool mida metallb kasutab teenuste jaoks 

$ cat ipaddress_pools.yaml 
apiVersion: metallb.io/v1beta1
kind: IPAddressPool
metadata:
  name: production
  namespace: metallb-system
spec:
  addresses:
  - 192.168.110.131-192.168.110.135

---
apiVersion: metallb.io/v1beta1
kind: L2Advertisement
metadata:
  name: l2-advert
  namespace: metallb-system

Tekitamiseks sobib öelda 

$ kubectl apply -f  ~/metallb/ipaddress_pools.yaml
ipaddresspool.metallb.io/production created
l2advertisement.metallb.io/l2-advert created

Tulemust on võimalik vaadelda nt Rancher webgui keskkonnas valides 

More Resources -> metallb.io -> IPAddressPools ja L2Advertisements

Kasulikud lisamaterjalid

NginX ingress kontroller

Paigaldamiseks deploymentina sobib öelda 

$ controller_tag=$(curl -s https://api.github.com/repos/kubernetes/ingress-nginx/releases/latest | grep tag_name | cut -d '"' -f 4)
$ wget -O nginx-ingress-controller-deploy.yaml https://raw.githubusercontent.com/kubernetes/ingress-nginx/${controller_tag}/deploy/static/provider/baremetal/deploy.yaml
imre@moraal:~/metallb$ /home/imre/kubectl --kubeconfig climretest.yaml apply -f nginx-ingress-controller-deploy.yaml

Leida Rancher webgui pealt 

Service: ingress-nginx-controller

ning pressida Edit YAML ning muuta 

spec.type -> LoadBalancer

Kasulikud lisamaterjalid

Deployment tekitamine ja publitseerimine metallb + nginx ingress kontrolleri abil

Paigaldada nt httpd, valida webgui liideses 

Workload -> Deployments

ning paremas paneelis täita

  • Namespace - default
  • Name - dm-httpd
  • Replicas - 1
  • General -> Image - httpd

ning pressida Create, tulemusena peab tekkima muu hulgas Pod. Panna tähele Deployment juures Pod Labels nime ja väärtust

  • Key - workload.user.cattle.io/workloadselector
  • Value - apps.deployment-default-dm-httpd

Tekitada Service svc-httpd, valida webgui liideses 

Service Discovery -> Services -> Create - Cluster IP
  • Port Name - port-httpd
  • Listening Port - 80
  • Protocol - tcp
  • Target Port - 80

Lisaks paremas paneelis ühendada kokku Service ja Deployment valides Selectors ning täita lahtrid

  • Key - workload.user.cattle.io/workloadselector
  • Value - apps.deployment-default-dm-httpd

Tulemusena peab lahtrite kohale ilmuma midagi sellist 

Matches 1 of 60 pods: "dm-httpd-b65cf6875-b7zjq"

Kirjeldada Ingress valides webgui liideses 

Service Discovery -> Ingresses -> Create

ning paremas paneelis täita

  • Namespace - default
  • Name - ing-httpd
  • Rules -> Request Host - httpd-imre-k8s-test.auul.pri.ee
  • Rules -> Path -> Prefix - /
  • Rules -> Path -> Target Service - svc-httpd
  • Ingress Class -> nginx

cert-manager kasutamine

cert-manager abil saab teenuste juures korraldada nt Lets Encrypt sertifikaatide kasutamise. Järneva eelduseks on

  • toimiv metallb + nginx-ingress-controller Rancher + Kubernetes klaster
  • nö tavalisel viisil toimiv deployment, mis on ligipääsetav üle metallb + nginx-ingress-controller'i
  • teenuse dns nimi viitab kõnealusele teenusele tema metallb avaliku ip aadressiga
  • avalikule ip aadressile on avalikust võrgust ligipääs (vastasel korral ei töötaks LE http-challenge)
  • tundub, et sellepärast ei pea muretsema, et LE asus päring LE isanda juurde paistaks samalt ip aadressilt, kus asub teenus ise

ClusterIssuer'ide moodustamine

ClusterIssuer moodustamine LE staging jaoks 

$ cat issuer-staging.yaml 
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    # The ACME server URL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: imre@auul.pri.ee
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class: nginx

ClusterIssuer moodustamine LE prod jaoks 

$ cat issuer-staging.yaml 
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    # The ACME server URL
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    # Email address used for ACME registration
    email: imre@auul.pri.ee
    # Name of a secret used to store the ACME account private key
    privateKeySecretRef:
      name: letsencrypt-staging
    # Enable the HTTP-01 challenge provider
    solvers:
    - http01:
        ingress:
          class: nginx

issueride tekitamiseks sobib öelda 

$ /home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml apply -f issuer-staging.yaml
$ /home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml apply -f issuer-prod.yaml

Tulemuse kontrolliks peab nägema 

imre@moraal:~/tls$ /home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml describe clusterissuer letsencrypt-prod
..

Status:
  Acme:
    Last Registered Email:  imre@auul.pri.ee
    Uri:                    https://acme-v02.api.letsencrypt.org/acme/acct/813299067
  Conditions:
    Last Transition Time:  2022-11-07T17:47:40Z
    Message:               The ACME account was registered with the ACME server
    Observed Generation:   1
    Reason:                ACMEAccountRegistered
    Status:                True
    Type:                  Ready
Events:                    <none>

Rancher webgui liideses peab olema ilus 

More Resources -> Cert Manager -> ClusterIssuers - letsencrypt-prod

Ingress moodustamine

Olemasoleva service jaoks tekitatakse ingress 

$ cat dokuwiki-ing.yaml 
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: dokuwiki
  annotations:
    cert-manager.io/cluster-issuer: "letsencrypt-prod"

spec:
  ingressClassName: nginx
  tls:
  - hosts:
    - dokuwiki-imre-k8s-test.auul.pri.ee
    secretName: dokuwiki-imre-k8s-test-tls
  rules:
  - host: dokuwiki-imre-k8s-test.auul.pri.ee
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: my-release-dokuwiki
            port:
              number: 80

kus

  • metadata.annotations kirjeldab cert-mananger kasutamise letsencrypt-prod issueriga
  • spec.ingressClassName - nginx kirjeldab seose ingress kontrolleriga

Tulemuse kontrollimine

Tulemuse kontrollimiseks peab tekkima Rancher webgui peale vasakus paneelis sektsiooni

  • More Resources -> Cert Manager -> CertificateRequests
  • More Resources -> Cert Manager -> Certificates
  • ...

Sertifikaadi uuesti väljastamise sundimiseks sobib öelda 

/home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml delete secret dokuwiki-imre-k8s-test-tls

Lisaks on näha klastri webgui avalehel Events sektsioonis kuidas sertifikaadid moodustuvad. 

imre@moraal:~/tls$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/climretest.yaml describe certificate dokuwiki-imre-k8s-test-tls

Brauseris peab ilusti avanema https://dokuwiki-imre-k8s-test.auul.pri.ee/

Kasulikud lisamaterjalid

helm kasutamine

Väited

  • helm on kubernetes paketihalduse vahend

helm paigaldamine

TODO

helm kasutamine

helm repo lisamine süsteemi 

imre@moraal:~/tls$ /home/imre/helm --kubeconfig /home/imre/metallb/climretest.yaml repo add bitnami https://charts.bitnami.com/bitnami

Süsteemis olemasolevate helm repode nimekirja esitamine 

imre@moraal:~/tls$ /home/imre/helm repo list
NAME    	URL                               
jetstack	https://charts.jetstack.io        
bitnami 	https://charts.bitnami.com/bitnami

helm repo seest tarkvara versiooni otsing 

imre@moraal:~/tls$ /home/imre/helm --kubeconfig /home/imre/metallb/climretest.yaml search repo bitnami/dokuwiki --versions

helm repost tarkvara versiooni paigaldamine 

imre@moraal:~/tls$ /home/imre/helm --kubeconfig /home/imre/metallb/climretest.yaml install my-release bitnami/dokuwiki

helm rakenduse kustutamine süsteemist 

imre@moraal:~/tls$ /home/imre/helm --kubeconfig /home/imre/metallb/climretest.yaml delete my-release

Kustomize

Väited

  • on olemas eraldi standalone 'kustomize' nimeline utiliit
  • tavaline kubectl utiliit sisaldab kustomize tuge

Tööpõhimõte

Eesmärgiks on lihtsustada mitmete sarnaste deploymentide tegemist, nt samast rakendusest dev, test ja prod deploymentide paigaldamist.

Kasutamine

Moodustada sellise kataloogid ja failid 

  291808      1 drwxrwxr-x   4 imre     imre            4 nov   10 23:40 .
   303260      1 drwxrwxr-x   2 imre     imre            4 nov   10 23:40 ./base
   295150      5 -rw-rw-r--   1 imre     imre          280 nov   10 23:27 ./base/deployment.yaml
   297710      1 -rw-rw-r--   1 imre     imre           70 nov   10 23:32 ./base/kustomization.yaml
   303262      1 drwxrwxr-x   4 imre     imre            4 nov   10 23:40 ./overlays
   302788      1 drwxrwxr-x   2 imre     imre            4 nov   11 00:03 ./overlays/prod
   303190      5 -rw-rw-r--   1 imre     imre          105 nov   11 00:01 ./overlays/prod/kustomization.yaml
   313554      1 -rw-rw-r--   1 imre     imre           83 nov   11 00:03 ./overlays/prod/replica-count.yaml
   302786      1 drwxrwxr-x   2 imre     imre            4 nov   11 00:03 ./overlays/dev
   313104      1 -rw-rw-r--   1 imre     imre           83 nov   11 00:03 ./overlays/dev/replica-count.yaml
   303078      1 -rw-rw-r--   1 imre     imre          103 nov   11 00:02 ./overlays/dev/kustomization.yaml

kus on failidel selline sisu

base/deployment.yaml 

imre@moraal:~/kustomize$ cat base/deployment.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx

base/kustomization.yaml 

imre@moraal:~/kustomize$ cat base/kustomization.yaml 
resources:
  - deployment.yaml

commonLabels:
  owner: label-imre

overlays/prod/kustomization.yaml 

imre@moraal:~/kustomize$ cat overlays/prod/kustomization.yaml
bases:
  - ../../base
  
namePrefix: prod-

commonLabels:
  env: prod

patches:
  - replica-count.yaml

overlays/prod/replica-count.yaml 

imre@moraal:~/kustomize$ cat overlays/prod/replica-count.yaml 
apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
spec:
  replicas: 6

Deployment prod tekitamiseks sobib öelda 

imre@moraal:~/kustomize$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/climretest.yaml create -k overlays/prod

ja kustutamiseks 

imre@moraal:~/kustomize$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/climretest.yaml delete -k overlays/prod

Misc

Automaatselt deployment.yaml genereerimine 

$ /home/imre/kubectl create deployment --kubeconfig /home/imre/Downloads/climretest.yaml --dry-run=client nginx --image nginx -o yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: nginx
  name: nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx
        name: nginx
        resources: {}
status: {}

Kasulikud lisamaterjalid

Pärit leheküljelt "https://www.auul.pri.ee/wiki/index.php?title=Kubernetes_-_2022_sügis&oldid=415"