Kubernetes - 2022 sügis
Allikas: Imre kasutab arvutit
Mine navigeerimisribaleMine otsikasti
Sissejuhatus
TODO
Rancher Management Cluster on Docker
Käivitamine
docker run -d --restart=unless-stopped \ -p 80:80 -p 443:443 \ --privileged \ rancher/rancher:latest
kus
- TODO
Jälgida käivitumist, kulub mitu minutit
root@tf-vm-1:~# docker logs -f practical_mcnulty
Seejärel pöörduda webgui haldusliidese poole, https://192.168.110.11/
Kasulikud lisamaterjalid
Node tekitamine
Node driver - https://github.com/lnxbil/docker-machine-driver-proxmox-ve/releases/download/v4/docker-machine-driver-proxmoxve.linux-amd64
https://github.com/rancher/os/releases aadressilt saab kopeerida
https://github.com/rancher/os/releases/download/v1.5.8/rancheros-proxmoxve-autoformat.iso
Template, defaultist erinevad
- debugDriver - linnutada
- debugResty - linnutada
- provisionStrategy - cdrom
- proxmoxHost - 192.168.110.250
- proxmoxNode - pm-kns
- proxmoxPool - vaiki
- proxmoxRealm - pam
- proxmoxUserName - root
- proxmoxUserPassword - parool
- sshPassword - tühi (tundub, et toimib rancheos default)
- sshPort - 22
- sshUsername - tühi (tundub, et toimib rancheos default)
- vmCloneFull - 2
- vmCloneVmid - tühi
- vmCpu -
- vmCpuCores - 1
- vmCpuSockets - 4
- vmImageFile - local:iso/rancheros-proxmoxve-autoformat.iso
- vmMemory - 4
- vmNetBridge - vmbr0
- vmNetFirewall - 0
- vmNetModel - virtio
- vmNetTag - 0
- vmScsiController - virtio-scsi-pci
- vmStoragePath - sn_data (tundub, et peab olema proxmox Directory tüüpi)
- vmStorageSize - 32
- vmStorageType - QCOW2
ja sama json kujul
{
"annotations": {
"ownerBindingsCreated": "true"
},
"baseType": "nodeTemplate",
"cloudCredentialId": null,
"created": "2022-10-26T22:51:41Z",
"createdTS": 1666824701000,
"creatorId": "user-7rrms",
"driver": "proxmoxve",
"engineEnv": { },
"engineInstallURL": "https://releases.rancher.com/install-docker/20.10.sh",
"engineLabel": { },
"engineOpt": { },
"engineRegistryMirror": [ ],
"id": "cattle-global-nt:nt-d9w8b",
"labels": {
"cattle.io/creator": "norman"
},
"links": {
"nodePools": "…/v3/nodePools?nodeTemplateId=cattle-global-nt%3Ant-d9w8b",
"nodes": "…/v3/nodes?nodeTemplateId=cattle-global-nt%3Ant-d9w8b",
"self": "…/v3/nodeTemplates/cattle-global-nt:nt-d9w8b",
"update": "…/v3/nodeTemplates/cattle-global-nt:nt-d9w8b"
},
"logOpt": { },
"name": "tmplimre",
"principalId": "local://user-7rrms",
"proxmoxveConfig": {
"debugDriver": true,
"debugResty": true,
"provisionStrategy": "cdrom",
"proxmoxHost": "192.168.110.250",
"proxmoxNode": "pm-kns",
"proxmoxPool": "vaiki",
"proxmoxRealm": "pam",
"proxmoxUserName": "root",
"proxmoxUserPassword": "parool",
"sshPassword": "",
"sshPort": "22",
"sshUsername": "",
"vmCienabled": "",
"vmCitype": "",
"vmCloneFull": "2",
"vmCloneVmid": "",
"vmCpu": "",
"vmCpuCores": "1",
"vmCpuSockets": "2",
"vmImageFile": "local:iso/rancheros-proxmoxve-autoformat.iso",
"vmMemory": "2",
"vmNetBridge": "vmbr0",
"vmNetFirewall": "0",
"vmNetModel": "virtio",
"vmNetMtu": "",
"vmNetTag": "0",
"vmNuma": "",
"vmProtection": "",
"vmScsiAttributes": "",
"vmScsiController": "virtio-scsi-pci",
"vmStartOnboot": "",
"vmStoragePath": "sn_data",
"vmStorageSize": "12",
"vmStorageType": "QCOW2",
"vmVmidRange": ""
},
"state": "active",
"storageOpt": { },
"transitioning": "no",
"transitioningMessage": "",
"type": "nodeTemplate",
"useInternalIpAddress": true,
"uuid": "a6c62f53-9316-41bb-8242-2d641988e522"
}
Rancher hallatud klastriga suhtlemine kubectl utiliidiga
Rancher hallatud klastriga töökohaarvutist käsurealt st kubectl utiliidi abil suhtlemiseks tuleb esmalt kopeerida kubectl utiliit
TODO
Seejärel Rancher webgui liidesest kopeerida klastri seadistus
TODO
ja klastri poole pöördumiseks öelda
$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/clubu1.yaml get all --all-namespaces -o wide NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES cattle-fleet-system pod/fleet-agent-bfc5655cc-crbl6 1/1 Running 0 10m 10.42.0.12 clubu1 <none> <none> cattle-system pod/cattle-cluster-agent-674cc68d59-zgrkq 1/1 Running 1 (11m ago) 14m 10.42.0.5 clubu1 <none> <none> cattle-system pod/cattle-node-agent-n56dp 1/1 Running 0 14m 192.168.110.13 clubu1 <none> <none> cattle-system pod/helm-operation-7vpbz 0/2 Completed 0 9m36s 10.42.0.13 clubu1 <none> <none> ...
MetalLB load balanceri ja NginX ingress kontrolleri kasutamine
Paigaldamine
$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/clc.yaml apply -f \ https://raw.githubusercontent.com/metallb/metallb/v0.13.7/config/manifests/metallb-native.yaml
Tulemusena käivitatakse mitmesugused tegevused
imre@moraal:~/postgres-operator$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/clc.yaml get all -n metallb-system NAME READY STATUS RESTARTS AGE pod/controller-6c58495cbb-qnb8h 1/1 Running 0 2d15h pod/speaker-lpkf9 1/1 Running 0 2d15h NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE service/webhook-service ClusterIP 10.43.195.140 <none> 443/TCP 2d15h NAME DESIRED CURRENT READY UP-TO-DATE AVAILABLE NODE SELECTOR AGE daemonset.apps/speaker 1 1 1 1 1 kubernetes.io/os=linux 2d15h NAME READY UP-TO-DATE AVAILABLE AGE deployment.apps/controller 1/1 1 1 2d15h NAME DESIRED CURRENT READY AGE replicaset.apps/controller-6c58495cbb 1 1 1 2d15h
Tekitada aadresside pool mida metallb kasutab teenuste jaoks
$ cat ipaddress_pools.yaml apiVersion: metallb.io/v1beta1 kind: IPAddressPool metadata: name: production namespace: metallb-system spec: addresses: - 192.168.110.131-192.168.110.135 --- apiVersion: metallb.io/v1beta1 kind: L2Advertisement metadata: name: l2-advert namespace: metallb-system
Tekitamiseks sobib öelda
$ kubectl apply -f ~/metallb/ipaddress_pools.yaml ipaddresspool.metallb.io/production created l2advertisement.metallb.io/l2-advert created
Tulemust on võimalik vaadelda nt Rancher webgui keskkonnas valides
More Resources -> metallb.io -> IPAddressPools ja L2Advertisements
Kasulikud lisamaterjalid
NginX ingress kontroller
Paigaldamiseks deploymentina sobib öelda
$ controller_tag=$(curl -s https://api.github.com/repos/kubernetes/ingress-nginx/releases/latest | grep tag_name | cut -d '"' -f 4)
$ wget -O nginx-ingress-controller-deploy.yaml https://raw.githubusercontent.com/kubernetes/ingress-nginx/${controller_tag}/deploy/static/provider/baremetal/deploy.yaml
imre@moraal:~/metallb$ /home/imre/kubectl --kubeconfig climretest.yaml apply -f nginx-ingress-controller-deploy.yaml
Leida Rancher webgui pealt
Service: ingress-nginx-controller
ning pressida Edit YAML ning muuta
spec.type -> LoadBalancer
Kasulikud lisamaterjalid
- https://computingforgeeks.com/deploy-nginx-ingress-controller-on-kubernetes-using-helm-chart/
- https://platform9.com/blog/using-metallb-to-add-the-loadbalancer-service-to-kubernetes-environments/
- https://blog.opstree.com/2020/10/13/kubernetes-diary-software-loadbalancer/
Deployment tekitamine ja publitseerimine metallb + nginx ingress kontrolleri abil
Paigaldada nt httpd, valida webgui liideses
Workload -> Deployments
ning paremas paneelis täita
- Namespace - default
- Name - dm-httpd
- Replicas - 1
- General -> Image - httpd
ning pressida Create, tulemusena peab tekkima muu hulgas Pod. Panna tähele Deployment juures Pod Labels nime ja väärtust
- Key - workload.user.cattle.io/workloadselector
- Value - apps.deployment-default-dm-httpd
Tekitada Service svc-httpd, valida webgui liideses
Service Discovery -> Services -> Create - Cluster IP
- Port Name - port-httpd
- Listening Port - 80
- Protocol - tcp
- Target Port - 80
Lisaks paremas paneelis ühendada kokku Service ja Deployment valides Selectors ning täita lahtrid
- Key - workload.user.cattle.io/workloadselector
- Value - apps.deployment-default-dm-httpd
Tulemusena peab lahtrite kohale ilmuma midagi sellist
Matches 1 of 60 pods: "dm-httpd-b65cf6875-b7zjq"
Kirjeldada Ingress valides webgui liideses
Service Discovery -> Ingresses -> Create
ning paremas paneelis täita
- Namespace - default
- Name - ing-httpd
- Rules -> Request Host - httpd-imre-k8s-test.eenet.ee
- Rules -> Path -> Prefix - /
- Rules -> Path -> Target Service - svc-httpd
- Ingress Class -> nginx
cert-manager kasutamine
cert-manager abil saab teenuste juures korraldada nt Lets Encrypt sertifikaatide kasutamise. Järneva eelduseks on
- toimiv metallb + nginx-ingress-controller Rancher + Kubernetes klaster
- nö tavalisel viisil toimiv deployment, mis on ligipääsetav üle metallb + nginx-ingress-controller'i
- teenuse dns nimi viitab kõnealusele teenusele tema metallb avaliku ip aadressiga
- avalikule ip aadressile on avalikust võrgust ligipääs (vastasel korral ei töötaks LE http-challenge)
- tundub, et sellepärast ei pea muretsema, et LE asus päring LE isanda juurde paistaks samalt ip aadressilt, kus asub teenus ise
ClusterIssuer'ide moodustamine
ClusterIssuer moodustamine LE staging jaoks
$ cat issuer-staging.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-staging
spec:
acme:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: imre@auul.pri.ee
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-staging
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
ClusterIssuer moodustamine LE prod jaoks
$ cat issuer-staging.yaml
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
name: letsencrypt-prod
spec:
acme:
# The ACME server URL
server: https://acme-staging-v02.api.letsencrypt.org/directory
# Email address used for ACME registration
email: imre@auul.pri.ee
# Name of a secret used to store the ACME account private key
privateKeySecretRef:
name: letsencrypt-staging
# Enable the HTTP-01 challenge provider
solvers:
- http01:
ingress:
class: nginx
issueride tekitamiseks sobib öelda
$ /home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml apply -f issuer-staging.yaml $ /home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml apply -f issuer-prod.yaml
Tulemuse kontrolliks peab nägema
imre@moraal:~/tls$ /home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml describe clusterissuer letsencrypt-prod
..
Status:
Acme:
Last Registered Email: imre@auul.pri.ee
Uri: https://acme-v02.api.letsencrypt.org/acme/acct/813299067
Conditions:
Last Transition Time: 2022-11-07T17:47:40Z
Message: The ACME account was registered with the ACME server
Observed Generation: 1
Reason: ACMEAccountRegistered
Status: True
Type: Ready
Events: <none>
Ingress moodustamine
Olemasoleva service jaoks tekitatakse ingress
$ cat dokuwiki-ing.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: dokuwiki
annotations:
cert-manager.io/cluster-issuer: "letsencrypt-prod"
spec:
ingressClassName: nginx
tls:
- hosts:
- dokuwiki-imre-k8s-test.auul.pri.ee
secretName: dokuwiki-imre-k8s-test-tls
rules:
- host: dokuwiki-imre-k8s-test.auul.pri.ee
http:
paths:
- path: /
pathType: Prefix
backend:
service:
name: my-release-dokuwiki
port:
number: 80
kus
- metadata.annotations kirjeldab cert-mananger kasutamise letsencrypt-prod issueriga
- spec.ingressClassName - nginx kirjeldab seose ingress kontrolleriga
Tulemuse kontrollimine
Tulemuse kontrollimiseks peab tekkima Rancher webgui peale vasakus paneelis sektsiooni
- More Resources -> Cert Manager -> CertificateRequests
- More Resources -> Cert Manager -> Certificates
- ...
Sertifikaadi uuesti väljastamise sundimiseks sobib öelda
/home/imre/kubectl --kubeconfig /home/imre/metallb/climretest.yaml delete secret dokuwiki-imre-k8s-test-tls
Lisaks on näha klastri webgui avalehel Events sektsioonis kuidas sertifikaadid moodustuvad.
imre@moraal:~/tls$ /home/imre/kubectl --kubeconfig /home/imre/Downloads/climretest.yaml describe certificate dokuwiki-imre-k8s-test-tls
