Lehekülje Ceph kasutamine Debian Wheezy v. 7 operatsioonisüsteemiga lähteteksti vaatamine

===Sissejuhatus===

Ceph http://en.wikipedia.org/wiki/Ceph_(file_system) ...

Tundub, et 2019 aasta lõpus on Ceph kasutamiseks erinevaid võimalusi

* operatsioonisüsteemi paketihaldus - tõenäoliselt suhteliselt vana tarkvara
* ceph.io repositoorium
* proxmox.com ve tarkvara - väga mugav paigaldada ja uus tarkvara
* ceph-ansible abil - tavaliselt virtualbox + vagrant + ansible instrumentide abil; seda soovitavad harjutamiseks mitmed ceph teemalised uued raamatud

===Mõisted===

* RADOS - Reliable Autonomic Distributed Object Store
* RDB - RADOS block device
* MON
* OSD
* RGW
* MGR

===Tööpõhimõte===

Põhimõtteliselt sisaldab CEPH klaster selliseid komponente ja nad võiksid töötada iseseisvalt iga omas arvutis

 TODO

Praktiliselt võib olla mõistlik komponente konsolideerida, nt selliselt

 TODO

Seadistusfail

<pre>
root@ceph-client0:~# cat /etc/ceph/ceph.conf 
[global]
fsid = 5d4d2349-0fb0-4c0f-a361-5e18420ff755
mon_initial_members = ceph-mon0
mon_host = 192.168.110.182
auth_cluster_required = cephx
auth_service_required = cephx
auth_client_required = cephx
public network = 192.168.110.0/24
cluster network = 192.168.119.0/24
</pre>

kus

* TODO

===Arvutite ettevalmistamine===

 TODO

 # apt-get install python2.7 python-minimal

===Paigaldamine===

ceph-deploy abil paigaldamiseks sobib admin arvutis öelda
--cluster-network --public-network
 admin $ ceph-deploy --public-network=192.168.110.0/24 --cluster-network=192.168.119.0/24  new ceph-mon0
 admin $ ceph-deploy install --release=nautilus ceph-mon0 ceph-osd0 ceph-osd1 ceph-osd3 ceph-mgr0
 admin $ ceph-deploy mon create-initial
 admin $ ceph-deploy admin ceph-mon0 ceph-osd0 ceph-osd1 ceph-osd3 ceph-mgr0
 admin $ ceph-deploy mgr create ceph-mgr0

osd arvutite ettevalmistamine

 admin $ ceph-deploy osd create --data /dev/vdb ceph-osd0
 admin $ ceph-deploy osd create --data /dev/vdb ceph-osd1
 admin $ ceph-deploy osd create --data /dev/vdb ceph-osd2

Tulemuse kontrollimine

 admin $ ssh osd-mon0 sudo ceph health
 admin $ ssh osd-mon0 sudo ceph -s

===Klastri käivitamine ja seiskamine===

====Käivitamine====

* käivitada admin arvuti
* käivitada monitorid
* käivitada osd arvutid
* veenduda et arvutid käivitusid jne
* eemaldada lipud

<pre>
# ceph osd unset pause
# ceph osd unset nodown
# ceph osd unset nobackfill
# ceph osd unset norebalance
# ceph osd unset norecover
# ceph osd unset noout
</pre>

* kontrollida, et klastrid saab kasutada

====Seiskamine====

* lõpetada klastri kasutamine klientide poolt
* veenduda, et klaster on töökorras
* seadistada lipud

<pre>
# ceph osd set noout
# ceph osd set norecover
# ceph osd set norebalance
# ceph osd set nobackfill
# ceph osd set nodown
# ceph osd set pause
</pre>

* seisata osd arvutid
* seisata monitorid
* seisata admin arvuti

===Dashboard kasutamine===

Dashboard teenuse aadress on https://192.168.110.181:8443 Dashboard käivitamiseks tuleb öelda

 root@ceph-client0:~# ceph mgr module enable dashboard
 root@ceph-client0:~# ceph dashboard create-self-signed-cert
 root@ceph-client0:~# ceph dashboard set-login-credentials imre parool

===OSD arvuti lisamine===

Ceph tarkvara paigaldamiseks sobib öelda

 admin $ ceph-deploy install --release=nautilus ceph-osd3
 admin $ ceph-deploy admin ceph-osd3
 admin $ ceph-deploy disk list ceph-osd3
 admin $ ceph-deploy disk zap ceph-osd3 /dev/vdb
 admin $ ceph-deploy osd create --data /dev/vdb ceph-osd3

jälgida andmete laiali kopeerimist

 ceph-mon0 # ceph -w

===OSD arvuti eemaldamine===

* arvuti väljalülitamine klastrist

 root@ceph-mon0:~# ceph osd out 3
 marked out osd.3. 

* jälgida et andmed migreeritakse ära

 # ceph -w

* protsesside töö lõpetamine

 sepp@ceph-osd3:~$ sudo systemctl stop ceph-osd@3

* kustutada asjad

 # ceph osd purge 3 --yes-i-really-mean-it
 purged osd.3

* tarkvara eemaldamine

 admin $ ceph-deploy purge ceph-osd3
 admin $ ceph-deploy purgedata ceph-osd3

* kustutada plokkseadmelt lvm seadistus

 ceph-osd3 # vgremove grupinimi
 ceph-osd3 # pvremove /dev/vdb

===Monitor arvuti lisamine===

 admin $ ceph-deploy install --release=nautilus ceph-mon1
 admin $ ceph-deploy admin ceph-mon1

===Monitori eemaldamine===

TODO

===Manageri lisamine===

 admin $ ceph-deploy mgr create ceph-osd0

===Manageri eemaldamine===

 TODO

===Plokkseadme kasutamine===

Pool moodustamine

 root@ceph-client0:~# ceph osd pool create web-services 128
 root@ceph-client0:~# ceph osd pool application enable web-services rbd
 root@ceph-client0:~# ceph osd lspools
 1 web-services

RADOS Block Device moodustamine

 root@ceph-client0:~# rbd -p web-services create ceph-client1-rbd1 --size 10240
 root@ceph-client0:~# rbd -p web-services info ceph-client1-rbd1
 root@ceph-client0:~# rbd -p web-services feature disable ceph-client1-rbd1 exclusive-lock,object-map,fast-diff,deep-flatten
 root@ceph-client0:~# rbd -p web-services map ceph-client1-rbd1

Plokkseadme kasutamine

 root@ceph-client0:~# fdisk /dev/rbd0 -l
 root@ceph-client0:~# mkfs.ext4 /dev/rbd0
 root@ceph-client0:~# mkdir /mnt/rbd0
 root@ceph-client0:~# mount /dev/rbd0 /mnt/rbd0

Plokkseadme kustutamine

 root@ceph-client0:~# umount /mnt/rbd0
 root@ceph-client0:~# rbd unmap /dev/rbd0
 root@ceph-client0:~# rbd -p web-services rm ceph-client1-rbd1
 root@ceph-client0:~# rbd -p web-services ls

Pool kustutamine

 root@ceph-client0:~# ceph osd pool rm web-services web-services --yes-i-really-really-mean-it
 Error EPERM: pool deletion is disabled; you must first set the mon_allow_pool_delete config option to true before you can destroy a pool

Edasi aitab

<pre>
root@ceph-mon0:~# ceph daemon mon.ceph-mon0 config set mon_allow_pool_delete true
{
    "success": "mon_allow_pool_delete = 'true' "
}
</pre>

 root@ceph-mon0:~# ceph osd pool delete imretest1 imretest1 --yes-i-really-really-mean-it
 pool 'imretest1' removed

Koormuse testimine

 # rados bench -p web-services 100 write

===Arvutis ceph plokkseadme kasutamine===

Arvutis ceph ressursside kasutamiseks tuleb paigaldada ceph tarkvara

 # apt-get install ceph-common

Kopeerida ceph serverist keyring

<pre>
# cat /etc/ceph/ceph.client.admin.keyring
[client.admin]
	key = salajaneväärtus
	caps mds = "allow *"
	caps mgr = "allow *"
	caps mon = "allow *"
	caps osd = "allow *"
</pre>

Kopeerida ceph serverist seadistusfail, tundub, et oluline osa kliendi jaoks on selline

<pre>
[global]
	 fsid = natuke-salajane-väärtus
	 mon_host = 10.800.0.121 10.800.0.122 10.800.0.123 10.800.0.124

[client]
	 keyring = /etc/ceph/ceph.client.admin.keyring

</pre>

Seejärel saab hakata käsundama nö tavalisel viisil

 # ceph -s
 ...

Selleks, et /dev/rbd0 plokkseade tekiks automaatselt arvuti käivitamise ajal sobib kasutada faili

<pre>
root@ceph-client0:~# cat /etc/ceph/rbdmap 
# RbdDevice		Parameters
cephpool2/cephpool2-rbd1 id=admin,keyring=/etc/ceph/ceph.client.admin.keyring
</pre>

kus

* id - 
* TODO

käivitamine toimub

 # systemctl status rbdmap

Tulemusena tekib plokkseade /dev/rbd0 vms.

===Kasulikud lisamaterjalid===

* http://ceph.com/docs/master/
* http://ceph.com/
* http://blog.bob.sh/2012/02/basic-ceph-storage-kvm-virtualisation.html
* https://www.berrange.com/posts/2011/10/12/setting-up-a-ceph-cluster-and-exporting-a-rbd-volume-to-a-kvm-guest/
* http://www.inktank.com/
* https://icicimov.github.io/blog/virtualization/Adding-RBD-CEPH-remote-cluster-storage-to-Proxmox/