ZFS kasutamine operatsioonisüsteemiga Debian v. 11
Allikas: Imre kasutab arvutit
Mine navigeerimisribaleMine otsikasti
Sissejuhatus
TODO
Paigaldamine
2021 aasta talvel on root-on-zfs debian 11 tekitamiseks üks suhteliselt mugav võimalus
- kasutada skripti aadressilt https://github.com/hn/debian-buster-zfs-root (debian v. 10)
- uuendada süsteem v. 11 peale
- uuendada zfs pool
# zpool upgrade rpool
hn kasutaja skript töötab, aga tulemus erineb oluliselt https://openzfs.github.io/openzfs-docs/Getting%20Started/Debian/Debian%20Buster%20Root%20on%20ZFS.html juhendis kirjeldatud viisil saadud süsteemist
- fdisk layout
- dataset layout
- /tmp kataloog on no-exec
- ei ole eraldi bpool ja rpool, on kasutusel ainult rpool
- jne
Debian v. 11 zfs omadusel 2021 talvel
- sisaldab zfs v. 2.0 tarkvara
- os intalleris puudub zfs tugi
- zfs ja os vaheline integratsioon puudub (a la ubuntu zsys, zsysctl jms)
zfs põhine dual boot
zfs põhise sama pool datasettide abil tehtud dual boot kasutamiseks sobib
- lähtepunktiks on töötav süsteem
- moodustada juurfailisüsteemile vastavast datasetist snapshot ja kloon
# zfs snapshot rpool/ROOT/debian-buster@ss # zfs list -t snapshot # zfs clone rpool/ROOT/debian-buster@ss rpool/ROOT/debian-tmp
- grub bootloaderis muuta rpool/ROOT/debian-buster -> rpool/ROOT/debian-tmp (kahes kohas)
- bootida break=mount abil initramfs prompti
- eemaldada käesoleval juurikal mountpoint
zpool import rpool zfs set mountpoint=none rpool/ROOT/debian-buster
- lisada uuelda juurikale mountpoint
zfs set mountpoint=/ rpool/ROOT/debian-tmp zpool export rpool
- kuna /etc/fstab peale pole zfs ühenduskohta kirjeldatud, siis fstab failis muudatusi pole vaja teha
- bootida arvutit edasi, ctrl-d
- klooni jäädavalt kasutama jäädes tuleks teha promote ning eemaldada kloon ja snapshot
# zfs promote rpool/ROOT/debian-tmp
Tulemusena saab sisse logida, kasutada jne.
df -h vs zfs list
zpool list väljund
root@deb11-tookoht:~# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT bpool 480M 153M 327M - - 13% 31% 1.00x ONLINE - rpool 62.5G 58.4G 4.07G - - 84% 93% 1.00x ONLINE -
zfs list väljund
root@deb11-tookoht:~# zfs list NAME USED AVAIL REFER MOUNTPOINT bpool 152M 200M 96K / bpool/BOOT 151M 200M 96K none bpool/BOOT/debian 151M 200M 112M legacy rpool 59.4G 1.11G 96K / rpool/ROOT 31.3G 1.11G 96K none rpool/ROOT/debian 31.3G 1.11G 30.7G / rpool/home 18.2G 1.11G 584M /home rpool/home/imre 17.5G 1.11G 17.5G /home/imre rpool/home/root 94.2M 1.11G 94.2M /root rpool/swap 4.25G 2.12G 3.24G - rpool/var 5.68G 1.11G 96K /var rpool/var/cache 5.58G 1.11G 5.58G /var/cache rpool/var/lib 96K 1.11G 96K /var/lib rpool/var/log 109M 1.11G 109M legacy rpool/var/spool 120K 1.11G 120K legacy
kus
- rpool kõigi non-volume tüüpi datasettide jaoks on sama palju vaba ruumi kuna zfs nö olemus mõttes asuvad nad ühe nö juur-dataset all
- rpool/swap on volume tüüpi dataset ja seetõttu eristub
- dataset USED tulba väärtus sisaldab kõigi alla jäävate datasettide väärtuste summat
Esitada ainult volume tüüpi datasetid
root@deb11-tookoht:~# zfs list -t volume NAME USED AVAIL REFER MOUNTPOINT rpool/swap 4.25G 2.12G 3.24G -
zfs list -o space väljund
root@deb11-tookoht:~# zfs list -o space NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD bpool 200M 152M 0B 96K 0B 152M bpool/BOOT 200M 151M 0B 96K 0B 151M bpool/BOOT/debian 200M 151M 38.9M 112M 0B 0B rpool 1.11G 59.4G 0B 96K 0B 59.4G rpool/ROOT 1.11G 31.3G 0B 96K 0B 31.3G rpool/ROOT/debian 1.11G 31.3G 596M 30.7G 0B 0B rpool/home 1.11G 18.2G 0B 584M 0B 17.6G rpool/home/imre 1.11G 17.5G 0B 17.5G 0B 0B rpool/home/root 1.11G 94.2M 0B 94.2M 0B 0B rpool/swap 2.12G 4.25G 0B 3.24G 1.01G 0B rpool/var 1.11G 5.68G 0B 96K 0B 5.68G rpool/var/cache 1.11G 5.58G 0B 5.58G 0B 0B rpool/var/lib 1.11G 96K 0B 96K 0B 0B rpool/var/log 1.11G 109M 0B 109M 0B 0B rpool/var/spool 1.11G 120K 0B 120K 0B 0B
kus
- lisaks eelmisele 'zfs list' väljundile esitatakse '-o space' suvandi kasutamisel põhjalikum info
df -h väljund
root@deb11-tookoht:~# df -h Filesystem Size Used Avail Use% Mounted on udev 5.9G 0 5.9G 0% /dev tmpfs 1.2G 1.3M 1.2G 1% /run rpool/ROOT/debian 32G 31G 1.2G 97% / tmpfs 5.9G 572M 5.3G 10% /dev/shm tmpfs 5.0M 4.0K 5.0M 1% /run/lock tmpfs 4.0M 0 4.0M 0% /sys/fs/cgroup rpool/var/spool 1.2G 128K 1.2G 1% /var/spool rpool/var/log 1.3G 109M 1.2G 9% /var/log bpool/BOOT/debian 312M 113M 200M 37% /boot /dev/vda2 505M 5.3M 499M 2% /boot/efi tmpfs 5.9G 715M 5.2G 12% /tmp rpool/home 1.7G 585M 1.2G 34% /home rpool/home/root 1.3G 95M 1.2G 8% /root rpool/var/cache 6.7G 5.6G 1.2G 84% /var/cache rpool/home/imre 19G 18G 1.2G 95% /home/imre tmpfs 1.2G 68K 1.2G 1% /run/user/1000
kus
- klassikalise df utiliidi väljund ei sobi hästi kõigi zfs detailide esitamiseks, aga ülevaate saab
- Size tulba väärtuseks esitatakse Used + Avail summa (mis on vastavalt zfs list -o space väljundi USEDDS + AVAIL)
2022 kevadel zfs kasutamine
Kasutada olevad plokkseadmed
# lsscsi -s [0:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sda 1.00TB [1:0:0:0] disk ATA Samsung SSD 860 2B6Q /dev/sdb 1.00TB [2:0:0:0] disk ATA Samsung SSD 870 1B6Q /dev/sdc 4.00TB [3:0:0:0] disk ATA Samsung SSD 870 1B6Q /dev/sdd 4.00TB [N:0:4:1] disk Samsung SSD 970 EVO Plus 2TB__1 /dev/nvme0n1 2.00TB [N:1:4:1] disk Samsung SSD 970 EVO Plus 2TB__1 /dev/nvme1n1 2.00TB
kus
# ls -l /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB016* lrwxrwxrwx 1 root root 9 May 14 10:22 /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01688K -> ../../sdd lrwxrwxrwx 1 root root 9 May 14 10:22 /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01694L -> ../../sdc
Moodustatakse ssd 870 seadmetest 2-way mirror
# zpool create -o ashift=13 zp_evo_870 mirror /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01688K /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01694L # zpool set autoexpand=on zp_evo_870 # zfs set compression=zstd zp_evo_870 # zfs set mountpoint=none zp_evo_870
kus
- tark on kasutada füüsiliste seadmetele viitamiseks unikaalseid nimesid (alternatiiv oleks /dev/sdc ja /dev/sdd aga need väärtused sõltuvad ketaste sata portidesse kinnitamise järjekorrast jms)
- väidetavalt on samsung ssd ketaste puhul tark kasutada ashift=13 (ehk 8K plokk) väärtust - https://arstechnica.com/information-technology/2020/05/zfs-101-understanding-zfs-storage-and-performance/
ZFS failisüsteem on liiga täis ja läks katki
Peale küllaltki täis failisüsteemi otsas rekursiivse snapshot ütlemist
# zfs snapshot -r sn_zfs_ssd@replica20220515
ütleb failisüsteemi vastu nii
# zfs destroy sn_zfs_ssd/vm-1125-disk-0@replica20220515 internal error: cannot destroy snapshots: Channel number out of range Aborted
kusjuures lisaks sellele, et ta on täis
# zpool list NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT sn_zfs_ssd 1.81T 953G 903G - - 44% 51% 1.00x DEGRADED - # zfs list sn_zfs_ssd -o space NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD sn_zfs_ssd 0B 2.14T 0B 96K 0B 2.14T
on ta veel mirror-0 ja on katki ka
# zpool status -v
pool: sn_zfs_ssd
state: DEGRADED
status: One or more devices has experienced an error resulting in data
corruption. Applications may be affected.
action: Restore the file in question if possible. Otherwise restore the
entire pool from backup.
see: https://openzfs.github.io/openzfs-docs/msg/ZFS-8000-8A
scan: scrub repaired 0B in 00:22:56 with 0 errors on Sun Feb 13 00:46:58 2022
config:
NAME STATE READ WRITE CKSUM
sn_zfs_ssd DEGRADED 0 0 0
wwn-0x5002538e40a0eb32 DEGRADED 3 0 37 too many errors
wwn-0x5002538e40a05fa2 DEGRADED 3 2 1.39K too many errors
errors: Permanent errors have been detected in the following files:
sn_zfs_ssd/vm-102-disk-5:<0x1>
Tundub, et selles olukorras võib olla abi sellisest protsetuurist
- tekitada vabale meediale, nt lvm volume peale vastavatest scsi seadmetest dd abil plokkseadme tasemel koopiad
# dd if=/dev/sda of=/mnt/sdd/evo-860-sda.img bs=1M status=progress # dd if=/dev/sdb of=/mnt/sdd/evo-860-sdb.img bs=1M status=progress
- tekitada proxmox arvutit (nt virtuaalne) ja võtta seal külge koopiaid sisaldav failisüsteem ja lisaks veel suurem muu failisüsteem
- moodustada virtuaalne arvuti ja teha talle kõik seadmed (kokku neli) nähtavaks
- teha zpool import
# zpool import ..
- teha ühe kaupa zpool seadme asendus suurema vastu (toimub resilvering)
# zpool replace sn_zfs_ssd /dev/vdb /dev/vdd # zpool replace sn_zfs_ssd /dev/vdc /dev/vde # zpool status pool: sn_zfs_ssd state: ONLINE scan: resilvered 445G in 00:44:11 with 0 errors on Sun May 15 23:06:07 2022 config: NAME STATE READ WRITE CKSUM sn_zfs_ssd ONLINE 0 0 0 vdd ONLINE 0 0 0 vde ONLINE 0 0 0 <pre> * paistab, et süsteem on oluliselt paranenud, nüüd saab öelda edasi zfs online -e sn_zfs_ssd /dev/vdd ja zpool online -e sn_zfs_ssd /dev/vde Tulemusena on olukord jällegi kontrolli all, zpool ja zfs käsud töötavad jne (nt saab snapshotisid tekitada ja eemaldada jne) <pre> # zpool list sn_zfs_ssd NAME SIZE ALLOC FREE CKPOINT EXPANDSZ FRAG CAP DEDUP HEALTH ALTROOT sn_zfs_ssd 3.11T 953G 2.18T - - 25% 29% 1.00x ONLINE - # zfs list sn_zfs_ssd NAME USED AVAIL REFER MOUNTPOINT sn_zfs_ssd 1.52T 1.49T 96K /sn_zfs_ssd
parandatud tõmmised saab järgmises arvutis kasutusse võtta nt qemu-nbd utiliidi abil
# qemu-nbd -c /dev/nbd0 /mnt/sdd-ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01688K/dir/images/102/vm-102-disk-0.qcow2 # qemu-nbd -c /dev/nbd1 /mnt/sdd-ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01688K/dir/images/102/vm-102-disk-1.qcow2
Seejärel zpool import peaks avastama ressursi. Kasutamise lõpus
# zpool export poolnimi # qemu-nbd -d /dev/nbd0 # qemu-nbd -d /dev/nbd1
Raid-0 pool kasutamise jätkamiseks raid-1 poolina sobib öelda
# zpool attach sn_zfs_ssd /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01694L /dev/disk/by-id/ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01688K
Tulemusena algab resilverdamine
# zpool status pool: sn_zfs_ssd state: ONLINE status: One or more devices is currently being resilvered. The pool will continue to function, possibly in a degraded state. action: Wait for the resilver to complete. scan: resilver in progress since Wed May 18 18:11:05 2022 222G scanned at 1.98G/s, 29.1G issued at 266M/s, 1.58T total 29.1G resilvered, 1.79% done, 01:42:17 to go config: NAME STATE READ WRITE CKSUM sn_zfs_ssd ONLINE 0 0 0 mirror-0 ONLINE 0 0 0 ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01694L ONLINE 0 0 0 ata-Samsung_SSD_870_EVO_4TB_S6BCNS0RB01688K ONLINE 0 0 0 (resilvering) errors: No known data errors
Kasulikud lisamaterjalid
zpool sisu kopeerimine teisse zpool'i
Esmalt moodustatakse lähte poolis snapshot, tulemusena tekib kõigile datasettidele v.a. olemasolevad snapshotid jms, snapshot, sn_zfs_ssd on pool nimi; -r on rekursiivselt
# zfs snapshot -r sn_zfs_ssd@replica20220516
Seejärel kopeeritakse, zp_evo_870 on sihtkohaks oleva pool nimi
# zfs send -R sn_zfs_ssd@replica20220516 | zfs receive -udF zp_evo_870
kus
- -R - tekitada nn replication stream, see sisaldab nö kõike näidatud snapshotile eelnevat
- -u - sihtpunktis mitte püüda monteerida
- -F - kui kopeeritav dataset on olemas, kustutada
- -d - xxx
Tulemusena tekib destination pool peale see snaphost ja seejuures ka kõik nö mis sellest ettepoole jääb (dataset'id, varasemad snapshotid jms). Tõenäoliselt on asjakohane lähte poolist ja siht poolist nö kopeerimise-snapshot eemaldada
# zfs destroy -r sn_zfs_ssd@replica20220516 # zfs destroy -r zp_evo_870@replica20220516
Tundub, et kopeerimise ajal võib tekkida sarnaseid veateateid ja ehk see ei ole nö maailmalõpp
[Mon May 16 02:14:22 2022] block nbd1: Possible stuck request 0000000025b58da6: control (read@509541847040,131072B). Runtime 30 seconds
[Mon May 16 02:14:22 2022] block nbd0: Possible stuck request 00000000db090ca2: control (read@458513432576,126976B). Runtime 30 seconds
[Mon May 16 02:14:22 2022] block nbd1: Possible stuck request 00000000b3423ad4: control (read@509541978112,131072B). Runtime 30 seconds
[Mon May 16 02:14:22 2022] block nbd1: Possible stuck request 00000000e61e4d21: control (read@509542109184,131072B). Runtime 30 seconds
[Mon May 16 02:14:22 2022] block nbd0: Possible stuck request 00000000dcbd73bf: control (read@458513690624,131072B). Runtime 30 seconds
[Mon May 16 02:14:22 2022] block nbd0: Possible stuck request 000000001aa586ea: control (read@458513821696,106496B). Runtime 30 seconds
[Mon May 16 02:14:33 2022] ata1.00: exception Emask 0x0 SAct 0x26ec1000 SErr 0x0 action 0x6 frozen
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/00:60:d8:dd:d5/01:00:ac:01:00/40 tag 12 ncq dma 131072 in
res 40/00:01:06:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/d0:90:d8:de:d5/00:00:ac:01:00/40 tag 18 ncq dma 106496 in
res 40/00:01:00:00:00/00:00:00:00:00/40 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/e0:98:20:00:00/00:00:00:00:00/40 tag 19 ncq dma 114688 in
res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/b8:a8:20:dc:d5/00:00:ac:01:00/40 tag 21 ncq dma 94208 in
res 40/00:00:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/00:b0:48:5c:04/01:00:e1:00:00/40 tag 22 ncq dma 131072 in
res 40/00:00:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/e0:b8:20:00:00/00:00:00:00:00/40 tag 23 ncq dma 114688 in
res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/00:c8:48:5d:04/01:00:e1:00:00/40 tag 25 ncq dma 131072 in
res 40/00:01:01:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/e0:d0:20:00:00/00:00:00:00:00/40 tag 26 ncq dma 114688 in
res 40/00:01:00:4f:c2/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1.00: failed command: READ FPDMA QUEUED
[Mon May 16 02:14:33 2022] ata1.00: cmd 60/e0:e8:20:00:00/00:00:00:00:00/40 tag 29 ncq dma 114688 in
res 40/00:ff:00:00:00/00:00:00:00:00/00 Emask 0x4 (timeout)
[Mon May 16 02:14:33 2022] ata1.00: status: { DRDY }
[Mon May 16 02:14:33 2022] ata1: hard resetting link
[Mon May 16 02:14:39 2022] ata1: SATA link up 6.0 Gbps (SStatus 133 SControl 300)
[Mon May 16 02:14:39 2022] ata1.00: supports DRM functions and may not be fully accessible
[Mon May 16 02:14:39 2022] ata1.00: disabling queued TRIM support
[Mon May 16 02:14:39 2022] ata1.00: supports DRM functions and may not be fully accessible
[Mon May 16 02:14:39 2022] ata1.00: disabling queued TRIM support
[Mon May 16 02:14:39 2022] ata1.00: configured for UDMA/133
[Mon May 16 02:14:39 2022] ata1.00: device reported invalid CHS sector 0
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=46s
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#12 Sense Key : Illegal Request [current]
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#12 Add. Sense: Unaligned write command
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#12 CDB: Read(16) 88 00 00 00 00 01 ac d5 dd d8 00 00 01 00 00 00
[Mon May 16 02:14:39 2022] blk_update_request: I/O error, dev sda, sector 7194664408 op 0x0:(READ) flags 0x80700 phys_seg 5 prio class 0
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#18 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=46s
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#18 Sense Key : Illegal Request [current]
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#18 Add. Sense: Unaligned write command
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#18 CDB: Read(16) 88 00 00 00 00 01 ac d5 de d8 00 00 00 d0 00 00
[Mon May 16 02:14:39 2022] blk_update_request: I/O error, dev sda, sector 7194664664 op 0x0:(READ) flags 0x80700 phys_seg 4 prio class 0
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#21 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=46s
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#21 Sense Key : Illegal Request [current]
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#21 Add. Sense: Unaligned write command
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#21 CDB: Read(16) 88 00 00 00 00 01 ac d5 dc 20 00 00 00 b8 00 00
[Mon May 16 02:14:39 2022] blk_update_request: I/O error, dev sda, sector 7194663968 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#22 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=46s
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#22 Sense Key : Illegal Request [current]
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#22 Add. Sense: Unaligned write command
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#22 CDB: Read(16) 88 00 00 00 00 00 e1 04 5c 48 00 00 01 00 00 00
[Mon May 16 02:14:39 2022] blk_update_request: I/O error, dev sda, sector 3775159368 op 0x0:(READ) flags 0x80700 phys_seg 6 prio class 0
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#25 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=46s
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#25 Sense Key : Illegal Request [current]
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#25 Add. Sense: Unaligned write command
[Mon May 16 02:14:39 2022] sd 0:0:0:0: [sda] tag#25 CDB: Read(16) 88 00 00 00 00 00 e1 04 5d 48 00 00 01 00 00 00
[Mon May 16 02:14:39 2022] blk_update_request: I/O error, dev sda, sector 3775159624 op 0x0:(READ) flags 0x80700 phys_seg 8 prio class 0
[Mon May 16 02:14:39 2022] ata1: EH complete
[Mon May 16 02:14:39 2022] ata1.00: Enabling discard_zeroes_data
Misc
suuruste arvutamine
# a=0; poolname="sn_zfs_ssd"; for i in `zfs list -H -r $poolname -o name,used -p | grep "^$poolname/" | awk {'print $2'}`; do a=`echo $a + $i | bc`; done; sumg=`echo $a/1024^3 | bc`; echo $sumg
1645
# a=0; poolname="zp_evo_870"; for i in `zfs list -H -r $poolname -o name,used -p | grep "^$poolname/" | awk {'print $2'}`; do a=`echo $a + $i | bc`; done; sumg=`echo $a/1024^3 | bc`; echo $sumg
783
kopeerimine, variant 1
# for i in `zfs list -t volume -r sn_zfs_ssd | grep ^sn | awk {'print $1'} | cut -d "/" -f 2`; do echo $i; zfs send sn_zfs_ssd/$i | zfs receive zp_evo_870/$i; echo $?; done
kopeerimine, variant 2
zvol moodustamine ja kasutamine
zvol on zfs nö eripärane dataset ressurss peale moodustatakse reeglina mingi muu andmete haldamise kiht, nt lvm, failisüsteem vms (või jagatakse välja üle võrgu iscsi abil). zvol tekitamiseks sobib öelda
# zfs create -V 10G zpool_wdc/iscsi_pbs
zvol peale ext4 failisüsteemi tekitamiseks sobib öelda
# mkfs.ext4 /dev/zpool_wdc/iscsi_pbs
zvol ressursi suurendamiseks sobib öelda
# zfs set volsize=30G zpool_wdc/iscsi_pbs
zvol ressursi kustutamiseks sobib öelda
# zfs destroy zpool_wdc/iscsi_pbs
zvol suuruste arvutamine
Olgu paista selline ressursside kasutus
root@pm60-trt:~# zfs list -o space | head -n 8 NAME AVAIL USED USEDSNAP USEDDS USEDREFRESERV USEDCHILD sn_zfs_ssd 186G 3.33T 0B 192K 0B 3.33T sn_zfs_ssd/imretestvolume 194G 8.25G 0B 112K 8.25G 0B sn_zfs_ssd/subvol-161-disk-0 7.18G 835M 0B 835M 0B 0B sn_zfs_ssd/vm-100-disk-0 213G 82.5G 0B 55.1G 27.4G 0B sn_zfs_ssd/vm-100-disk-1 186G 3M 0B 176K 2.83M 0B sn_zfs_ssd/vm-102-disk-5 187G 22.7G 0B 21.6G 1.11G 0B sn_zfs_ssd/vm-1033-disk-0 187G 12.4G 0B 11.9G 525M 0B
kusjuures nt
root@pm60-trt:~# zfs get all sn_zfs_ssd/vm-100-disk-0 | egrep " used | available | referenced | volsize | refreservation | usedbydataset | usedbyrefreservation" sn_zfs_ssd/vm-100-disk-0 used 82.5G - sn_zfs_ssd/vm-100-disk-0 available 213G - sn_zfs_ssd/vm-100-disk-0 referenced 55.1G - sn_zfs_ssd/vm-100-disk-0 volsize 80G local sn_zfs_ssd/vm-100-disk-0 refreservation 82.5G local sn_zfs_ssd/vm-100-disk-0 usedbydataset 55.1G - sn_zfs_ssd/vm-100-disk-0 usedbyrefreservation 27.4G -
Tulemust sobib tõlgendada selliselt
- used ja refreservation (ja vist ka volsize) väljendavad proxmox webgui liideses vmid=100 arvuti kettale näidatud mahtu
- referenced ja usedbydataset näitavad kui palju reaalselt virtuaalne arvuti kasutab
- usedbyrefreservation näitab kui palju on nö zpool ressursist kinni täna refreservationile, st 82.5 - 55.1 = 27.4
- available näitab kui palju on zpool peal reaalselt kasutada arvestades sinna hulla refreservation all kinni oleva mahu, 186 + 27.4 = 213.4
Kasulikud lisamaterjalid
- TODO
