Proxmox v. 9 kasutamine

Sissejuhatus

TODO

Virtuaalne riistvara

Klaviatuur ja hiir

root@pwrk-02:~# systool -b serio
Bus = "serio"

  Device = "serio0"
  Device = "serio1"

root@pwrk-02:~# systool -b serio -v
Bus = "serio"

  Device = "serio0"
  Device path = "/sys/devices/platform/i8042/serio0"
    bind_mode           = "auto"
    description         = "i8042 KBD port"
    drvctl              = <store method only>
    err_count           = "0"
    extra               = "0"
    firmware_id         = "PNP: PNP0303"
    force_release       = "369-370"
    modalias            = "serio:ty06pr00id00ex00"
    scroll              = "0"
    set                 = "2"
    softraw             = "1"
    softrepeat          = "0"
    uevent              = "DRIVER=atkbd
SERIO_TYPE=06
SERIO_PROTO=00
SERIO_ID=00
SERIO_EXTRA=00
MODALIAS=serio:ty06pr00id00ex00
SERIO_FIRMWARE_ID=PNP: PNP0303"

  Device = "serio1"
  Device path = "/sys/devices/platform/i8042/serio1"
    bind_mode           = "auto"
    description         = "i8042 AUX port"
    drvctl              = <store method only>
    firmware_id         = "PNP: PNP0f13"
    modalias            = "serio:ty01pr00id00ex00"
    protocol            = "VirtualPS/2"
    rate                = "100"
    resetafter          = "5"
    resolution          = "200"
    resync_time         = "0"
    uevent              = "DRIVER=psmouse
SERIO_TYPE=01
SERIO_PROTO=00
SERIO_ID=00
SERIO_EXTRA=00
MODALIAS=serio:ty01pr00id00ex00
SERIO_FIRMWARE_ID=PNP: PNP0f13"

remote zfs over iscsi

Mõisted

• FUA (Forced Unit Access) -
• WCE (Write Cache Enable) -
• TPU (Thin Provisioning UNMAP) -
• TPWS (WRITE_SAME / Zeroing) -
• VFS (virtual file system) -

Tööpõhimõte

Virtuaalse arvuti vaatest andmetega tegelemine

pve virtuaalne arvuti -> pve füüsiline arvuti -> over-iscsi-zfs-storage-server

Sõltuvalt qemu virtuaalse arvuti virtuaalse plokkseadme 'Cache' seadistustest on võimalik erinevate cache komplektide-kombinatsioonide kasutamine andmete liikumisel virtuaalses arvutis töötava protsessi juurest zfs storage serveri füüsilisele plokkseadmele. Tundub, et 2025 aastal on kõige keskmisemale kasutusele sobiv valik 'Cache: nocache'

• hea jõudlus
• hea terviklus

Andmete liikumise teekond

• virtuaalse arvuti rakendus
• virtuaalse arvuti vfs failisüsteem
• virtuaalse arvuti page cache (vfs'ga seotud)
• virtuaalse arvuti ext4 failisüsteem (ext4 draiver)
• virtuaalse arvuti lvm
• virtuaalse arvuti fdisk partitsioonid
• virtuaalse arvuti virtuaalne sata vms kontroller
• füüsilise arvuti qemu protsess
• füüsilise arvuti open-iscsi poolt teostatud /dev/sda scsi plokkseade (puudub cache)
• füüsilises arvutis storage'ga seotud cache puudub
• storage arvuti targetcli (puutub cache)
• storage arvuti zfs ressurss (sisaldab zfs cache)
• storage avuti füüsilise plokkseadme controller-cache

'Cache: nocache' parameeter on üks vähestest parameetritest, mis avaldab mõju nö mõlemas suunas

• virtuaalse arvuti seest kasutada olevale storage ressursile
• kuidas füüsilise arvuti sees virtuaalsele arvutile vastav qemu protsess kasutab allolevat plokkseadet

Käesolevas tekstis kirjeldatakse nelja komponendi seadistamist

• virtuaalsele arvutile vastav qemu protsess
• iscsi initiator
• iscsi target
• zfs lahendus

zfs lahendus iscsi target arvutis

Füüsilised kettad on kasutusel nö kõige tavalisemal viisil, eriti tähendab see, et ketastel on sisselülitatud nö tavaline controller-cache

root@pve-svc-02:~# lsscsi -s | grep 4.00T
[2:0:0:0]    disk    ATA      CT4000MX500SSD1  045   /dev/sdc   4.00TB
[3:0:0:0]    disk    ATA      CT4000MX500SSD1  045   /dev/sdd   4.00TB
[N:0:1:1]    disk    Samsung SSD 990 PRO with Heatsink 4TB__1   /dev/nvme0n1  4.00TB
[N:1:1:1]    disk    Samsung SSD 990 PRO with Heatsink 4TB__1   /dev/nvme1n1  4.00TB

zfs lülituse moodustamine

root@pve-svc-02:~# cat create-zpool-raidz1-x4.sh
zpool create -o ashift=13 zp_data raidz1 \
  /dev/disk/by-id/ata-CT4000MX500SSD1_2246E686FE58 \
  /dev/disk/by-id/ata-CT4000MX500SSD1_2246E686FF7B \
  /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_with_Heatsink_4TB_S7DSNJ0X501827B \
  /dev/disk/by-id/nvme-Samsung_SSD_990_PRO_with_Heatsink_4TB_S7DSNJ0X501856Z

zfs cache töötab tavalisel viisil, st on sisse lülitatud olekus, nt

root@pve-svc-02:~# zfs get all | grep -i cache | grep vm-106-disk-0
zp_crucial_mx_4/vm-106-disk-0     primarycache          all                     default
zp_crucial_mx_4/vm-106-disk-0     secondarycache        all                     default

targetcli iscsi target arvutis

targetcli seadistamine koosneb kahest tegevusest

• zfs põhise alus-storage storage publitseerimine
• publitseerimisel sobivalt scsi protokolli parameetrite kasutamine (eriti cache, sparse jms esitamine)

/> /iscsi create iqn.2025-10.moraal.srv:storage.zfstarget
/> /iscsi/iqn.2025-10.moraal.srv:storage.zfstarget/tpg1/acls  create iqn.1993-08.org.debian:01:4cbe32bd26b

Sobiv seadistus, mis tuleb plokk-ressursi jaoks kehtestada

root@pve-svc-02:~# targetcli /backstores/block/zp_crucial_mx_4-vm-108613-disk-4 get attribute | grep =
======================
alua_support=1
block_size=512
emulate_3pc=1
emulate_caw=1
emulate_dpo=1
emulate_fua_read=1
emulate_fua_write=1
emulate_model_alias=1
emulate_pr=1
emulate_rest_reord=0
emulate_rsoc=1
emulate_tas=1
emulate_tpu=0
emulate_tpws=0
emulate_ua_intlck_ctrl=0
emulate_write_cache=0
enforce_pr_isids=1
force_pr_aptpl=0
hw_block_size=512 [ro]
hw_max_sectors=32768 [ro]
hw_pi_prot_type=0 [ro]
hw_queue_depth=128 [ro]
is_nonrot=1
max_unmap_block_desc_count=1
max_unmap_lba_count=131072
max_write_same_len=65535
optimal_sectors=32768
pgr_support=1
pi_prot_format=0
pi_prot_type=0
pi_prot_verify=0
queue_depth=128
submit_type=0
unmap_granularity=8
unmap_granularity_alignment=0
unmap_zeroes_data=0

kus

• TODO

Muudatuse tegemine

/backstores/b...108613-disk-4> set attribute emulate_tpws=0
/backstores/b...108613-disk-4> set attribute emulate_tpu=0

Kasutamine paistab välja selline

root@pve-svc-02:~# targetcli
targetcli shell version 2.1.53
Copyright 2011-2013 by Datera, Inc and others.
For help on commands, type 'help'.

/> ls
o- / ......................................................................................................................... [...]
  o- backstores .............................................................................................................. [...]
  | o- block .................................................................................................. [Storage Objects: 3]
  | | o- zp_crucial_mx_4-vm-100-disk-0 ......................... [/dev/zp_crucial_mx_4/vm-100-disk-0 (20.0GiB) write-thru activated]
  | | | o- alua ................................................................................................... [ALUA Groups: 1]
  | | |   o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
  | | o- zp_crucial_mx_4-vm-106-disk-5 ......................... [/dev/zp_crucial_mx_4/vm-106-disk-5 (16.0GiB) write-thru activated]
  | | | o- alua ................................................................................................... [ALUA Groups: 1]
  | | |   o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
  | | o- zp_crucial_mx_4-vm-108613-disk-1 .................... [/dev/zp_crucial_mx_4/vm-108613-disk-1 (4.0GiB) write-thru activated]
  | |   o- alua ................................................................................................... [ALUA Groups: 1]
  | |     o- default_tg_pt_gp ....................................................................... [ALUA state: Active/optimized]
  | o- fileio ................................................................................................. [Storage Objects: 0]
  | o- pscsi .................................................................................................. [Storage Objects: 0]
  | o- ramdisk ................................................................................................ [Storage Objects: 0]
  o- iscsi ............................................................................................................ [Targets: 1]
  | o- iqn.2025-10.moraal.srv:storage.zfstarget .......................................................................... [TPGs: 1]
  |   o- tpg1 ............................................................................................... [no-gen-acls, no-auth]
  |     o- acls .......................................................................................................... [ACLs: 1]
  |     | o- iqn.1993-08.org.debian:01:4cbe32bd26b ................................................................ [Mapped LUNs: 3]
  |     |   o- mapped_lun0 ...................................................... [lun0 block/zp_crucial_mx_4-vm-108613-disk-1 (rw)]
  |     |   o- mapped_lun1 ......................................................... [lun1 block/zp_crucial_mx_4-vm-106-disk-5 (rw)]
  |     |   o- mapped_lun2 ......................................................... [lun2 block/zp_crucial_mx_4-vm-100-disk-0 (rw)]
  |     o- luns .......................................................................................................... [LUNs: 3]
  |     | o- lun0 .............. [block/zp_crucial_mx_4-vm-108613-disk-1 (/dev/zp_crucial_mx_4/vm-108613-disk-1) (default_tg_pt_gp)]
  |     | o- lun1 .................... [block/zp_crucial_mx_4-vm-106-disk-5 (/dev/zp_crucial_mx_4/vm-106-disk-5) (default_tg_pt_gp)]
  |     | o- lun2 .................... [block/zp_crucial_mx_4-vm-100-disk-0 (/dev/zp_crucial_mx_4/vm-100-disk-0) (default_tg_pt_gp)]
  |     o- portals .................................................................................................... [Portals: 1]
  |       o- 0.0.0.0:3260 ..................................................................................................... [OK]
  o- loopback ......................................................................................................... [Targets: 0]
  o- srpt ............................................................................................................. [Targets: 0]
  o- vhost ............................................................................................................ [Targets: 0]
  o- xen-pvscsi ....................................................................................................... [Targets: 0]
/>

iscsi initiator

TODO

qemu protsess

Kasutamine

Kasulikud lisamaterjalid

TODO

Kasulikud lisamaterjalid

• TODO