Linux plokkseadme kasutamine

Sissejuhatus

TODO

Tööpõhimõte

TODO

Mõõtmine - üksik sata-ssd ketas=

fio seadistusfail

root@rae-cf-04:~# cat fio.fio 
[global]
ioengine=libaio
invalidate=1
ramp_time=5
runtime=180
time_based
direct=1
group_reporting=1
per_job_logs=0
bs=8k
readwrite=randrw
size=40g
iodepth=32
end_fsync=1
# gtod_reduce=1
  
[test1-subtest1-blocksize32k]
filename=/mnt/sde/fio.img
numjobs=12

kus

• TODO

ning käivitamisel paistab

root@rae-cf-04:~# iostat -m -x 30 /dev/sde | grep ^sde
Device            r/s     rMB/s   rrqm/s  %rrqm r_await rareq-sz     w/s     wMB/s   wrqm/s  %wrqm w_await wareq-sz     d/s     dMB/s   drqm/s  %drqm d_await dareq-sz     f/s f_await  aqu-sz  %util
..
sde               6,87      0,18     0,72   9,51    3,58    27,08    15,48      0,61    13,74  47,02    2,18    40,20    0,01      9,24     0,00   0,00   16,31 1048569,46  0,02    0,22    0,06   0,23
sde           19900,50    155,47     0,07   0,00    1,51     8,00 19901,57    155,48     0,33   0,00    1,55     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,40    1,67   60,91 100,00
sde           19831,50    154,93     0,03   0,00    1,52     8,00 19854,03    155,11     0,30   0,00    1,55     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,40    1,83   60,91 100,00
sde           19761,47    154,39     0,10   0,00    1,52     8,00 19830,37    154,93     0,50   0,00    1,56     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,40    2,50   60,91 100,00
sde           19995,50    156,22     0,03   0,00    1,51     8,00 19992,33    156,19     0,50   0,00    1,54     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,40    2,08   60,91 100,00
sde           19697,80    153,89     0,10   0,00    1,52     8,00 19723,87    154,09     0,33   0,00    1,57     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,40    1,92   60,91 100,00
sde           19959,23    155,93     0,07   0,00    1,51     8,00 19939,47    155,78     0,43   0,00    1,54     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,40    1,67   60,91 100,00
sde             553,23      4,32     0,00   0,00    1,54     8,00   557,47      4,35     0,07   0,01    1,67     8,00    0,00      0,00     0,00   0,00    0,00     0,00    0,13    0,75    1,78   2,95
sde               0,17      0,00     0,00   0,00    1,00     0,30     0,00      0,00     0,00   0,00    0,00     0,00    0,00      0,00     0,00   0,00    0,00     0,00    0,00    0,00    0,00   0,03
...

kus

• arvulised väärtused väljendavad linuxi kernelist füüsilise plokkseadme suunal toimuvat, nt wareq-sz (write request size) väljendab, et toimub 8 kBait suuruse plokiga kirjutamine
• wareq-sz ja rareq-sz - kasutatava ploki suurus
• w/s ja r/s - operatsioonide arv sekundis (tuntud kui iops)
• wrqm/s ja rrqm/s - write ja read request merges sekundis - antud juhul seda ei toimu praktiliselt, aga teatud tingimuste puhul linux kernel käigult agregeerib kokku sisse tulevaid kirjutamisi suuremateks komplektideks, ja saadab need korraga füüsilisele plokkseadmele
• w_await ja r_await - kirjutamise ja lugemisega seotud viivutus - üldiselt loetakse normaalseks alla 4 ms
• wMB/s ja rMB/s - mahu kirjutamise ja lugemise kiirus

ning fio tulemus

root@rae-cf-04:~# fio fio.fio 
test1-subtest1-blocksize32k: (g=0): rw=randrw, bs=(R) 8192B-8192B, (W) 8192B-8192B, (T) 8192B-8192B, ioengine=libaio, iodepth=32
...
fio-3.28
Starting 12 processes
Jobs: 12 (f=12): [F(1),m(1),F(3),m(1),F(1),m(5)][11.6%][r=149MiB/s,w=149MiB/s][r=19.0k,w=19.1k IOPS][eta 23m:36s]
test1-subtest1-blocksize32k: (groupid=0, jobs=12): err= 0: pid=3266168: Sat Feb  1 23:15:16 2025
  read: IOPS=19.9k, BW=155MiB/s (163MB/s)(27.3GiB/180007msec)
    slat (usec): min=2, max=19609, avg=298.37, stdev=637.42
    clat (usec): min=363, max=51174, avg=9379.33, stdev=3656.91
     lat (usec): min=542, max=52640, avg=9677.78, stdev=3755.98
    clat percentiles (usec):
     |  1.00th=[ 3556],  5.00th=[ 4752], 10.00th=[ 5473], 20.00th=[ 6456],
     | 30.00th=[ 7242], 40.00th=[ 8029], 50.00th=[ 8717], 60.00th=[ 9503],
     | 70.00th=[10552], 80.00th=[11863], 90.00th=[13960], 95.00th=[16188],
     | 99.00th=[21627], 99.50th=[23987], 99.90th=[29230], 99.95th=[31327],
     | 99.99th=[35914]
   bw (  KiB/s): min=126053, max=189343, per=100.00%, avg=158884.11, stdev=918.85, samples=4309
   iops        : min=15756, max=23667, avg=19860.25, stdev=114.85, samples=4309
  write: IOPS=19.9k, BW=155MiB/s (163MB/s)(27.3GiB/180007msec); 0 zone resets
    slat (usec): min=2, max=18651, avg=302.93, stdev=641.72
    clat (usec): min=380, max=52645, avg=9353.20, stdev=3745.15
     lat (usec): min=384, max=53573, avg=9656.22, stdev=3846.76
    clat percentiles (usec):
     |  1.00th=[ 3261],  5.00th=[ 4555], 10.00th=[ 5342], 20.00th=[ 6390],
     | 30.00th=[ 7177], 40.00th=[ 7963], 50.00th=[ 8717], 60.00th=[ 9503],
     | 70.00th=[10552], 80.00th=[11863], 90.00th=[14091], 95.00th=[16319],
     | 99.00th=[21890], 99.50th=[23987], 99.90th=[29230], 99.95th=[31327],
     | 99.99th=[36439]
   bw (  KiB/s): min=126768, max=191136, per=100.00%, avg=159008.05, stdev=903.82, samples=4309
   iops        : min=15846, max=23892, avg=19875.76, stdev=112.97, samples=4309
  lat (usec)   : 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.06%, 4=2.24%, 10=62.52%, 20=33.52%, 50=1.67%
  lat (msec)   : 100=0.01%
  cpu          : usr=0.59%, sys=1.62%, ctx=2589547, majf=0, minf=699
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.0%, 32=100.0%, >=64=0.0%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.1%, 64=0.0%, >=64=0.0%
     issued rwts: total=3573369,3576278,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=32

Run status group 0 (all jobs):
   READ: bw=155MiB/s (163MB/s), 155MiB/s-155MiB/s (163MB/s-163MB/s), io=27.3GiB (29.3GB), run=180007-180007msec
  WRITE: bw=155MiB/s (163MB/s), 155MiB/s-155MiB/s (163MB/s-163MB/s), io=27.3GiB (29.3GB), run=180007-180007msec

Disk stats (read/write):
  sde: ios=3674207/3677588, merge=13/73, ticks=5564118/5703731, in_queue=11267990, util=99.97%

kus

• TODO

Kasulikud lisamaterjalid

• 'Chapter 32. Factors affecting I/O and file system performance' - https://docs.redhat.com/en/documentation/red_hat_enterprise_linux/9/html/monitoring_and_managing_system_status_and_performance/factors-affecting-i-o-and-file-system-performance_monitoring-and-managing-system-status-and-performance#tools-for-monitoring-and-diagnosing-i-o-and-file-system-issues_factors-affecting-i-o-and-file-system-performance
• TODO