Konstruktori arvuti - Asrock Taichi X570 + AMD Ryzen 5500

Allikas: Imre kasutab arvutit
Mine navigeerimisribaleMine otsikasti

Sissejuhatus

TODO

Riistvara

Arvutikomplekti komponentide koosseis

  • emaplaat - Asrock Taichi X570
  • protsessor - AMD Ryzen 5500
  • mälu - 4 x 32 GB
  • toiteplokk -
  • videokaart - Nvidia quadro k600 (lspci: 'NVIDIA Corporation GK107GL [Quadro K600]')
  • nvme seade - 2 x 'Samsung SSD 990 PRO with Heatsink 4TB'
  • nvme seade - 2 x 'micron 7450 pro u.3'

pcie nvme adapteri kasutamine

nvme u.3 micron 7450 pro plokkseade, iseloomulikud kontaktid (andmed + toide)

20250317-nvme-u3-micron-7450-pro-01.jpeg

salvestuskomplekt

  • kaks nvme ketast
  • supermicro AOC-SLG4-4E4T pcie nvme adapter
  • ühenduskaablid (toide tuleb tavalise nn molex küljest)

20250317-nvme-u3-micron-7450-pro-02.jpeg

andmekaabli kontaktid, kinnitub adapteri kaardi külge ja ühendab ketta

20250317-nvme-u3-micron-7450-pro-03.jpeg

kus

  • valge lapik kontakt ja tema vastus kaardil on 'slimsas' ehk 'sff-8654 8i' - tavaliselt x8 pcie laiune, st teenindab ära kaks nvme ketast (antud juhul nii kasutatakse)
  • valge lapik kontakt ei ole MCIO (Mini Cool Edge IO) - tavaliselt x8 pcie laiune, st teenindab ära kaks nvme ketast (antud juhul nii kasutatakse)

arvuti bios/setup keskkond, tuleb väljendada, et pci liides töötab bifurcation režiimis - paistab, et antud juhul saab kasutada kahte x4 kanalit ja ka füüsilisi kettaid on parasjagu kaks

20250317-nvme-u3-micron-7450-pro-04.jpeg

muud bifurcation valikud

20250317-nvme-u3-micron-7450-pro-05.jpeg

Sissevaada arvuti korpusse, nö kõige kangemas pesas on videokaart, siis võrgukaart ja kõnealune storage adapter

20250317-nvme-u3-micron-7450-pro-06.jpeg

Tulemusena on arvutis operatsioonisüsteemi jaoks olemas kaks füüsilist ketast

root@pve-svc-01:~# lspci | grep Micron
01:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 PRO NVMe SSD (rev 01)
10:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 PRO NVMe SSD (rev 01)

202604 - Mellanox kaardi lisamise trauma

root@pve-svc-01:~# zpool status
  pool: rpool
 state: DEGRADED
status: One or more devices has been removed by the administrator.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using zpool online' or replace the device with
	'zpool replace'.
  scan: scrub repaired 0B in 00:00:16 with 0 errors on Sun Apr 12 00:24:17 2026
config:

	NAME                                                 STATE     READ WRITE CKSUM
	rpool                                                DEGRADED     0     0     0
	  mirror-0                                           DEGRADED     0     0     0
	    nvme-eui.000000000000000a00a075244cbac2ea-part3  ONLINE       0     0     0
	    nvme-eui.000000000000000200a075254da4607e-part3  REMOVED      0     0     0

errors: No known data errors

  pool: zp_svc
 state: DEGRADED
status: One or more devices has been removed by the administrator.
	Sufficient replicas exist for the pool to continue functioning in a
	degraded state.
action: Online the device using zpool online' or replace the device with
	'zpool replace'.
  scan: scrub repaired 0B in 00:35:07 with 0 errors on Sun Apr 12 00:59:09 2026
config:

	NAME                                                STATE     READ WRITE CKSUM
	zp_svc                                              DEGRADED     0     0     0
	  mirror-0                                          DEGRADED     0     0     0
	    nvme-Micron_7450_MTFDKCC15T3TFR_24484CBAC2EA_2  ONLINE       0     0     0
	    nvme-Micron_7450_MTFDKCC15T3TFR_25024DA4607E_2  REMOVED      0     0     0

errors: No known data errors
root@pve-svc-01:~# lsscsi -s
[N:0:1:1]    disk    Samsung SSD 990 PRO with Heatsink 4TB__1   /dev/nvme0n1  4.00TB
sdev_scandir_sort: left parse failed: nvme1n3
sdev_scandir_sort: left parse failed: nvme1n2
[N:1:0:2]    disk    Micron_7450_MTFDKCC15T3TFR__2              /dev/nvme1n2       -
[N:1:0:3]    disk    Micron_7450_MTFDKCC15T3TFR__3              /dev/nvme1n3       -
[N:1:0:1]    disk    Micron_7450_MTFDKCC15T3TFR__1              /dev/nvme1n1       -
[N:2:1:1]    disk    Samsung SSD 990 PRO with Heatsink 4TB__1   /dev/nvme2n1  4.00TB
sdev_scandir_sort: left parse failed: nvme3n1
sdev_scandir_sort: left parse failed: nvme3n3
[N:3:0:3]    disk    Micron_7450_MTFDKCC15T3TFR__3              /dev/nvme3n3  7.45TB
[N:3:0:1]    disk    Micron_7450_MTFDKCC15T3TFR__1              /dev/nvme3n1   214GB
[N:3:0:2]    disk    Micron_7450_MTFDKCC15T3TFR__2              /dev/nvme3n2  7.69TB
root@pve-svc-01:~# ./check_pcie.sh
Scanning PCIe Bus for active errors...
======================================
DEVICE: 00:01.2 PCI bridge: Advanced Micro Devices, Inc. [AMD] Renoir/Cezanne PCIe GPP Bridge (prog-if 00 [Normal decode])
  [!!!] Physical Lane Issues: 		LaneErrStat: LaneErr at lane: 0 1 2 3
--------------------------------------
DEVICE: 01:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 PRO NVMe SSD (rev 01) (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr+ BadTLP+ BadDLLP+ Rollover- Timeout+ AdvNonFatalErr+
  [!!] Uncorrectable: 		UESta:	DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq+ ACSViol-
  [!!!] Physical Lane Issues: 		LaneErrStat: LaneErr at lane: 0 1 2 3
  [W] Link Speed Downgraded: 8GT/s (Capable: 16GT/s)
--------------------------------------
DEVICE: 05:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
--------------------------------------
DEVICE: 06:00.0 PCI bridge: ASMedia Technology Inc. ASM1184e 4-Port PCIe x1 Gen2 Packet Switch (prog-if 00 [Normal decode])
  [!] Correctable: 		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
--------------------------------------
DEVICE: 08:00.0 Network controller: Intel Corporation Wi-Fi 6 AX200 (rev 1a)
  [!] Correctable: 		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout+ AdvNonFatalErr+
--------------------------------------
DEVICE: 0c:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
  [!] Correctable: 		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr-
  [W] Link Speed Downgraded: 		LnkSta:	Speed 2.5GT/s, Width x1 (Capable: 		LnkCap:	Port #4, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <128ns, L1 <64us)
--------------------------------------
DEVICE: 10:00.0 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
  [!] Correctable: 		CESta:	RxErr- BadTLP+ BadDLLP+ Rollover- Timeout+ AdvNonFatalErr+
  [!!!] Physical Lane Issues: 		LaneErrStat: LaneErr at lane: 0 1 2 3 4 5 6 7
  [W] Link Speed Downgraded: 8GT/s (Capable: 16GT/s)
--------------------------------------
DEVICE: 10:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
  [!] Correctable: 		CESta:	RxErr- BadTLP+ BadDLLP+ Rollover- Timeout+ AdvNonFatalErr+
  [W] Link Speed Downgraded: 8GT/s (Capable: 16GT/s)
--------------------------------------
DEVICE: 11:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+
  [W] Link Speed Downgraded: 8GT/s (Capable: 16GT/s)
--------------------------------------
DEVICE: 12:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 PRO NVMe SSD (rev 01) (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr+ BadTLP+ BadDLLP+ Rollover- Timeout+ AdvNonFatalErr+
  [!!!] Physical Lane Issues: 		LaneErrStat: LaneErr at lane: 0 1 2 3
  [W] Link Speed Downgraded: 8GT/s (Capable: 16GT/s)
--------------------------------------

202605 - ebastabiilsuse parandamine

gemini.google vestluses tekkis hüpotees, et kasutuses olev protsessor Ryzen 5 5500' on üsna tagasihoidlik seade

  • toetab kuni pcie v. 3.0
  • algselt laptop kasutusjuhule orienteeritud
  • mitte-paindlik pcie ressursside esitamises (nt iommu abil grupeerimine)
  • materjalivaliku osas mitte kõige kvaliteetsem (signaalide läbikostvus, müra jms)

ja hüpotees, et protsessor 'Ryzen 9 5900XT' võiks anda palju parema tulemuse

  • toetab kuni pcie v. 4.0
  • pigem serveri laadne, ja downgrade'itud desktop kasutusjuhuks
  • pcie ressursside paindlik korraldamine (nt bifurcation osakonda ilmus juurde mõni valik (nt 4 x4)
  • kvaliteetsemad materjalid, võimsam signaal pcie peal jne

ning tõesti, tulemus on looduses palju parem

root@pve-svc-01:~# ./check_pcie.sh
Scanning PCIe Bus for active errors...
======================================
DEVICE: 00:01.3 PCI bridge: Advanced Micro Devices, Inc. [AMD] Starship/Matisse GPP Bridge (prog-if 00 [Normal decode])
  [!!!] Physical Lane Issues: 		LaneErrStat: LaneErr at lane: 0 1 2 3
--------------------------------------
DEVICE: 04:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 05:00.0 PCI bridge: ASMedia Technology Inc. ASM1184e 4-Port PCIe x1 Gen2 Packet Switch (prog-if 00 [Normal decode])
  [!] Correctable: 		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 08:00.0 Ethernet controller: Intel Corporation I211 Gigabit Network Connection (rev 03)
  [!] Correctable: 		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CorrIntErr- HeaderOF-
  [W] Link Speed Downgraded: 		LnkSta:	Speed 2.5GT/s, Width x1 (Capable: 		LnkCap:	Port #5, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <16us)
--------------------------------------
DEVICE: 0a:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network Connection
  [!] Correctable: 		CESta:	RxErr+ BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CorrIntErr- HeaderOF-
  [W] Link Speed Downgraded: 		LnkSta:	Speed 2.5GT/s, Width x1 (Capable: 		LnkCap:	Port #4, Speed 2.5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <128ns, L1 <64us)
--------------------------------------
DEVICE: 0e:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal] (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 0f:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 PRO NVMe SSD (rev 01) (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 10:00.0 Non-Volatile memory controller: Micron Technology Inc 7450 PRO NVMe SSD (rev 01) (prog-if 02 [NVM Express])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 11:00.0 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 11:00.1 Ethernet controller: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 13:00.0 Non-Essential Instrumentation [1300]: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Reserved SPP
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 13:00.1 Encryption controller: Advanced Micro Devices, Inc. [AMD] Starship/Matisse Cryptographic Coprocessor PSPCPP
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 13:00.3 USB controller: Advanced Micro Devices, Inc. [AMD] Matisse USB 3.0 Host Controller (prog-if 30 [XHCI])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------
DEVICE: 13:00.4 Audio device: Advanced Micro Devices, Inc. [AMD] Starship/Matisse HD Audio Controller (prog-if 00 [HDA compatible])
  [!] Correctable: 		CESta:	RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ CorrIntErr- HeaderOF-
--------------------------------------

kus

  • 8 GT/s downgrade'isid ei ole (nt nvme x4 pcie seadmed töötavad oma loomuliku x4 ehk 16 GT/s kiiruse lane'iga)
  • LaneErr esineb ainult 00:01.3 juures, paistab, et seal on üks seade ja oluline seade, gemini arvates on see lingi kokkuleppimise ühekordne nähtus, eriti arvestades, et süsteem säilitab kiirust 16 GT/s, näis
# lspci -PP -s  0e:00.0
00:01.3/0e:00.0 Non-Volatile memory controller: Samsung Electronics Co Ltd NVMe SSD Controller S4LV008[Pascal]

20260530 - bifurcation märkused

SuperMicro nvme retimer kaardi kasutamist aitab

  • bios juures bifruction valikute vahel katsetamine (4 x4, 2 x8 jne)
  • füüsiliste kaablite ühendamine ringi nt ühe Y kaabli küljes on kaks seadete; kummagi Y kaabli küljes on üks seade
  • pcie kaardi passimine ühte ja teisse pcie füüsilisse pessa (tõenäoliselt on kõige parem kaarti paigutada nn video kaardi pessa)
  • videokaardina võiks kasutada midagi võimalikult tagasihoidlikku, nt pcie v. 2.0 nvidia midagi 56 euro eest; ja nö madalas pcie pesas

Kasulikud lisamaterjalid

  • TODO