VPP - Vector Packet Processing

Allikas: Imre kasutab arvutit
Redaktsioon seisuga 17. august 2024, kell 20:55 kasutajalt Imre (arutelu | kaastöö)
Mine navigeerimisribaleMine otsikasti

Sissejuhatus

TODO

Tööpõhimõte

Väited

  • VPP lahendus on 'data plane'
  • VPP lahendus ei sisalda otseselt control plain osakonda
  • VPP lahendus sisaldab API liidest data plane juhtimiseks - vppctl on teatud mõttes control plane ja kohmakas
  • VPP kasutuskohtadega tegelejad hoolitsevad control plane eest (nt netgate oma toodete puhul, linux-cp plugin abil on võimalik linux host pealt tegeleda)

Paigaldamine - Proxmox virtuaalne arvuti

Väited 

[  162.223607] Lockdown: vpp: raw io port access is restricted; see man kernel_lockdown.7

Virtuaalne arvuti peab toetama 2M suurust hugepagendust ja 1024 tükki, nt sobib sellise seadistusega virtuaalne arvuti 

root@pve-moraal-x570:~# cat /etc/pve/qemu-server/9911.conf 
agent: 1
bios: ovmf
boot: order=virtio0;ide2;net0
cores: 4
cpu: host
efidisk0: sn_srv_btrfs:9911/vm-9911-disk-0.raw,efitype=4m,pre-enrolled-keys=1,size=528K
ide2: none,media=cdrom
machine: q35
memory: 8192
meta: creation-qemu=7.1.0,ctime=1674411406
name: ubuntu-2204-vpp-01
net0: virtio=BC:24:11:9B:C3:E4,bridge=vmbr1,firewall=1,tag=111
numa: 1
ostype: l26
parent: enne-vpp-katset-puhas
scsihw: virtio-scsi-single
smbios1: uuid=174e8d69-5f6e-4296-8bd4-73e7247dae1e
sockets: 1
vga: virtio
virtio0: sn_srv_btrfs:9911/vm-9911-disk-1.raw,iothread=1,size=20G
virtio1: sn_srv_btrfs:9911/vm-9911-disk-2.raw,iothread=1,size=2G
vmgenid: 8de853d2-a9eb-4888-a3c6-b36d13896a96

ning kerneli rida 

root@ubuntu-2204-vpp-01:~/20240813# cat /proc/cmdline 
BOOT_IMAGE=/boot/vmlinuz-5.15.0-118-generic root=/dev/mapper/system-root ro default_hugepagesz=2M hugepagesz=2M hugepages=2048

kus

  • tundub, et numa: 1 ega kerneli hugepages osakond ei ole tegelikult vajalik
  • host cpu tüüp on vajalik

apt repo seadistuse lisamiseks sobib avada https://packagecloud.io/fdio/release ning pressida nupul 'Debian', tulemusena pakutakse kopeerida selline käsk 

# curl -s https://packagecloud.io/install/repositories/fdio/release/script.deb.sh | bash

käsu andmise tulemusena tekitatakse muu hulgas nt selline apt seadistus Ubuntu 22.04 puhul 

# ls -ld /etc/apt/sources.list.d/fdio_release.list /etc/apt/keyrings/fdio_release-archive-keyring.gpg
-rw-r--r-- 1 root root 2823 Aug 12 22:26 /etc/apt/keyrings/fdio_release-archive-keyring.gpg
-rw-r--r-- 1 root root  357 Aug 12 22:26 /etc/apt/sources.list.d/fdio_release.list

# cat /etc/apt/sources.list.d/fdio_release.list                                                   

deb [signed-by=/etc/apt/keyrings/fdio_release-archive-keyring.gpg] https://packagecloud.io/fdio/release/ubuntu/ jammy main
deb-src [signed-by=/etc/apt/keyrings/fdio_release-archive-keyring.gpg] https://packagecloud.io/fdio/release/ubuntu/ jammy mai

vpp tarkvara põhiosa paigaldamiseks sobib öelda 

# apt-get install vpp vpp-plugin-core vpp-plugin-dpdk

..
Setting up vpp (24.06-release) ...
* Applying /etc/sysctl.d/10-console-messages.conf ...
kernel.printk = 4 4 1 7
* Applying /etc/sysctl.d/10-ipv6-privacy.conf ...
net.ipv6.conf.all.use_tempaddr = 2
net.ipv6.conf.default.use_tempaddr = 2
* Applying /etc/sysctl.d/10-kernel-hardening.conf ...
kernel.kptr_restrict = 1
* Applying /etc/sysctl.d/10-magic-sysrq.conf ...
kernel.sysrq = 176
* Applying /etc/sysctl.d/10-network-security.conf ...
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.all.rp_filter = 2
* Applying /etc/sysctl.d/10-ptrace.conf ...
kernel.yama.ptrace_scope = 1
* Applying /etc/sysctl.d/10-zeropage.conf ...
vm.mmap_min_addr = 65536
* Applying /usr/lib/sysctl.d/50-default.conf ...
kernel.core_uses_pid = 1
net.ipv4.conf.default.rp_filter = 2
net.ipv4.conf.default.accept_source_route = 0
sysctl: setting key "net.ipv4.conf.all.accept_source_route": Invalid argument
net.ipv4.conf.default.promote_secondaries = 1
sysctl: setting key "net.ipv4.conf.all.promote_secondaries": Invalid argument
net.ipv4.ping_group_range = 0 2147483647
net.core.default_qdisc = fq_codel
fs.protected_hardlinks = 1
fs.protected_symlinks = 1
fs.protected_regular = 1
fs.protected_fifos = 1
* Applying /usr/lib/sysctl.d/50-pid-max.conf ...
kernel.pid_max = 4194304
* Applying /etc/sysctl.d/80-vpp.conf ...
vm.nr_hugepages = 1024
vm.max_map_count = 3096
vm.hugetlb_shm_group = 0
kernel.shmmax = 2147483648
* Applying /usr/lib/sysctl.d/99-protect-links.conf ...
fs.protected_fifos = 1
fs.protected_hardlinks = 1
fs.protected_regular = 2
fs.protected_symlinks = 1
* Applying /etc/sysctl.d/99-sysctl.conf ...
* Applying /etc/sysctl.conf ...
Created symlink /etc/systemd/system/multi-user.target.wants/vpp.service → /lib/systemd/system/vpp.service.
Setting up vpp-plugin-dpdk (24.06-release) ...
Setting up vpp-plugin-core (24.06-release) ...
...

Lisaks dpdk teemaga tegelemiseks driverctl utiliit 

# apt-get install driverctl

Virtuaalse arvuti operatsioonisüsteemi ettevalmistamiseks, arvutis on kolm virtio tüüpi võrgukaart, kusjuures dpdk sisaldab virtio tuge 

root@ubuntu-2204-vpp-01:~# driverctl list-devices network
0000:06:12.0 virtio-pci
0000:06:13.0 virtio-pci
0000:06:14.0 virtio-pci

root@ubuntu-2204-vpp-01:~# driverctl set-override 0000:06:13.0 vfio-pci
root@ubuntu-2204-vpp-01:~# driverctl set-override 0000:06:14.0 vfio-pci

Lisaks tuleks arvuti käivitada no-iommu režiimis 

root@ubuntu-2204-vpp-01:~# grep iomm /etc/default/grub
GRUB_CMDLINE_LINUX_DEFAULT="vfio.enable_unsafe_noiommu_mode=1"

Tulemusena on arvutis sellised pci seadmed, võrguseadmed 

root@ubuntu-2204-vpp-01:~# lspci -vvv | grep IOMM
	IOMMU group: 1
	IOMMU group: 0

Misc

/etc/vpp/startup.conf fail mõlemas arvutis 

root@ubuntu-2204-vpp-01:~# less /etc/vpp/startup.conf
unix {
  nodaemon
  log /var/log/vpp/vpp.log
  full-coredump
  cli-listen /run/vpp/cli.sock
  gid vpp
  exec /etc/vpp/bootstrap.vpp
}

api-trace { on }
api-segment { gid vpp }
socksvr { default }

cpu {
        main-core 0
        corelist-workers 1-3
}

plugins {
        plugin default { disable }
        plugin dpdk_plugin.so { enable }
        plugin acl_plugin.so { enable }
        plugin linux_cp_plugin.so { enable }
        plugin linux_nl_plugin.so { enable }
}

logging {
   default-log-level debug
   default-syslog-log-level info
}

linux-cp { default netns dataplane }

bootstrap fail ühes ja teises arvutis 

root@ubuntu-2204-vpp-01:~# cat /etc/vpp/bootstrap.vpp 
# set logging class linux-cp rate-limit 1000 level warn syslog-level notice

lcp default netns dataplane
lcp lcp-sync on
lcp lcp-auto-subint on

create loopback interface instance 0
lcp create loop0 host-if loop0
set interface state loop0 up
set interface ip address loop0 10.1.163.131/32
set interface ip address loop0 2999:678:d78::131/128

lcp create GigabitEthernet6/13/0 host-if e1-0
lcp create GigabitEthernet6/14/0 host-if e1-1

set interface mtu packet 1500 GigabitEthernet6/13/0
set interface ip address GigabitEthernet6/13/0 10.6.13.131/24
set interface state GigabitEthernet6/13/0 up

Teine arvuti 

root@ubuntu-2204-vpp-02:~# less /etc/vpp/bootstrap.vpp 
# set logging class linux-cp rate-limit 1000 level warn syslog-level notice

lcp default netns dataplane
lcp lcp-sync on
lcp lcp-auto-subint on

create loopback interface instance 0
lcp create loop0 host-if loop0
set interface state loop0 up
set interface ip address loop0 10.1.163.132/32
set interface ip address loop0 2999:678:d78::132/128

lcp create GigabitEthernet6/13/0 host-if e1-0
lcp create GigabitEthernet6/14/0 host-if e1-1

set interface mtu packet 1500 GigabitEthernet6/13/0
set interface ip address GigabitEthernet6/13/0 10.6.13.132/24
set interface state GigabitEthernet6/13/0 up

Tulemuse esitamine 

root@ubuntu-2204-vpp-01:~# ip netns exec dataplane ip link show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
21: loop0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 9000 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether de:ad:00:00:00:00 brd ff:ff:ff:ff:ff:ff
22: e1-0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
    link/ether bc:24:11:9b:c3:e5 brd ff:ff:ff:ff:ff:ff
23: e1-1: <BROADCAST,MULTICAST> mtu 9000 qdisc mq state DOWN mode DEFAULT group default qlen 1000
    link/ether bc:24:11:9b:c3:e6 brd ff:ff:ff:ff:ff:ff

Operatsioonisüsteemi poolelt paistab koormus selline

20240817-vpp-01.png

kus

  • TODO

acl

vpp# set acl-plugin acl permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 sport 100 dport 1

vpp# show acl-plugin acl                                                                             
acl-index 1 count 1 tag {cli}
          0: ipv4 permit+reflect src 192.81.1.1/32 dst 192.82.1.1/32 proto 17 sport 100 dport 1

vpp# delete acl-plugin acl index 1
Deleted ACL index:1

Misc

Võrguliidese ip seadistus 

vpp# show interface addr GigabitEthernet6/13/0
GigabitEthernet6/13/0 (up):
  L3 10.6.13.133/24

vpp# set interface ip address GigabitEthernet6/13/0 10.6.13.134/24
vpp# set interface ip address GigabitEthernet6/13/0 10.6.13.135/24

vpp# set interface ip address del GigabitEthernet6/13/0 10.6.13.135/24

Ruuting 

vpp# ip route add 10.100.111.0/24 via 10.6.13.135
vpp# ip route del 10.100.111.0/24 via 10.6.13.135
vpp# show ip fib

Ruutingute automaatne jõudmine nö linuxist vpp osakonda 

root@ubuntu-2204-vpp-01:~# for j in `seq 1 200`; do for i in `seq 1 200`; do ip netns exec dataplane ip route add 10.$j.$i.0/24 via 10.6.13.$i; done; done

root@ubuntu-2204-vpp-01:~# for j in `seq 1 200`; do for i in `seq 1 200`; do ip netns exec dataplane ip route del 10.$j.$i.0/24 via 10.6.13.$i; done; done

Tulemust on näha 

root@ubuntu-2204-vpp-01:~# vppctl show ip fib | grep ^1 | nl
     1	10.6.13.0/32
     2	10.6.13.0/24
     3	10.6.13.131/32
     4	10.6.13.132/32
     5	10.6.13.134/32
     6	10.6.13.255/32
     7	194.1.163.35/32
     ...

Probleemid

  • Millegipärast ruutingud jõuavad automaatselt linux juurest vpp juurde, aga mitte vastupidi.

Mõisted

  • SPP - Scalar Packet Processing
  • FD.io - Fast Data input-output

Kasulikud lisamaterjalid

Pärit leheküljelt "https://www.auul.pri.ee/wiki/index.php?title=VPP_-_Vector_Packet_Processing&oldid=1323"