I build my own lab with a vmware server and 10 vmxs (version 17.2) running. All vfps are running in lite-mode and if I look at the needed cpu ressources I noticed the following:
Now I tried to setup another server, this one is running kvm. I've tested gns3, eve-ng and a ubuntu with the vmx images and original juniper libvirt scripts for kvm and all running in lite mode and all have the same issue.
The qemu instance of the vfp is always using about 150% of my cpu, which is approximately 3Ghz. So the lite mode is using more ressources in kvm than in vmware, is this expected?
I've already asked this in the eve-ng forum (attached on the bottom) in the past, and though problem was about nested virtualisation, but now I did this on a bare-metal server and its the same.
I've read many thing about dpdk and it looks like that vfp is not running in the dpdk interrupt mode, but I didn't found any way to verify it or to change it. show chassis hardware shows that I'm running in lite-mode.
Any help is appreciated. kind regards
I'm certain this is not an eve issue, but I need some data to compare. And hope that someone has enough knowledge of qemu and juniper images.
I'm running Vmware 6.5 with a Intel(R) Xeon(R) CPU E5-2620 v3 @ 2.40GHz with 6 Cores and HT = 12 Cores and 128GB RAM.
Before I used eve, I installed all virtual machines (csr, juniper, etc.) directly in VMware, which worked pretty well.
But now I'ld like to migrate all vms to eve, just to gain a litte bit more flexibility.
So I installed the ova of eve and assign all cpu and half of RAM to the VM. Then I tried to run just a single juniper vcp and vfp instance. Running these works fine, but the cpu load of the vfp vm after connecting to the vcp is much higher compared to vmware. Both have just a basic config, nothing with load. Tried different configs, but didn't change anything.
I know that this is a nested virtualization and can't be as fast as only on layer of virtualization, but I think this is too much loss.
Before I'm getting to technical for some people I've just a question how much cpu usage they have running juniper vms?
Since with this high cpu I'm not able to run all my 10 juniper routers which worked fine with plain vmware.
I also tried gns3 and have the same issue, so I believe it's not an eve issue, I'm not certain if this is something with the juniper image or something with qemu on my setup. Thats why I post it here to hope someone can share their usage.
For people with some technical background, please look at my Test results and If you know anything that can help, please let me know.
Test results:
on plain vmware:
vcp = uses 23Mhz in vmware
vfp = uses 767 Mhz in vmware
on eve nested in vmware only running 1 juniper device (1 vcp/1vfp)
eve use 3.96 Ghz
in top under eve:
After some investigation I found out that on the vfp some software interrupts steal my cpu compared to vmware.
top from the vfp:
Code: Select all
/opt/unetlab/wrappers/qemu_wrapper -T 0 -D 1 -t vMX-VCP -F /opt/qemu-2.6.2/bin/qemu-system-x86_64 -d 0 -- -nographic -device e1000,netdev=net0,mac=50:00:00:01:00:00 -netdev tap,id=net0,ifname=vunl0_1_0,script=no -device e1000,netdev=net1,mac=50:00:00:01:00:01 -netdev tap,id=net1,ifname=vunl0_1_1,script=no -smp 1 -m 2048 -name vMX-VCP -uuid 468dd6d8-80b6-4d72-a48a-8b82e4683775 -hda hda.qcow2 -hdb hdb.qcow2 -hdc hdc.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
6879 ? S 0:00 \_ /opt/unetlab/wrappers/qemu_wrapper -T 0 -D 1 -t vMX-VCP -F /opt/qemu-2.6.2/bin/qemu-system-x86_64 -d 0 -- -nographic -device e1000,netdev=net0,mac=50:00:00:01:00:00 -netdev tap,id=net0,ifname=vunl0_1_0,script=no -device e1000,netdev=net1,mac=50:00:00:01:00:01 -netdev tap,id=net1,ifname=vunl0_1_1,script=no -smp 1 -m 2048 -name vMX-VCP -uuid 468dd6d8-80b6-4d72-a48a-8b82e4683775 -hda hda.qcow2 -hdb hdb.qcow2 -hdc hdc.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
6880 ? S 0:00 \_ sh -c /opt/qemu-2.6.2/bin/qemu-system-x86_64 -nographic -device e1000,netdev=net0,mac=50:00:00:01:00:00 -netdev tap,id=net0,ifname=vunl0_1_0,script=no -device e1000,netdev=net1,mac=50:00:00:01:00:01 -netdev tap,id=net1,ifname=vunl0_1_1,script=no -smp 1 -m 2048 -name vMX-VCP -uuid 468dd6d8-80b6-4d72-a48a-8b82e4683775 -hda hda.qcow2 -hdb hdb.qcow2 -hdc hdc.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
6881 ? Sl 15:16 \_ /opt/qemu-2.6.2/bin/qemu-system-x86_64 -nographic -device e1000,netdev=net0,mac=50:00:00:01:00:00 -netdev tap,id=net0,ifname=vunl0_1_0,script=no -device e1000,netdev=net1,mac=50:00:00:01:00:01 -netdev tap,id=net1,ifname=vunl0_1_1,script=no -smp 1 -m 2048 -name vMX-VCP -uuid 468dd6d8-80b6-4d72-a48a-8b82e4683775 -hda hda.qcow2 -hdb hdb.qcow2 -hdc hdc.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
12893 ? S 0:00 /opt/unetlab/wrappers/qemu_wrapper -T 0 -D 2 -t vMX-VFP -F /opt/qemu-2.9.0/bin/qemu-system-x86_64 -d 0 -- -nographic -device virtio-net-pci,netdev=net0,mac=50:00:00:02:00:00 -netdev tap,id=net0,ifname=vunl0_2_0,script=no -device virtio-net-pci,netdev=net1,mac=50:00:00:02:00:01 -netdev tap,id=net1,ifname=vunl0_2_1,script=no -device virtio-net-pci,netdev=net2,mac=50:00:00:02:00:02 -netdev tap,id=net2,ifname=vunl0_2_2,script=no -device virtio-net-pci,netdev=net3,mac=50:00:00:02:00:03 -netdev tap,id=net3,ifname=vunl0_2_3,script=no -device virtio-net-pci,netdev=net4,mac=50:00:00:02:00:04 -netdev tap,id=net4,ifname=vunl0_2_4,script=no -device virtio-net-pci,netdev=net5,mac=50:00:00:02:00:05 -netdev tap,id=net5,ifname=vunl0_2_5,script=no -device virtio-net-pci,netdev=net6,mac=50:00:00:02:00:06 -netdev tap,id=net6,ifname=vunl0_2_6,script=no -device virtio-net-pci,netdev=net7,mac=50:00:00:02:00:07 -netdev tap,id=net7,ifname=vunl0_2_7,script=no -device virtio-net-pci,netdev=net8,mac=50:00:00:02:00:08 -netdev tap,id=net8,ifname=vunl0_2_8,script=no -device virtio-net-pci,netdev=net9,mac=50:00:00:02:00:09 -netdev tap,id=net9,ifname=vunl0_2_9,script=no -device virtio-net-pci,netdev=net10,mac=50:00:00:02:00:0a -netdev tap,id=net10,ifname=vunl0_2_10,script=no -device virtio-net-pci,netdev=net11,mac=50:00:00:02:00:0b -netdev tap,id=net11,ifname=vunl0_2_11,script=no -smp 3 -m 4096 -name vMX-VFP -uuid 95375231-3628-47d3-87dc-87e2fd3b5720 -hda hda.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
12894 ? S 0:00 \_ /opt/unetlab/wrappers/qemu_wrapper -T 0 -D 2 -t vMX-VFP -F /opt/qemu-2.9.0/bin/qemu-system-x86_64 -d 0 -- -nographic -device virtio-net-pci,netdev=net0,mac=50:00:00:02:00:00 -netdev tap,id=net0,ifname=vunl0_2_0,script=no -device virtio-net-pci,netdev=net1,mac=50:00:00:02:00:01 -netdev tap,id=net1,ifname=vunl0_2_1,script=no -device virtio-net-pci,netdev=net2,mac=50:00:00:02:00:02 -netdev tap,id=net2,ifname=vunl0_2_2,script=no -device virtio-net-pci,netdev=net3,mac=50:00:00:02:00:03 -netdev tap,id=net3,ifname=vunl0_2_3,script=no -device virtio-net-pci,netdev=net4,mac=50:00:00:02:00:04 -netdev tap,id=net4,ifname=vunl0_2_4,script=no -device virtio-net-pci,netdev=net5,mac=50:00:00:02:00:05 -netdev tap,id=net5,ifname=vunl0_2_5,script=no -device virtio-net-pci,netdev=net6,mac=50:00:00:02:00:06 -netdev tap,id=net6,ifname=vunl0_2_6,script=no -device virtio-net-pci,netdev=net7,mac=50:00:00:02:00:07 -netdev tap,id=net7,ifname=vunl0_2_7,script=no -device virtio-net-pci,netdev=net8,mac=50:00:00:02:00:08 -netdev tap,id=net8,ifname=vunl0_2_8,script=no -device virtio-net-pci,netdev=net9,mac=50:00:00:02:00:09 -netdev tap,id=net9,ifname=vunl0_2_9,script=no -device virtio-net-pci,netdev=net10,mac=50:00:00:02:00:0a -netdev tap,id=net10,ifname=vunl0_2_10,script=no -device virtio-net-pci,netdev=net11,mac=50:00:00:02:00:0b -netdev tap,id=net11,ifname=vunl0_2_11,script=no -smp 3 -m 4096 -name vMX-VFP -uuid 95375231-3628-47d3-87dc-87e2fd3b5720 -hda hda.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
12895 ? S 0:01 \_ sh -c /opt/qemu-2.9.0/bin/qemu-system-x86_64 -nographic -device virtio-net-pci,netdev=net0,mac=50:00:00:02:00:00 -netdev tap,id=net0,ifname=vunl0_2_0,script=no -device virtio-net-pci,netdev=net1,mac=50:00:00:02:00:01 -netdev tap,id=net1,ifname=vunl0_2_1,script=no -device virtio-net-pci,netdev=net2,mac=50:00:00:02:00:02 -netdev tap,id=net2,ifname=vunl0_2_2,script=no -device virtio-net-pci,netdev=net3,mac=50:00:00:02:00:03 -netdev tap,id=net3,ifname=vunl0_2_3,script=no -device virtio-net-pci,netdev=net4,mac=50:00:00:02:00:04 -netdev tap,id=net4,ifname=vunl0_2_4,script=no -device virtio-net-pci,netdev=net5,mac=50:00:00:02:00:05 -netdev tap,id=net5,ifname=vunl0_2_5,script=no -device virtio-net-pci,netdev=net6,mac=50:00:00:02:00:06 -netdev tap,id=net6,ifname=vunl0_2_6,script=no -device virtio-net-pci,netdev=net7,mac=50:00:00:02:00:07 -netdev tap,id=net7,ifname=vunl0_2_7,script=no -device virtio-net-pci,netdev=net8,mac=50:00:00:02:00:08 -netdev tap,id=net8,ifname=vunl0_2_8,script=no -device virtio-net-pci,netdev=net9,mac=50:00:00:02:00:09 -netdev tap,id=net9,ifname=vunl0_2_9,script=no -device virtio-net-pci,netdev=net10,mac=50:00:00:02:00:0a -netdev tap,id=net10,ifname=vunl0_2_10,script=no -device virtio-net-pci,netdev=net11,mac=50:00:00:02:00:0b -netdev tap,id=net11,ifname=vunl0_2_11,script=no -smp 3 -m 4096 -name vMX-VFP -uuid 95375231-3628-47d3-87dc-87e2fd3b5720 -hda hda.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
12896 ? Sl 979:44 \_ /opt/qemu-2.9.0/bin/qemu-system-x86_64 -nographic -device virtio-net-pci,netdev=net0,mac=50:00:00:02:00:00 -netdev tap,id=net0,ifname=vunl0_2_0,script=no -device virtio-net-pci,netdev=net1,mac=50:00:00:02:00:01 -netdev tap,id=net1,ifname=vunl0_2_1,script=no -device virtio-net-pci,netdev=net2,mac=50:00:00:02:00:02 -netdev tap,id=net2,ifname=vunl0_2_2,script=no -device virtio-net-pci,netdev=net3,mac=50:00:00:02:00:03 -netdev tap,id=net3,ifname=vunl0_2_3,script=no -device virtio-net-pci,netdev=net4,mac=50:00:00:02:00:04 -netdev tap,id=net4,ifname=vunl0_2_4,script=no -device virtio-net-pci,netdev=net5,mac=50:00:00:02:00:05 -netdev tap,id=net5,ifname=vunl0_2_5,script=no -device virtio-net-pci,netdev=net6,mac=50:00:00:02:00:06 -netdev tap,id=net6,ifname=vunl0_2_6,script=no -device virtio-net-pci,netdev=net7,mac=50:00:00:02:00:07 -netdev tap,id=net7,ifname=vunl0_2_7,script=no -device virtio-net-pci,netdev=net8,mac=50:00:00:02:00:08 -netdev tap,id=net8,ifname=vunl0_2_8,script=no -device virtio-net-pci,netdev=net9,mac=50:00:00:02:00:09 -netdev tap,id=net9,ifname=vunl0_2_9,script=no -device virtio-net-pci,netdev=net10,mac=50:00:00:02:00:0a -netdev tap,id=net10,ifname=vunl0_2_10,script=no -device virtio-net-pci,netdev=net11,mac=50:00:00:02:00:0b -netdev tap,id=net11,ifname=vunl0_2_11,script=no -smp 3 -m 4096 -name vMX-VFP -uuid 95375231-3628-47d3-87dc-87e2fd3b5720 -hda hda.qcow2 -machine type=pc-1.0,accel=kvm -serial mon:stdio -nographic
20845 ? S< 0:00 cpulimit -q -p 12896 -l 150 -b
If you need any information, please let me know.