MX304 installs the full Internet tables at 47,000 routes per second, and we will show you how we are testing it.
Introduction
If you test the MX304 in the lab in front of route/traffic generator, you will see an impressive result of 50,000 prefixes / seconds programmed in hardware. But on the Internet, your MX will receive real routes from actual routers and we can wonder if the FIB install rate is the same.
In this article, we will describe a test methodology to provide realistic results.
TLDR: the MX304 supports an internet route install rate of 46,990 routes per second for IPv4 Internet profile and 46,955 routes per second for IPv4 +IPv6 stack with 23.2R1 Junos Image.
Test Methodology
The Topology consists of two routers connected back to back and each linked to the route/traffic generator. The testing device connected to DUT advertises a single route and the one connected to the helper DUT advertises the internet prefixes.
EBGP peering is enabled across the routers as well as across the router and traffic generator. The DUT learns a single route from the traffic generator and 939k IPv4 Internet prefixes from the helper DUT. Traffic is sent from the single route as the source and internet routes as the destination with 99.9% of interface load with an iMIX packet size distribution..
The test starts with disabling the EBGP peering between the DUT and helper DUT:
- the traffic is dropped on the DUT (can't resolve the destination addresses)
- on the DUT, Tx is MAX rate and Rx is ZERO.
Once the interface between the two routers is enabled:
- the BGP session will establish
- the Helper router will advertise the internet prefixes to the DUT
- Rx Rate will increase as DUT installs the routes in the Packet Forwarding Engine
The total routes divided by T2-T1 gives the number of routes installed per second.
MX304 Test Configurations for IPv4 Internet Profile
regress@DUT> show configuration groups FIB_TEST
interfaces {
lo0 {
unit 0 {
family inet {
address 12.1.1.1/32;
}
}
}
et-0/2/1 {
mtu 9018;
unit 0 {
family inet {
address 15.1.1.1/24;
}
}
}
et-0/0/11 {
mtu 9018;
unit 0 {
family inet {
address 15.2.1.1/24;
}
}
}
}
routing-options {
router-id 12.1.1.1;
autonomous-system 65201;
}
protocols {
bgp {
group FIB_Test {
type external;
family inet {
unicast;
}
neighbor 15.1.1.2 {
peer-as 45001;
}
neighbor 15.2.1.2 {
peer-as 65202;
}
}
}
}
|
regress@Helper-DUT> show configuration groups FIB_TEST
interfaces {
lo0 {
unit 0 {
family inet {
address 12.1.1.2/32;
}
}
}
et-0/1/2 {
mtu 9018;
unit 0 {
family inet {
address 15.2.1.2/24;
}
}
}
et-0/0/0 {
mtu 9018;
unit 0 {
family inet {
address 15.3.1.1/24;
}
}
}
}
routing-options {
router-id 12.1.1.2;
autonomous-system 65202;
}
protocols {
bgp {
group FIB_Test {
type external;
family inet {
unicast;
}
neighbor 15.2.1.1 {
peer-as 65201;
}
neighbor 15.3.1.2 {
peer-as 45002;
}
}
}
}
|
The BGP Peering is in IDLE State after disabling the interface.
regress@DUT> configure
Entering configuration mode
[edit]
regress@DUT# set interfaces et-0/0/11 disable
[edit]
regress@DUT# commit and-quit
commit complete
Exiting configuration mode
regress@DUT> show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 1 Peers: 2 Down peers: 1
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
1 1 0 0 0 0
inet6.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
15.1.1.2 45001 98 435151 188387 2 47:26 Establ
inet.0: 1/1/1/0
15.2.1.2 65202 130159 32 0 6 3 Idle
regress@DUT>
Once the interface is enabled, the BGP peering comes up and DUT starts learning the routes
regress@DUT> configure
Entering configuration mode
[edit]
regress@DUT# delete interfaces et-0/0/11 disable
[edit]
regress@DUT# commit and-quit
commit complete
Exiting configuration mode
regress@DUT> show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 1 Peers: 2 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
939805 939805 0 0 0 0
inet6.0
0 0 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
15.1.1.2 45001 72 435080 0 2 34:13 Establ
inet.0: 1/1/1/0
15.2.1.2 65202 130130 4 0 5 32 Establ
inet.0: 939804/939804/939804/0
regress@DUT>
The 12 Core CPU of MX304 stays stable with most of the cores 100% Idle during IPv4 Internet Route learning as shown below:
As the routes are getting learnt in the DUT, the traffic starts flowing and once entire routes are installed Rx and Tx Rate matches as follows:
We write down T1 and T2, and the test is repeated three times.
For 939,804 IPv4 Routes, we measured:
| Iteration |
T1 |
T2 |
T2-T1(seconds) |
Nb of routes/(T2-T1) in RPS |
| Iter-1 |
3:39:58 |
3:40:18 |
20 |
46,990 |
| Iter-2 |
4:11:48 |
4:12:08 |
20 |
46,990 |
| Iter-3 |
4:21:22 |
4:21:42 |
20 |
46,990 |
| Total Routes |
939,804 |
|
Average |
46,990 |
IPv4 + IPv6 Internet Dual Stack Profile
MX304 Test Configurations for IPv4 and IPv6 Internet Profile
regress@DUT> show configuration groups FIB_TEST
interfaces {
lo0 {
unit 0 {
family inet {
address 12.1.1.1/32;
}
}
}
et-0/2/1 {
mtu 9018;
unit 0 {
family inet {
address 15.1.1.1/24;
}
family inet6 {
address 2002:15:1:1::1/64;
}
}
}
et-0/0/11 {
mtu 9018;
unit 0 {
family inet {
address 15.2.1.1/24;
}
family inet6 {
address 2002:15:2:1::1/64;
}
}
}
}
routing-options {
router-id 12.1.1.1;
autonomous-system 65201;
}
protocols {
bgp {
group FIB_Test {
type external;
family inet {
unicast;
}
neighbor 15.1.1.2 {
peer-as 45001;
}
neighbor 15.2.1.2 {
peer-as 65202;
}
}
group FIBv6_Test {
type external;
family inet6 {
unicast;
}
neighbor 2002:15:1:1::2 {
peer-as 45001;
}
neighbor 2002:15:2:1::2 {
peer-as 65202;
}
}
}
}
|
regress@Helper-DUT> show configuration groups FIB_TEST
interfaces {
lo0 {
unit 0 {
family inet {
address 12.1.1.2/32;
}
}
}
et-0/1/2 {
mtu 9018;
unit 0 {
family inet {
address 15.2.1.2/24;
}
family inet6 {
address 2002:15:2:1::2/64;
}
}
}
et-0/0/0 {
mtu 9018;
unit 0 {
family inet {
address 15.3.1.1/24;
}
family inet6 {
address 2002:15:3:1::1/64;
}
}
}
}
routing-options {
router-id 12.1.1.2;
autonomous-system 65202;
}
protocols {
bgp {
group FIB_Test {
type external;
family inet {
unicast;
}
neighbor 15.2.1.1 {
peer-as 65201;
}
neighbor 15.3.1.2 {
peer-as 45002;
}
}
group FIBv6_Test {
type external;
family inet6 {
unicast;
}
neighbor 2002:15:2:1::1 {
peer-as 65201;
}
neighbor 2002:15:3:1::2 {
peer-as 45002;
}
}
}
}
|
The BGP summary info with both IPv4 and IPv6 families are as follows:
regress@DUT> show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 2 Peers: 4 Down peers: 0
Table Tot Paths Act Paths Suppressed History Damp State Pending
inet.0
939805 939805 0 0 0 0
inet6.0
202353 202353 0 0 0 0
Peer AS InPkt OutPkt OutQ Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
15.1.1.2 45001 120 165340 0 1 58:10 Establ
inet.0: 1/1/1/0
15.2.1.2 65202 180506 124 0 0 55:21 Establ
inet.0: 939804/939804/939804/0
2002:15:1:1::2 45001 120 46920 0 1 58:08 Establ
inet6.0: 1/1/1/0
2002:15:2:1::2 65202 48469 124 0 0 55:22 Establ
inet6.0: 202352/202352/202352/0
The 12 Core CPU of MX304 was stable, most of the cores being at100% idle state during route-learning:
The FIB Install Rate with traffic flow is measured as follows:
Total number of routes: 939,804 (ipv4) + 202,352 (ipv6) = 1,142,156
| Iteration |
T1 |
T2 |
T2-T1(seconds) |
Nb of routes/(T2-T1) in RPS |
| Iter-1 |
1:15:00 |
1:15:24 |
24 |
47,589 |
| Iter-2 |
1:24:40 |
1:25:05 |
25 |
45,686 |
| Iter-3 |
1:48:20 |
1:48:44 |
24 |
47,589 |
| Total Routes |
1,142,156 |
|
Average |
46,955 |
Conclusion
MX304 running on 23.2R1 JUNOS image shows a route-learning rate of ~46,990 routes per second for IPv4 Internet profile and 46,955 routes per second for dual-stack profile. This test methodology provides more realistic results with the routes learnt from a real router BGP stack and using real internet tables.
Glossary
- BGP: Border Gateway Protocol
- DUT: Device Under Test
- FIB: Forwarding Information Base
- IPv4: Internet Protocol Version 4
- IPv6: Internet Protocol Version 6
- PFE: Packet Forwarding Engine
Acknowledgement
Many thanks to Kedar Kawadkar, Suresh Kumar Boyapati and Nicolas Fevrier for reviewing the article and providing the feedback.