Hello,
I spent a little more time on this strange evpn duplicate packet problem. I attached a high level diagram of my lab devices. One level deeper, from a tcpdump capture and an mon int traff command , I see that the single homed PE4 R4 in the diagram is duplicating the icmp request packet from H1 and sending both packets on, ie one to r2 and one to r3 (in the capture I saw a frame with a push of an mpls label (132) to r2 and I saw another frame with a push of an mpls label (141) to r3). Here is the the evpn database entry and I can see both MAC and IP for the far end multihomed CE2/H2 192.168.100.2/24.
root@test-r4-mx5> show evpn database mac-address b0:a8:6e:b8:0d:90 extensive
Instance: EVPN-BUNDLED-VLANS
VLAN ID: 100, MAC address: b0:a8:6e:b8:0d:90
State: 0x0
Source: 00:99:88:77:66:55:44:33:22:11, Rank: 1, Status: Active
Remote origin: 172.16.1.2
Remote state: <Mac-Only-Adv>
Remote origin: 172.16.1.3
Remote state: <Mac-Only-Adv>
Mobility sequence number: 0 (minimum origin address 172.16.1.2)
Timestamp: Aug 01 01:14:32.000000 (0x66aae178)
State: <>
MAC advertisement route status: Not created (no local state present)
IP address: 192.168.100.2
Remote origin: 172.16.1.2
Remote origin: 172.16.1.3
History db: <No entries>
So I need to dig deeper and figure out what is going on in PE4 R4.
Any ideas on why the single homed PE4 is duplicating icmp requests would be appreciated.
------------------------------
JohnK
------------------------------
Original Message:
Sent: 07-30-2024 03:49
From: PER GRANATH
Subject: EVPN/MPLS MX Duplicate packet
I have seen duplicate packets returned to ping a couple of time, in this scenario with ESI-LAG (both over MPLS and over VXLAN). Problem has been BGP policy in the overlay.
When the ingress PE/VTEP receives the packet it does not have a EVPN route to the MAC, so it will broadcast the packet to both remote PE/VTEP (that it has an IP subnet EVN route to). So the destination host is receiving two ICMP packets, and reply to both. The remote PE/VTEP should learn the destination MAC and advertise it to the other PE/VTEP (especially the "ingress" one)... but due to some BGP policy put in place for some optimization... the MAC route was never received or installed on the ingress PE so future packets cannot be unicast to only one of the egress PE/VTEP.
------------------------------
PER GRANATH
Original Message:
Sent: 07-29-2024 13:54
From: John Karsch
Subject: EVPN/MPLS MX Duplicate packet
Hi Quentin,
Did you ever figure out why you were getting duplicate packets?
I have the exact same problem. I mocked it up in our lab with 3 eol mx104s (out of support agreement) as PEs running 21.2R3.8, using mpls, ospf, ldp (track igp), multihomed with nonzero ESI and all-active. DF and BDFs are clean and properly forwarding BUM traffic. MAC/IPs to evpn instance tables are clean. But end point hosts see duplicate icmp packets from their CEs. We are using MX10003s in production and single-active on our multihomed LAGs and that works fine (LAGs made from multiple et-* interfaces). They are running 22.2R3. When I set single-active in the lab on the ESI-LAG the duplicates go away. Problem is I can't open a JTAC case with the old mx104s and don't want to experiment with the MX10003s in production. Also, a PR search found nothing.
BTW I did see on a reddit thread that someone experiencing the same problem got feedback from Juniper with the conclusion, "The duplication is expected with the current design". I am just not buying it. If time permits I want to see if it is just icmp doing this. I used tcp with all-active and don't see duplicates causing problems on the hosts but likely tcp stacks on the hosts are ignoring any duplicates.
test-r2-mx104> show configuration interfaces ae0| display set | match active # r2 is a PE
set interfaces ae0 esi all-active
test-r3-mx104> show configuration interfaces ae0| display set | match active # r3 is a PE
set interfaces ae0 esi all-active
test-h1-srx> ping 192.168.100.2 routing-instance ri-100 count 3 # h1 is a CE connected end point
PING 192.168.100.2 (192.168.100.2): 56 data bytes
64 bytes from 192.168.100.2: icmp_seq=0 ttl=64 time=1.725 ms
64 bytes from 192.168.100.2: icmp_seq=0 ttl=64 time=2.097 ms (DUP!)
64 bytes from 192.168.100.2: icmp_seq=1 ttl=64 time=1.744 ms
64 bytes from 192.168.100.2: icmp_seq=1 ttl=64 time=2.115 ms (DUP!)
64 bytes from 192.168.100.2: icmp_seq=2 ttl=64 time=1.736 ms
--- 192.168.100.2 ping statistics ---
3 packets transmitted, 3 packets received, +2 duplicates, 0% packet loss
round-trip min/avg/max/stddev = 1.725/1.883/2.115/0.182 ms
For interest sake, yes the lacp system id is the same on both PEs for the MC/ESI - LAG to the one CE as is the esi.
------------------------------
JohnK
Original Message:
Sent: 04-02-2024 14:23
From: QUENTIN
Subject: EVPN/MPLS MX Duplicate packet
Hi !
I'm facing an issue regarding EVPN configuration.
Everything seems work fine but I can explain there are duplicate packets when I ping internet from PC or from internet to PC (see the attached file).
Using EVE-NG as lab, I do not have this issue. I see the duplicate packets only in production.
The goal of the implementation of EVPN/MPLS is to replace VRRP protocol, offer redundancy in case of gateway failure and provide a distributed gateway IP.
That's why I have a layer 2 between the EX switches. In this use case, vlan 666 is configured on MX and EX. PC is also in this vlan with a /30 prefix.
I tried with the same ESI and different ESi between PE but same result (all-active is used).
Do you know what can explain this duplicate packets ?
"show evpn instance extensive" shows that there is one designated forwarder and one backup forwarder.
The routing table of instance.evpn.0 and bgp.evpn.0 is okay
Thanks for your help !