We have the default route setup like "route 0.0.0.0/0 next-hop [ x.x.x.97 x.x.x.9 ]" and it is somewhat load balanced. x.x.x.97 is ISP1 and x.x.x.9 is ISP2. The IP that doesn't respond to ping is x.x.x.10, which is our IP side of the BGP config(.9 is BGP neighbor). We need both ISPs.
The ICMP traffic comes in on the correct interface(ae2). ae2 is configured with the IP address x.x.x.10. The ICMP goes out irb.200, which is the interface for ISP1 on a completely different IP.
Here is the output of when I monitor the ICMP traffic on each interface.
Listening on ae2, capture size 9999 bytes
In IP x.x.x.205 > x.x.x.10: ICMP echo request, id 3819, seq 1, length 64
In IP x.x.x.205 > x.x.x.10: ICMP echo request, id 3819, seq 2, length 64
In IP x.x.x.205 > x.x.x.10: ICMP echo request, id 3819, seq 3, length 64
In IP x.x.x.205 > x.x.x.10: ICMP echo request, id 3819, seq 4, length 64
In IP x.x.x.205 > x.x.x.10: ICMP echo request, id 3819, seq 5, length 64
Listening on irb.200, capture size 9999 bytes
Out IP x.x.x.10 > x.x.x.205: ICMP echo reply, id 3819, seq 1, length 64
Out IP x.x.x.10 > x.x.x.205: ICMP echo reply, id 3819, seq 2, length 64
Out IP x.x.x.10 > x.x.x.205: ICMP echo reply, id 3819, seq 3, length 64
Out IP x.x.x.10 > x.x.x.205: ICMP echo reply, id 3819, seq 4, length 64
Out IP x.x.x.10 > x.x.x.205: ICMP echo reply, id 3819, seq 5, length 64
This is a MX480, in case that is helpful info.
I was thinking maybe some kind of filter that takes traffic matching protocol ICMP on destination ip x.x.x.10 to go out interface ae2. But I'm not entirely sure that is possible.
Thanks
Original Message:
Sent: 04-21-2023 06:56
From: Erik Slagter
Subject: ICMP coming in one interface then out another
You will probably see this not only for icmp traffic ;-)
I guess this is the situation (which is a known challenge): you have two ISP's connected, both can deliver traffic to you, at separate ip addresses but only one has the default gateway, because you want all (new) egress traffic going to just one ISP. Or you want to load balance and have an ECMP route to both, wich is asking for trouble as both have different ip adresses ;-) So I assume the first situation.
Now this is what happens: you have a default gw to ISP 1 (ether on the switch on a router or firewall somewhere downstream). You get a ping (or something else) from some random address via ISP 2. The switch (or router/firewall or connected device) will respond and finally it's routed using the default gateway and the packet goes out via ISP 1, because that's where the default gateway points to. Neither your switch or a router or firewall will remember what ISP this "connection" was set up from (mind you: icmp is connectionless too).
This is a very difficult issue to solve, and possible solutions involve NAT and use of VRF's. The best way is to ditch one of the ISP as soon as possible.
------------------------------
Erik Slagter
Original Message:
Sent: 04-20-2023 09:39
From: JJnet479
Subject: ICMP coming in one interface then out another
Hello,
I have ICMP requests coming in on the interface called ae2, which is configured with the IP x.x.x.10 and going out a completed different interface called irb.200 causing 100% packet loss. The issue is that ae2 is one ISP and irb.200 is a different ISP.
How do I stop that from happening?
Is it possible to route ICMP traffic coming into an interface to go out the same interface? For example all ICMP traffic destined for x.x.x.10 go out interface ae2?
Thanks