Data Center

 View Only
last person joined: 8 days ago 

Ask questions and share experiences about Data Center Architecture and approaches.
  • 1.  proxy-macip-advertisement: What is the purpose?

    Posted 12-15-2024 00:10

    Hi everyone,

    I am trying to understand what problem  proxy-macip-advertisement  knob solves.  I have read several juniper links but  none of them make any sense to me.

    I built a small CRB topology, where  EX2/EX3 are layer 2 gateways while EX1 is a spine with RR enabled.

    Can anyone explain what is the point of using proxy-macip advertisement in below set up?

    Much appreciated!!

    Configuration:

    E1(S1):

    E2(L2):

    L3 ( E3):

    W



    ------------------------------
    Be kind!!
    ------------------------------


  • 2.  RE: proxy-macip-advertisement: What is the purpose?

    Posted 12-15-2024 00:30

    It looks like  this feature is useful in muri homed site as per below link, I do not understand why juniper recommends using it for CRB case 

    https://www.ietf.org/archive/id/draft-rbickhart-evpn-ip-mac-proxy-adv-02.html



    ------------------------------
    Be kind!!
    ------------------------------



  • 3.  RE: proxy-macip-advertisement: What is the purpose?
    Best Answer

    Posted 12-19-2024 01:17

    That's right, it's recommended to use for CRB Designs when the Spines serve as the Layer 3 gateway.  Basically, it's a way for the spines to synchronize their ARP tables and make sure they have the same EVPN Type-2 MAC+IP Routes.

    Let's say you have a basic two leaf and two spine multihomed design with one host behind each leaf (h1 and h2).  Both spines are running virtual-gateway across.  

    If h1 and h2 are in different subnets, then h1 will ARP for the default gateway when trying to communicate with h2.  Leaf1 hashes this traffic and forwards it across the link to spine1.  The leaf1, running Layer 2 only, has advertised an EVPN Type 2 MAC only route to spine1.  Traffic hits spine1 then goes to h2.  Now when h2 responds, it could take the path to spine2.  Without the proxy-macip-advertisement feature enabled, spine2 would have to re-ARP back to h1 causing a small delay in traffic.

    What happens when it is enabled, is that spine1 now knows that h1 lies behind leaf1 and generates a Type 2 MAC+IP route on behalf of leaf1 pointing to leaf1's VTEP.  So now spine2 has the MAC+IP route and can forward directly to leaf1 without having to re-ARP to learn h1's IP Address.  Although layer 2 only leafs can snoop ARP packets and create ARP entries plus generate a Type 2 MAC+IP route even when running only layer 2.




  • 4.  RE: proxy-macip-advertisement: What is the purpose?

    Posted 12-20-2024 17:40

    thanks jsullivan!!

    I have to read it real slow as it contains a lot of useful info.



    ------------------------------
    Be kind!!
    ------------------------------



  • 5.  RE: proxy-macip-advertisement: What is the purpose?

    Posted 12-20-2024 23:27
    Edited by LEEBAHI 12-20-2024 23:27

    Hi Jsullivan,

    Thanks so much for responding. Below I have quoted you for easy reference:

    f h1 and h2 are in different subnets, then h1 will ARP for the default gateway when trying to communicate with h2.  Leaf1 hashes this traffic and forwards it across the link to spine1.  The leaf1, running Layer 2 only, has advertised an EVPN Type 2 MAC only route to spine1.  Traffic hits spine1 then goes to h2.  Now when h2 responds, it could take the path to spine2.  Without the proxy-macip-advertisement feature enabled, spine2 would have to re-ARP back to h1 causing a small delay in traffic.

    But question becomes why L1 is able to  send BGP MAC/IP ( h1 mac/ip) to S1 but not to S2? In the lab testing , I do not see that. What I see L1 sends BGP MAC/IP update to S1 and S2 ( via RR),  both S1 and S2 are able to create arp caches using MAC/IP without using this proxy mac/ip advertisement.

    Much appreciated!!



    ------------------------------
    Be kind!!
    ------------------------------



  • 6.  RE: proxy-macip-advertisement: What is the purpose?

    Posted 12-21-2024 10:25

    L1 does send to both Spine 1 and Spine 2.  I just included Spine 1 because in my example, that is the Spine that answered the ARP request for the default gateway.

    Also in my scenario,  the leaf only sends a MAC Type 2 route to both spines, not a MAC+IP Type 2 route since the leaf is running layer 2 only.  So even if Spine 2 has the MAC Type 2 route, it doesn't have the IP address associated with it; hence the re-ARP that happens when the reply traffic from h2 is hashed toward Spine 2.

    This example assumes that the Layer 2 only leafs cannot send MAC+IP Type 2 routes when the IRBs are moved to the spines.  When the proxy-macip-advertisement option was first introduced, that was the original use case.  However, in time as more features were added we now know that the Leafs can snoop the ARP packets toward the Spines and build/advertise MAC+IP Type 2 routes even when running in Layer 2.  There are still more nuanced use cases though for the command, but they are more convoluted.

    If you're interested in really getting to the nuts and bolts of all this, a great book on Juniper and EVPN VXLAN is Deploying Juniper Data Centers with EVPN VXLAN by Aninda Chatterjee.  It goes into a great level of detail for all the typical deployments, and the labs are fairly easy to setup and follow along.  It also goes over all these extra knobs that aren't really explained in the Juniper KB.  Once I got about halfway through the book, everything just started clicking.




  • 7.  RE: proxy-macip-advertisement: What is the purpose?

    Posted 12-25-2024 19:20
    Edited by LEEBAHI 12-31-2024 22:46

    I just got the book and it is very great book. But it did not address the why Juniper still recommends using  Proxy MAC/IP in CRB design.

    (With all respect and credit to ChatterJee  the author of this great book)

    Chapter 5 page #255 Section:

     Historical (and Present Day) Relevance of proxy-macip-advertisement:

     With time and new enhancements, these L2-only leafs were able to snoop ARP packets inbound and create an ARP entry in 
    the micro-kernel of the PFE. This allowed such leafs to generate EVPN Type-2 MAC+IP routes directly without having the 
    need for a Layer 3–capable device to proxy for them. This rendered the use of the proxy-macip-advertisement configura
    tion option obsolete, for this particular use case.

    Then author explains a use case where we can still use this knob as explained below:

    Since these L2-only leafs are snooping ARP packets, they maintain a MAC-to-IP binding in the micro-kernel as well, which 
    has an aging timer associated to it. Once this timer expires, these leafs re-ARP for the destination. This is not a problem in 
    and of itself. The problem is that for re-ARP, the leafs use a source MAC address that belongs to an interface called pip0, and 
    this MAC address is unique to every device.

     This ARP reply can get hashed toward leaf2 or leaf3. If it gets hashed toward leaf3, this is a problem—since this pip0 inter
    face MAC address is unique per device, leaf3 does not have this MAC address in its table, which leads to a destination MAC 
    miss for this ARP reply, as shown in Figure 7-15

     Since this is a MAC address miss in leaf3’s MAC address table, the ARP reply gets unknown unicast flooded in the fabric, 
    using ingress replication, as shown in Figure 7-16. At a small scale, this is irrelevant, but when you consider large-scale data 
    centers with hundreds of thousands of hosts, this is lot of unnecessary flooding of traffic, potentially impacting the network 
    negatively.

     To get past this issue, the proxy-macip-advertisement configuration option continues to be a recommendation on CRB spines. 
    With this configuration in place, the spines associate an aging timer against the ARP entries that are installed via EVPN 
    Type-2 MAC+IP routes as well. Once the timer expires, the spines re-ARP for the destination, and when the destination host 
    responds with an ARP reply, the leafs refresh their timer as well, circumventing this problem.

    ####################################################################

    The whole point is to  mitigate ingress replication when arp entry expires on  Leaves  in multihomed case as explained by the author above

    But it  does not help mitigate at all:

    When arp entry on  spine do age aout, it forces spine to do ingress replication for arp request so we still end up with ingress replication when arp entry ages out on spine, so the problem is shifted form leaves to spine using this knob.

    I appreciate anyone from Juniper to share their thoughts.

    Much appreciated!!



    ------------------------------
    Be kind!!
    ------------------------------