Switching

Expand all | Collapse all

dual-side mc-lag with QFX 5100

  • 1.  dual-side mc-lag with QFX 5100

    Posted 12-08-2018 13:35

    Hello

     

    I am looking for any guidelines regarding  how to configure dual-side mc-lag between two pairs of QFX5100 (two mc-laged spines connected via the mc-lag to two mc-lagged leafs). I am especially intrested in the list on parameters that need to be unique between the two pairs.

     

    Thank you in advance for any answer

    Regards

    Lukas



  • 2.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-08-2018 16:10
    Do you mean two MC-LAG peers fully meshed to another pair of MC-LAG peers? Can you upload a topology?


  • 3.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-09-2018 03:43

    I'll be interested to see if there is a working example of mc-lag to mc-lag connections.  I tried to get this work in lab setups a few years back unsuccessfully.

     

    The only configuration examples I could find all had mc-lag connected to standard ae ports on the other side.  These showed connecting two mc-lag on one set of chassis to individual ae bundles on the other two chassis.  So I assumed connecting mc-lag to mc-lag directly was at least not recommended but could not find anything definitive.

     

    Your other option here is to put the spine nodes into a virtual chassis as a pair.  Once in a VC your would be able to link the two VC spine using a normal ae bundle with a physical port on each switch.

     



  • 4.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-09-2018 06:52

    Yes, it works fine--I've used it to connect aggregation to core layers (QXF to EX9200). I can post relevant configs later.



  • 5.  RE: dual-side mc-lag with QFX 5100

    Posted 12-09-2018 12:41

    Hello

     

    I would appraciate it if you can share the working configs.

     

    Regards

    Lukas



  • 6.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-09-2018 16:01

    These are redacted configs so apologies for any typos. This is from a pair of mclag peers (core & agg) fully-meshed with each other, as well as an edge device off the agg layer. Layer 3 for the two sample VLANs live on the cores through VRRP. ICCP communication between peers uses a local non-routed subnet on an irb interface. While it’s possible to use the same vlan-id (11) at each layer I tag them differently in case it accidentally gets added to the trunk and breaks everything.

     

    07FC02DC-8904-4AB6-A9AC-4D8C8D0E34C5.jpeg

    Attachment(s)

    rtf
    core1.rtf   4K 1 version
    rtf
    agg2.rtf   4K 1 version
    rtf
    core2.rtf   4K 1 version
    rtf
    agg1.rtf   4K 1 version


  • 7.  RE: dual-side mc-lag with QFX 5100

    Posted 12-09-2018 16:47

    Thanks for sharing. 

     

    I'm trying to do the almost identical setup, but running with a pair of links for the IC-CP (ae0) and a pair for IC-PL in each switch pair.

     

    When i bring everthing up, it establishes fine, but I get a huge traffic storm when mac addresses jump from the pair links to the interswitch links.  Along with duplicated packets during a ping test

     

    I'm just doing Layer2 on this pair of pairs, as Layer3 I'm doing on different hardware as I have mixed usages for the network (firewall rules etc)

     

    Any ideas?   the show ethernet-switching mac-learning-log shows flips at exactly the same time as the dups are created.



  • 8.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-09-2018 17:29

    Can you share your configs? It sounds like one or more of the crossconnects is not a member of the link aggregation.



  • 9.  RE: dual-side mc-lag with QFX 5100

    Posted 12-09-2018 18:18

    Configs are attached.  I pulled out the authentication and login details, as well as the default storm control on the access ports.

     

    I did notice the AE device count was not the same, fixed it, didn't make any difference.

    Attachment(s)

    txt
    sw01b.txt   8K 1 version
    txt
    sw01a.txt   17K 1 version
    txt
    sw02a.txt   7K 1 version
    txt
    sw02b.txt   6K 1 version


  • 10.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-09-2018 19:35

    The core configs look okay,which makes me think its downstream on the edge switches. Do you see the behavior with none of the edge switches connected? What does one of the edge switch configs look like? Adding one at a time could point you in the right direction. What is ge-46 on sw02b? I don’t see that in the config. 



  • 11.  RE: dual-side mc-lag with QFX 5100

    Posted 12-10-2018 00:39

    testing with a single server, and the mgmnt switch, results in a storm facing any active port

    sw01b                             Seconds: 56                  Time: 00:30:20
    
    Interface    Link  Input packets        (pps)     Output packets        (pps)
     gr-0/0/0      Up              0          (0)                0          (0)
     pfh-0/0/0     Up              0                             0
     xe-0/0/0      Up          94665          (0)           552598          (6)
     xe-0/0/2    Down          43139          (0)       8775544275          (0)
     xe-0/0/4      Up           6013          (0)       7475166519     (670804)
     xe-0/0/6    Down          63539          (0)       7459039080          (0)
     xe-0/0/8    Down          11228          (0)       7444012154          (0)
     xe-0/0/46   Down              0          (0)                0          (0)
     xe-0/0/47     Up         322698          (8)            97977          (0)
     et-0/0/50     Up    23805121311     (335561)      32188696666     (334767)
     et-0/0/52     Up    11661520203     (334286)       8134729449          (0)
     et-0/0/53     Up    20527408162          (0)      15670470954     (336205)
     ae0           Up         417363          (8)           650575          (6)
     ae2         Down          43132          (0)       8767750835          (0)
     ae4           Up           6013          (0)       7475166519     (670804)
     ae6         Down          63537          (0)       7459039057          (0)
     ae8         Down          11224          (0)       7444012133          (0)
     ae50          Up    23805121311     (335561)      32188696666     (334767)
     ae52          Up    32188928365     (334286)      23805200403     (336205)
    

    If I disable AE50 on any of the 4 switches, things go back to working, and "expected" behaviours. 

     

    All 4 are QFX5100's - 2 Sfp ones, and 2 baseT ones.



  • 12.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-10-2018 01:12
    Are the server’s two uplinks configured with LACP or is it just bridging the traffic?


  • 13.  RE: dual-side mc-lag with QFX 5100

    Posted 12-10-2018 12:31

    Xenserver 7.6 - lacp for just the management interface.  no guests active.



  • 14.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-10-2018 14:51
    Can you attach output for the four switches?

    show lldp neighbors
    show iccp brief
    show interfaces mc-ae
    show lacp interfaces extensive


  • 15.  RE: dual-side mc-lag with QFX 5100

    Posted 12-13-2018 14:42

    Hi smicker. Thanks for the help. Here's the requested output for the four switches.

    Attachment(s)

    rtf
    sw01a-19116-c.rtf   9K 1 version
    rtf
    sw02b-19116-c.rtf   4K 1 version
    rtf
    sw01b-19116-c.rtf   9K 1 version
    rtf
    sw02a-19116-c.rtf   4K 1 version


  • 16.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-17-2018 07:43

    I’ve looked at the lacp and lldp output and I don’t see any indication of loops between the four switches. It looks like you’re using ge-0/0/46 on sw2a to extend the mgmt network through your core and my guess is that you have a mgmt loop somewhere on your upstream or downstream equipment, or you’ve set a mgmt port to layer 2 somewhere while also trunking it. You might try discconecting each mgmt interface and reading one at a time until the issue reappears.



  • 17.  RE: dual-side mc-lag with QFX 5100

    Posted 12-09-2018 21:42

    Thx. I have noticed that you use the same lacp system-id on both sides of ae1? Shoudn't they be different between the core and agg switches?



  • 18.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-09-2018 23:26
    The LACP system-id is a globally-unique value for the benefits of remote partners forming a LAG with a given actor. In my case it so happens that I actually do have different id’s but I changed them in the configs I posted to keep things consistent and easier to understand. I don’t think partner id’s and actor id’s have bearing on each other but it’s one of those things I’d probably not feel comfortable putting into production. In any case it would be good to lab it out and see 🙂


  • 19.  RE: dual-side mc-lag with QFX 5100

    Posted 12-19-2018 01:26

    I have confirmed with the JTAC. The lacp sys-id need to be different between the both mc-lag sides.



  • 20.  RE: dual-side mc-lag with QFX 5100

     
    Posted 12-21-2018 07:58

    Thanks for the follow-up--I've updated the configs above.



  • 21.  RE: dual-side mc-lag with QFX 5100

     
    Posted 02-20-2020 20:10

    Hi Folks,

     

    Couple of questions:

     

    With this setup, would it possible to run OSPF, IBGP and MPLS between the aggregation and the core layer?

     

    In addition to running VRRP and IRB on the Core layer, is it also possible to run similar function on the Aggregation layer?

     

    We basically have a requirement where we need to stretch VLAN existing in the Core layer to the Aggregate layer and finally to the access layer.

     

    --