Switching

Hello

 

I am looking for any guidelines regarding  how to configure dual-side mc-lag between two pairs of QFX5100 (two mc-laged spines connected via the mc-lag to two mc-lagged leafs). I am especially intrested in the list on parameters that need to be unique between the two pairs.

 

Thank you in advance for any answer

Regards

Lukas

Do you mean two MC-LAG peers fully meshed to another pair of MC-LAG peers? Can you upload a topology?

I'll be interested to see if there is a working example of mc-lag to mc-lag connections.  I tried to get this work in lab setups a few years back unsuccessfully.

 

The only configuration examples I could find all had mc-lag connected to standard ae ports on the other side.  These showed connecting two mc-lag on one set of chassis to individual ae bundles on the other two chassis.  So I assumed connecting mc-lag to mc-lag directly was at least not recommended but could not find anything definitive.

 

Your other option here is to put the spine nodes into a virtual chassis as a pair.  Once in a VC your would be able to link the two VC spine using a normal ae bundle with a physical port on each switch.

 

Yes, it works fine--I've used it to connect aggregation to core layers (QXF to EX9200). I can post relevant configs later.

Hello

 

I would appraciate it if you can share the working configs.

 

Regards

Lukas

These are redacted configs so apologies for any typos. This is from a pair of mclag peers (core & agg) fully-meshed with each other, as well as an edge device off the agg layer. Layer 3 for the two sample VLANs live on the cores through VRRP. ICCP communication between peers uses a local non-routed subnet on an irb interface. While it’s possible to use the same vlan-id (11) at each layer I tag them differently in case it accidentally gets added to the trunk and breaks everything.

 

Thanks for sharing. 

 

I'm trying to do the almost identical setup, but running with a pair of links for the IC-CP (ae0) and a pair for IC-PL in each switch pair.

 

When i bring everthing up, it establishes fine, but I get a huge traffic storm when mac addresses jump from the pair links to the interswitch links.  Along with duplicated packets during a ping test

 

I'm just doing Layer2 on this pair of pairs, as Layer3 I'm doing on different hardware as I have mixed usages for the network (firewall rules etc)

 

Any ideas?   the show ethernet-switching mac-learning-log shows flips at exactly the same time as the dups are created.

Can you share your configs? It sounds like one or more of the crossconnects is not a member of the link aggregation.

Configs are attached.  I pulled out the authentication and login details, as well as the default storm control on the access ports.

 

I did notice the AE device count was not the same, fixed it, didn't make any difference.

The core configs look okay,which makes me think its downstream on the edge switches. Do you see the behavior with none of the edge switches connected? What does one of the edge switch configs look like? Adding one at a time could point you in the right direction. What is ge-46 on sw02b? I don’t see that in the config. 

testing with a single server, and the mgmnt switch, results in a storm facing any active port

sw01b                             Seconds: 56                  Time: 00:30:20

Interface    Link  Input packets        (pps)     Output packets        (pps)
 gr-0/0/0      Up              0          (0)                0          (0)
 pfh-0/0/0     Up              0                             0
 xe-0/0/0      Up          94665          (0)           552598          (6)
 xe-0/0/2    Down          43139          (0)       8775544275          (0)
 xe-0/0/4      Up           6013          (0)       7475166519     (670804)
 xe-0/0/6    Down          63539          (0)       7459039080          (0)
 xe-0/0/8    Down          11228          (0)       7444012154          (0)
 xe-0/0/46   Down              0          (0)                0          (0)
 xe-0/0/47     Up         322698          (8)            97977          (0)
 et-0/0/50     Up    23805121311     (335561)      32188696666     (334767)
 et-0/0/52     Up    11661520203     (334286)       8134729449          (0)
 et-0/0/53     Up    20527408162          (0)      15670470954     (336205)
 ae0           Up         417363          (8)           650575          (6)
 ae2         Down          43132          (0)       8767750835          (0)
 ae4           Up           6013          (0)       7475166519     (670804)
 ae6         Down          63537          (0)       7459039057          (0)
 ae8         Down          11224          (0)       7444012133          (0)
 ae50          Up    23805121311     (335561)      32188696666     (334767)
 ae52          Up    32188928365     (334286)      23805200403     (336205)

If I disable AE50 on any of the 4 switches, things go back to working, and "expected" behaviours. 

 

All 4 are QFX5100's - 2 Sfp ones, and 2 baseT ones.

Are the server’s two uplinks configured with LACP or is it just bridging the traffic?

Xenserver 7.6 - lacp for just the management interface.  no guests active.

Can you attach output for the four switches?

show lldp neighbors
show iccp brief
show interfaces mc-ae
show lacp interfaces extensive

Hi smicker. Thanks for the help. Here's the requested output for the four switches.

I’ve looked at the lacp and lldp output and I don’t see any indication of loops between the four switches. It looks like you’re using ge-0/0/46 on sw2a to extend the mgmt network through your core and my guess is that you have a mgmt loop somewhere on your upstream or downstream equipment, or you’ve set a mgmt port to layer 2 somewhere while also trunking it. You might try discconecting each mgmt interface and reading one at a time until the issue reappears.

Thx. I have noticed that you use the same lacp system-id on both sides of ae1? Shoudn't they be different between the core and agg switches?

The LACP system-id is a globally-unique value for the benefits of remote partners forming a LAG with a given actor. In my case it so happens that I actually do have different id’s but I changed them in the configs I posted to keep things consistent and easier to understand. I don’t think partner id’s and actor id’s have bearing on each other but it’s one of those things I’d probably not feel comfortable putting into production. In any case it would be good to lab it out and see 🙂

I have confirmed with the JTAC. The lacp sys-id need to be different between the both mc-lag sides.

Thanks for the follow-up--I've updated the configs above.

Hi Folks,

 

Couple of questions:

 

With this setup, would it possible to run OSPF, IBGP and MPLS between the aggregation and the core layer?

 

In addition to running VRRP and IRB on the Core layer, is it also possible to run similar function on the Aggregation layer?

 

We basically have a requirement where we need to stretch VLAN existing in the Core layer to the Aggregate layer and finally to the access layer.

 

--