Innovators Circle

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.

Expand all | Collapse all

Issues with LAG between 2 EX3400 VC's

  • 1.  Issues with LAG between 2 EX3400 VC's

    Posted 05-27-2021 17:56
    I am having issues getting a LAG up between 2 different EX3400 Virtual Chassis.

    I may be missing something very simple but really can't seem to figure this out.

    When we connect the second LAG port we experience sporadic connectivity. It may work for a few minutes or a few hours, but eventually we will begin to have issues.

    When we only run 1 uplink (Disable the second) the connectivity issues resolve immediatly.

    Any assistance would be greatly appreciated.

    Switch -1
    JUNOS 18.3R3.8 Kernel 32-bit JNPR-11.0-20190907.2386fa2_buil
    root@LTIIMTV-MOCC-JN3400SKPOE-DSwc> show configuration | display set | grep 0/47
    set interfaces ge-0/0/47 speed 1g
    set interfaces ge-0/0/47 ether-options 802.3ad ae0
    set interfaces ge-1/0/47 speed 1g
    set interfaces ge-1/0/47 ether-options 802.3ad ae0
    set poe interface ge-0/0/47 disable
    set poe interface ge-1/0/47 disable

    root@LTIIMTV-MOCC-JN3400SKPOE-DSwc> show configuration | display set | grep ae0
    set interfaces ge-0/0/47 ether-options 802.3ad ae0
    set interfaces ge-1/0/47 ether-options 802.3ad ae0
    set interfaces ae0 description "Uplink to LTIIMTV-MOCC-JN3400SKPOE-DSwc"
    set interfaces ae0 native-vlan-id 1
    set interfaces ae0 aggregated-ether-options minimum-links 1
    set interfaces ae0 aggregated-ether-options link-speed 1g
    set interfaces ae0 aggregated-ether-options lacp active
    set interfaces ae0 aggregated-ether-options lacp periodic slow
    set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae0 unit 0 family ethernet-switching vlan members all
    set protocols rstp interface ae0

    {master:1}
    root@LTIIMTV-MOCC-JN3400SKPOE-DSwc> show configuration | display set | grep aggregated-d
    set chassis aggregated-devices ethernet device-count 2

    root@LTIIMTV-MOCC-JN3400SKPOE-DSwc> show lacp statistics interfaces ae0
    Aggregated interface: ae0
    LACP Statistics:  LACP Rx      LACP Tx Unknown Rx   Illegal Rx
    ge-1/0/47               3067             3065                   0                   0
    ge-0/0/47               2642             2642                   0                   0

    root@LTIIMTV-MOCC-JN3400SKPOE-DSwc> show lacp interfaces ae0
    Aggregated interface: ae0
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    ge-1/0/47 Actor No No Yes Yes Yes Yes Slow Active
    ge-1/0/47 Partner No No Yes Yes Yes Yes Slow Active
    ge-0/0/47 Actor No Yes No No No Yes Slow Active
    ge-0/0/47 Partner No Yes No No No Yes Fast Passive
    LACP protocol: Receive State Transmit State Mux State
    ge-1/0/47 Current Slow periodic Collecting distributing
    ge-0/0/47 Port disabled No periodic Detached



    Switch -2
    JUNOS 18.3R2.7 Kernel 32-bit JNPR-11.0-20190316.df99236_buil

    liadmin2@LTIIMTV-WHF1-JN3400SKPOE-DSwc> show configuration | display set | grep 0/47
    set interfaces ge-0/0/47 description "Uplink to LTIIMTV-MOCC-JN3400SKPOE-DSwc"
    set interfaces ge-0/0/47 disable
    set interfaces ge-0/0/47 speed 1g
    set interfaces ge-0/0/47 ether-options 802.3ad ae0
    set interfaces ge-1/0/47 description "Uplink to LTIIMTV-MOCC-JN3400SKPOE-DSwc"
    set interfaces ge-1/0/47 speed 1g
    set interfaces ge-1/0/47 ether-options 802.3ad ae0
    set poe interface ge-0/0/47 disable
    set poe interface ge-1/0/47 disable


    liadmin2@LTIIMTV-WHF1-JN3400SKPOE-DSwc> show configuration | display set | grep ae0
    set interfaces ge-0/0/47 ether-options 802.3ad ae0
    set interfaces ge-1/0/47 ether-options 802.3ad ae0
    set interfaces ae0 description "Uplink to LTIIMTV-MOCC-JN3400SKPOE-DSwc "
    set interfaces ae0 native-vlan-id 1
    set interfaces ae0 aggregated-ether-options minimum-links 1
    set interfaces ae0 aggregated-ether-options link-speed 1g
    set interfaces ae0 aggregated-ether-options lacp active
    set interfaces ae0 aggregated-ether-options lacp periodic slow
    set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae0 unit 0 family ethernet-switching vlan members all
    set protocols rstp interface ae0

    {master:0}
    liadmin2@LTIIMTV-WHF1-JN3400SKPOE-DSwc> show configuration | display set | grep aggregated-d
    set chassis aggregated-devices ethernet device-count 1

    liadmin2@LTIIMTV-WHF1-JN3400SKPOE-DSwc# run show lacp statistics interfaces ae0
    Aggregated interface: ae0
    LACP Statistics:      LACP Rx        LACP Tx           Unknown Rx          Illegal Rx
    ge-0/0/47                    2642               2672                   0                                     0
    ge-1/0/47                    3065               3098                   0                                     0

    liadmin2@LTIIMTV-WHF1-JN3400SKPOE-DSwc# run show lacp interfaces ae0
    Aggregated interface: ae0
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    ge-1/0/47 Actor No No Yes Yes Yes Yes Slow Active
    ge-1/0/47 Partner No No Yes Yes Yes Yes Slow Active
    ge-0/0/47 Actor No Yes No No No Yes Slow Active
    ge-0/0/47 Partner No Yes No No No Yes Fast Passive
    LACP protocol: Receive State Transmit State Mux State
    ge-1/0/47 Current Slow periodic Collecting distributing
    ge-0/0/47 Port disabled No periodic Detached


  • 2.  RE: Issues with LAG between 2 EX3400 VC's

     
    Posted 05-28-2021 05:34
    Hi Sean, you mention that if you run things on only the first link things are fine but when both links are connected in the LAG things slowly start breaking down, a few questions that might help get you in the right direction:

    - Have you tried also testing it the other way around? So disabling the first link and running only on the second link? (that should help isolate whether there might be an issue on the second link)
    - Have you checked all 4 interfaces (both sides of the 2 links) for errors, drops etc?
    - Something else to check: What if you run the LACP on 2 interfaces on the same chassis, does that help? (that might isolate an issue on the VC link)
    - Is RSTP triggering by something maybe? I doubt it, since I do sort of assume you checked the logs and that would stick out very much
    - In that same vein: when you mention connectivity issues, no interfaces are flapping right, just traffic dropping away for a bit? Roughly how frequently and how long of an interruption are we talking?
    - A rather obscure one that we came across at some point: It might help to look at the ethernet-switching table when you are having the connectivity issues, to see if the switch might not be incorrectly learning (for example) the gateway mac from the "wrong" direction. It's a rather highly specific issue, but we had an issue where our switch was connected to an EVPN-VXLAN fabric but a internal FPC filter (that should be automatically installed when EVPN is configured) wasnt working and was causing occasional packets to loop back, causing the switch to "learn" a mac that is supposed to be behind it, suddenly from the QFX side (because it saw packets with that source-mac from there). In the end a reboot fixed the issue in the QFX, so a reboot of both switch stacks might be worth a shot to try if that is an option (heavy-handed, but unfortunately it *does* sometimes help with odd issues)

    If it's none of those then I fear you might need to call in the help of JTAC. We run dozens of EX3400 VCs with LACPs very much the same as what you're describing (even on the same ports ;)) so it's not an inherent problem in the EX3400 (unless you happen to hit a very odd and specific bug in the particular Junos release you're running of course)

    Hopefully some of these questions/ideas help point you in the right direction to find a cause for the issue, it sounds like a quite strange one.

    Regards,
    -Mark

    ------------------------------
    Mark Weijers
    ------------------------------