Switching

 View Only
last person joined: 3 days ago 

Ask questions and share experiences about EX and QFX portfolios and all switching solutions across your data center, campus, and branch locations.

MC-LAG Local and remote state is 'down' for some ae interfaces while it is active for other

  • 1.  MC-LAG Local and remote state is 'down' for some ae interfaces while it is active for other

    This message was posted by a user wishing to remain anonymous
    Posted 08-31-2021 09:36
    This message was posted by a user wishing to remain anonymous

    Hi all

    I need your guidance on a mc-lag setup I have been working since last week. We need to create x16 mc-lags for servers that connect to our switches across two interfaces.
    The switch model is QFX5110. Each server has an interface connected to each of the two switches and we aim to have this in active/active mode. I am hoping if any of you might be able to spot a misconfig or perhaps let me know if there is additional config lacking? Thank you in advance

    Guide used is available  https://www.juniper.net/documentation/us/en/software/junos/mc-lag/topics/topic-map/examples-mc-lag.html#concept_ddp_glt_jpb


    On issuing "show interfaces mc-ac" on switch A, we have the following:
    Member Link : ae3
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : up
    Peer Status : active
    Peer State : up
    Logical Interface : ae3.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.1 ae0.0 up

    Member Link : ae2
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : up
    Peer Status : active
    Peer State : up
    Logical Interface : ae2.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.1 ae0.0 up

    Member Link : ae1
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : down
    Peer Status : active
    Peer State : down
    Logical Interface : ae1.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.1 ae0.0 up

    Member Link : ae4
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : down
    Peer Status : active
    Peer State : down
    Logical Interface : ae4.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.1 ae0.0 up

    On issuing "show interfaces mc-ac" on switch B, we have the following:
    Member Link : ae3
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : up
    Peer Status : active
    Peer State : up
    Logical Interface : ae3.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.2 ae0.0 up

    Member Link : ae2
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : up
    Peer Status : active
    Peer State : up
    Logical Interface : ae2.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.2 ae0.0 up

    Member Link : ae1
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : down
    Peer Status : active
    Peer State : down
    Logical Interface : ae1.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.2 ae0.0 up

    Member Link : ae4
    Current State Machine's State: mcae active state
    Local Status : active
    Local State : down
    Peer Status : active
    Peer State : down
    Logical Interface : ae4.0
    Topology Type : bridge
    Local State : up
    Peer State : up
    Peer Ip/MCP/State : 10.0.0.2 ae0.0 up

    As you can see on both outputs from switch A and B, the 'Peer state' and 'Local state' for ae1 and ae4  shown as 'down'
    -----------------------------------

    ICCP  state is active
    Switch A:
    Redundancy Group Information for peer 10.0.0.1
    TCP Connection : Established
    Liveliness Detection : Up
    Redundancy Group ID Status
    1 Up

    Client Application: lacpd
    Redundancy Group IDs Joined: 1

    Client Application: l2ald_iccpd_client
    Redundancy Group IDs Joined: 1

    Client Application: MCSNOOPD
    Redundancy Group IDs Joined: None

    Switch B
    Redundancy Group Information for peer 10.0.0.2
    TCP Connection : Established
    Liveliness Detection : Up
    Redundancy Group ID Status
    1 Up

    Client Application: lacpd
    Redundancy Group IDs Joined: 1

    Client Application: l2ald_iccpd_client
    Redundancy Group IDs Joined: 1

    Client Application: MCSNOOPD
    Redundancy Group IDs Joined: None

    Note: MC Lag members are not in the same vlan as the iccp peering.

    ----------------------
    Switch A:
    set chassis aggregated-devices ethernet device-count 20
    set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae2 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae3 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae4 aggregated-ether-options mc-ae chassis-id 0

    set multi-chassis multi-chassis-protection 10.0.0.1 interface ae0
    set multicast-snooping-options multichassis-lag-replicate-state
    set switch-options service-id 100

    deactivate interfaces xe-0/0/0 unit 0
    deactivate interfaces xe-0/0/1 unit 0
    deactivate interfaces xe-0/0/2 unit 0
    deactivate interfaces xe-0/0/3 unit 0

    set interfaces xe-0/0/0 ether-options 802.3ad ae1
    set interfaces xe-0/0/1 ether-options 802.3ad ae2
    set interfaces xe-0/0/2 ether-options 802.3ad ae3
    set interfaces xe-0/0/3 ether-options 802.3ad ae4

    set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae0 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae1 aggregated-ether-options minimum-links 1
    set interfaces ae1 aggregated-ether-options lacp active
    set interfaces ae1 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae1 aggregated-ether-options lacp admin-key 1
    set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 1
    set interfaces ae1 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae1 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae1 aggregated-ether-options mc-ae mode active-active
    set interfaces ae1 aggregated-ether-options mc-ae status-control active
    set interfaces ae1 unit 0 family ethernet-switching interface-mode access
    set interfaces ae1 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae2 aggregated-ether-options minimum-links 1
    set interfaces ae2 aggregated-ether-options lacp active
    set interfaces ae2 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae2 aggregated-ether-options lacp admin-key 1
    set interfaces ae2 aggregated-ether-options mc-ae mc-ae-id 2
    set interfaces ae2 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae2 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae2 aggregated-ether-options mc-ae mode active-active
    set interfaces ae2 aggregated-ether-options mc-ae status-control active
    set interfaces ae2 unit 0 family ethernet-switching interface-mode access
    set interfaces ae2 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae3 aggregated-ether-options minimum-links 1
    set interfaces ae3 aggregated-ether-options lacp active
    set interfaces ae3 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae3 aggregated-ether-options lacp admin-key 1
    set interfaces ae3 aggregated-ether-options mc-ae mc-ae-id 3
    set interfaces ae3 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae3 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae3 aggregated-ether-options mc-ae mode active-active
    set interfaces ae3 aggregated-ether-options mc-ae status-control active
    set interfaces ae3 unit 0 family ethernet-switching interface-mode access
    set interfaces ae3 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae4 aggregated-ether-options minimum-links 1
    set interfaces ae4 aggregated-ether-options lacp active
    set interfaces ae4 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae4 aggregated-ether-options lacp admin-key 4
    set interfaces ae4 aggregated-ether-options mc-ae mc-ae-id 4
    set interfaces ae4 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae4 aggregated-ether-options mc-ae chassis-id 0
    set interfaces ae4 aggregated-ether-options mc-ae mode active-active
    set interfaces ae4 aggregated-ether-options mc-ae status-control standby
    set interfaces ae4 unit 0 family ethernet-switching interface-mode access
    set interfaces ae4 unit 0 family ethernet-switching vlan members vl05

    Switch B:
    set chassis aggregated-devices ethernet device-count 20
    set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae2 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae3 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae4 aggregated-ether-options mc-ae chassis-id 1

    set multi-chassis multi-chassis-protection 10.0.0.2 interface ae0
    set multicast-snooping-options multichassis-lag-replicate-state
    set switch-options service-id 100

    deactivate interfaces xe-0/0/0 unit 0
    deactivate interfaces xe-0/0/1 unit 0
    deactivate interfaces xe-0/0/2 unit 0
    deactivate interfaces xe-0/0/3 unit 0

    set interfaces xe-0/0/0 ether-options 802.3ad ae1
    set interfaces xe-0/0/1 ether-options 802.3ad ae2
    set interfaces xe-0/0/2 ether-options 802.3ad ae3
    set interfaces xe-0/0/3 ether-options 802.3ad ae4

    Note: ae0 is not mc-lag. It is the aggregated interface between Switch A and B.

    set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae0 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae1 aggregated-ether-options minimum-links 1
    set interfaces ae1 aggregated-ether-options lacp active
    set interfaces ae1 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae1 aggregated-ether-options lacp admin-key 1
    set interfaces ae1 aggregated-ether-options mc-ae mc-ae-id 1
    set interfaces ae1 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae1 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae1 aggregated-ether-options mc-ae mode active-active
    set interfaces ae1 aggregated-ether-options mc-ae status-control standby
    set interfaces ae1 unit 0 family ethernet-switching interface-mode access
    set interfaces ae1 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae2 aggregated-ether-options minimum-links 1
    set interfaces ae2 aggregated-ether-options lacp active
    set interfaces ae2 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae2 aggregated-ether-options lacp admin-key 1
    set interfaces ae2 aggregated-ether-options mc-ae mc-ae-id 2
    set interfaces ae2 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae2 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae2 aggregated-ether-options mc-ae mode active-active
    set interfaces ae2 aggregated-ether-options mc-ae status-control standby
    set interfaces ae2 unit 0 family ethernet-switching interface-mode access
    set interfaces ae2 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae3 aggregated-ether-options minimum-links 1
    set interfaces ae3 aggregated-ether-options lacp active
    set interfaces ae3 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae3 aggregated-ether-options lacp admin-key 1
    set interfaces ae3 aggregated-ether-options mc-ae mc-ae-id 3
    set interfaces ae3 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae3 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae3 aggregated-ether-options mc-ae mode active-active
    set interfaces ae3 aggregated-ether-options mc-ae status-control standby
    set interfaces ae3 unit 0 family ethernet-switching interface-mode access
    set interfaces ae3 unit 0 family ethernet-switching vlan members vl05

    set interfaces ae4 aggregated-ether-options minimum-links 1
    set interfaces ae4 aggregated-ether-options lacp active
    set interfaces ae4 aggregated-ether-options lacp system-id 00:00:00:00:00:01
    set interfaces ae4 aggregated-ether-options lacp admin-key 4
    set interfaces ae4 aggregated-ether-options mc-ae mc-ae-id 4
    set interfaces ae4 aggregated-ether-options mc-ae redundancy-group 1
    set interfaces ae4 aggregated-ether-options mc-ae chassis-id 1
    set interfaces ae4 aggregated-ether-options mc-ae mode active-active
    set interfaces ae4 aggregated-ether-options mc-ae status-control active
    set interfaces ae4 unit 0 family ethernet-switching interface-mode access
    set interfaces ae4 unit 0 family ethernet-switching vlan members vl05

    set protocols rstp interface ae0 mode point-to-point
    set protocols rstp interface ae0 disable

    -----------------------------------------------
    LACP  states on switch A:
    Aggregated interface: ae3
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/2 Actor No No Yes Yes Yes Yes Fast Active
    xe-0/0/2 Partner No No Yes Yes Yes Yes Slow Active
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/2 Current Slow periodic Collecting distributing

    Aggregated interface: ae2
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/1 Actor No No Yes Yes Yes Yes Fast Active
    xe-0/0/1 Partner No No Yes Yes Yes Yes Slow Active
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/1 Current Slow periodic Collecting distributing

    Aggregated interface: ae1
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/0 Actor No Yes No No No Yes Fast Active
    xe-0/0/0 Partner No Yes No No No Yes Fast Passive
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/0 Defaulted Fast periodic Detached

    Aggregated interface: ae4
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/3 Actor No Yes No No No Yes Fast Active
    xe-0/0/3 Partner No Yes No No No Yes Fast Passive
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/3 Defaulted Fast periodic Detached
     Here you can see that Mux state for both ae1 and ae4 are 'detached'

    LACP State on switch B:
    Aggregated interface: ae3
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/2 Actor No No Yes Yes Yes Yes Fast Active
    xe-0/0/2 Partner No No Yes Yes Yes Yes Slow Active
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/2 Current Slow periodic Collecting distributing

    Aggregated interface: ae2
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/1 Actor No No Yes Yes Yes Yes Fast Active
    xe-0/0/1 Partner No No Yes Yes Yes Yes Slow Active
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/1 Current Slow periodic Collecting distributing

    Aggregated interface: ae1
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/0 Actor No Yes No No No Yes Fast Active
    xe-0/0/0 Partner No Yes No No No Yes Fast Passive
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/0 Defaulted Fast periodic Detached

    Aggregated interface: ae4
    LACP state: Role Exp Def Dist Col Syn Aggr Timeout Activity
    xe-0/0/3 Actor No Yes No No No Yes Fast Active
    xe-0/0/3 Partner No Yes No No No Yes Fast Passive
    LACP protocol: Receive State Transmit State Mux State
    xe-0/0/3 Defaulted Fast periodic Detached
    Here you can see that Mux state for both ae1 and ae4 are 'detached'

    On both switch, the  LACP information for system ID for ae1 and ae4 are set to all 0s:
    LACP Information
    Local Partner System ID : 00:00:00:00:00:00
    Peer Partner System ID : 00:00:00:00:00:00