Switching

Expand all | Collapse all

Interface errors on AE, not physical ports

Jump to Best Answer
  • 1.  Interface errors on AE, not physical ports

    Posted 09-11-2019 12:30

    So i've been investigating issues on an interface where I see the following on the ae when I run "show interfaces extensive ae9" 

    Input errors: Errors: 28182975, Drops: 0, Framing errors: 28156078, Runts: 0, Giants: 26897, Policed discards: 0, Resource errors: 0

     

    I don't see these errors on the physical ports that make up the ae when I run the same command on them.  Is there a way to diagnose which link is responsible other than just checking every individual cable and component?



  • 2.  RE: Interface errors on AE, not physical ports

     
    Posted 09-11-2019 13:23

    What platform is it? If there's no physical port reporting framing error, but only ae reporting, this is something interesting. May not be related with L1. What is remote side?

     

    Please get following:

    1. config on ae

    2. show interface xx extensive (for both ae and member link)

    3. More information on the remote / transmission devices 



  • 3.  RE: Interface errors on AE, not physical ports

     
    Posted 09-11-2019 13:25

    another way to check. Remove all the interfaces from ae bundle. If this is physical issue, you should see framing error on one or several interfaces. This way, you don't need to go on site and switching cable optics



  • 4.  RE: Interface errors on AE, not physical ports

    Posted 09-16-2019 05:23

    Sorry for the delay in response.  This is a QFX-5100 VC.  The remote side is a Tintri VMstore T-850.  I have them cabled using DAC cables from Juniper and I've got another identical Tintri unit on this switch cabled the same way and no errors so it can't be an incompatibility.  Here's the output requested

     

     

    %> show configuration | display set | match ae9
    set interfaces xe-0/0/11 ether-options 802.3ad ae9
    set interfaces xe-1/0/11 ether-options 802.3ad ae9
    set interfaces ae9 description ********
    set interfaces ae9 aggregated-ether-options minimum-links 1
    set interfaces ae9 aggregated-ether-options lacp active
    set interfaces ae9 aggregated-ether-options lacp periodic slow
    set interfaces ae9 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae9 unit 0 family ethernet-switching vlan members 504-505
    
    {master:1}
    %> show interfaces ae9 extensive
    Physical interface: ae9, Enabled, Physical link is Up
      Interface index: 649, SNMP ifIndex: 563, Generation: 217
      Description: SP: AE, Mall-Tintri-2:Data B
      Link-level type: Ethernet, MTU: 1514, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1,
      Minimum bandwidth needed: 0
      Device flags   : Present Running
      Interface flags: SNMP-Traps Internal: 0x4000
      Current address: 10:0e:7e:b3:4b:49, Hardware address: 10:0e:7e:b2:98:89
      Last flapped   : 2019-07-31 11:16:40 EDT (6w4d 20:53 ago)
      Statistics last cleared: Never
      Traffic statistics:
       Input  bytes  :      353521069280834                    0 bps
       Output bytes  :       74239734949079                 1128 bps
       Input  packets:         241388622839                    0 pps
       Output packets:          67732251873                    2 pps
       IPv6 transit statistics:
        Input  bytes  :                   0
        Output bytes  :                   0
        Input  packets:                   0
        Output packets:                   0
      Input errors:
        Errors: 28183181, Drops: 0, Framing errors: 28156284, Runts: 0, Giants: 26897, Policed discards: 0, Resource errors: 0
      Output errors:
        Carrier transitions: 62, Errors: 0, Drops: 0, MTU errors: 0, Resource errors: 0
      Egress queues: 12 supported, 5 in use
      Queue counters:       Queued packets  Transmitted packets      Dropped packets
        0 best-effort                    0          67473706689                    0
        3 fcoe                           0                    0                    0
        4 no-loss                        0                    0                    0
        7 network-cont                   0              9718718                    0
        8 mcast                          0            248826472                    0
      Queue number:         Mapped forwarding classes
        0                   best-effort
        3                   fcoe
        4                   no-loss
        7                   network-control
        8                   mcast
    
      Logical interface ae9.0 (Index 627) (SNMP ifIndex 628) (Generation 355)
        Flags: SNMP-Traps 0x20024000 Encapsulation: Ethernet-Bridge
        Statistics        Packets        pps         Bytes          bps
        Bundle:
            Input : 4918700452339422880          0 8079587992859            0
            Output: 8164732829696          0 1776827211271309528            0
        Link:
          xe-0/0/11.0
            Input :             0          0             0            0
            Output:       2505316          0     856817813            0
          xe-1/0/11.0
            Input :             0          0             0            0
            Output:       2513073          0     859470966            0
        LACP info:        Role     System             System      Port    Port  Port
                                 priority          identifier  priority  number   key
          xe-0/0/11.0    Actor        127  10:0e:7e:b3:4a:80       127      32    10
          xe-0/0/11.0  Partner      65535  90:e2:ba:a5:9b:ec       255       2    33
          xe-1/0/11.0    Actor        127  10:0e:7e:b3:4a:80       127      34    10
          xe-1/0/11.0  Partner      65535  90:e2:ba:a5:9b:ec       255       1    33
        LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx
          xe-0/0/11.0          2301125     2318030            0            0
          xe-1/0/11.0          2320450     2325221            0            0
        Marker Statistics:   Marker Rx     Resp Tx   Unknown Rx   Illegal Rx
          xe-0/0/11.0                0           0            0            0
          xe-1/0/11.0                0           0            0            0
        Protocol eth-switch, MTU: 1514, Generation: 375, Route table: 6
          Flags: Trunk-Mode
    
    {master:1}
    %> show interfaces xe-0/0/11 extensive
    Physical interface: xe-0/0/11, Enabled, Physical link is Up
      Interface index: 731, SNMP ifIndex: 537, Generation: 307
      Description: SP: DAC, Mall-Tintri-2:Data B:2a
      Link-level type: Ethernet, MTU: 1514, Speed: 10Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Media type: Fiber
      Device flags   : Present Running
      Interface flags: SNMP-Traps Internal: 0x4000
      Link flags     : None
      CoS queues     : 12 supported, 12 maximum usable queues
      Hold-times     : Up 0 ms, Down 0 ms
      Current address: 10:0e:7e:b3:4b:49, Hardware address: 10:0e:7e:b2:97:ce
      Last flapped   : 2019-07-31 11:16:38 EDT (6w4d 20:53 ago)
      Statistics last cleared: Never
      Traffic statistics:
       Input  bytes  :       58609636476750                    0 bps
       Output bytes  :       35280860920615                 1072 bps
       Input  packets:          42055024966                    0 pps
       Output packets:          35504738199                    1 pps
       IPv6 transit statistics:
        Input  bytes  :                   0
        Output bytes  :                   0
        Input  packets:                   0
        Output packets:                   0
      Input errors:
        Errors: 28183181, Drops: 0, Framing errors: 28156284, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
      Output errors:
        Carrier transitions: 31, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
      Egress queues: 12 supported, 5 in use
      Queue counters:       Queued packets  Transmitted packets      Dropped packets
        0 best-effort                    0          35341674250                    0
        3 fcoe                           0                    0                    0
        4 no-loss                        0                    0                    0
        7 network-cont                   0              4823350                    0
        8 mcast                          0            158240600                    0
      Queue number:         Mapped forwarding classes
        0                   best-effort
        3                   fcoe
        4                   no-loss
        7                   network-control
        8                   mcast
      Active alarms  : None
      Active defects : None
      MAC statistics:                      Receive         Transmit
        Total octets                58609636476750   35280860920615
        Total packets                  42055024966      35504738199
        Unicast packets                42041774992      35343375621
        Broadcast packets                 10949169        156539071
        Multicast packets                  2300805          4823507
        CRC/Align errors                  28156284                0
        FIFO errors                              0                0
        MAC control frames                       0                0
        MAC pause frames                       302                0
        Oversized frames                     26897
        Jabber frames                        81940
        Fragment frames                      39786
        VLAN tagged frames             42052724161
        Code violations                          0
      MAC Priority Flow Control Statistics:
        Priority :  0                             0                0
        Priority :  1                             0                0
        Priority :  2                             0                0
        Priority :  3                             0                0
        Priority :  4                             0                0
        Priority :  5                             0                0
        Priority :  6                             0                0
        Priority :  7                             0                0
      Filter statistics:
        Input packet count                       0
        Input packet rejects                     0
        Input DA rejects                         0
        Input SA rejects                         0
        Output packet count                                       0
        Output packet pad count                                   0
        Output packet error count                                 0
        CAM destination filters: 1, CAM source filters: 0
      Packet Forwarding Engine configuration:
        Destination slot: 0 (0x00)
      CoS information:
        Direction : Output
        CoS transmit queue               Bandwidth               Buffer Priority   Limit
                                  %            bps     %           usec
        0 best-effort             5      500000000     5              0      low    none
        3 fcoe                   35     3500000000    35              0      low    none
        4 no-loss                35     3500000000    35              0      low    none
        7 network-control         5      500000000     5              0      low    none
        8 mcast                  20     2000000000    20              0      low    none
      Interface transmit statistics: Disabled
    
      Logical interface xe-0/0/11.0 (Index 595) (SNMP ifIndex 647) (Generation 404)
        Flags: SNMP-Traps 0x24024000 Encapsulation: Ethernet-Bridge
        Traffic statistics:
         Input  bytes  :                39928
         Output bytes  :            856818155
         Input  packets:                  322
         Output packets:              2505317
        Local statistics:
         Input  bytes  :                39928
         Output bytes  :            856818155
         Input  packets:                  322
         Output packets:              2505317
        Transit statistics:
         Input  bytes  :                    0                    0 bps
         Output bytes  :                    0                    0 bps
         Input  packets:                    0                    0 pps
         Output packets:                    0                    0 pps
        Protocol aenet, AE bundle: ae9.0, Generation: 436, Route table: 0
    
    {master:1}
    %> show interfaces xe-1/0/11 extensive
    Physical interface: xe-1/0/11, Enabled, Physical link is Up
      Interface index: 720, SNMP ifIndex: 648, Generation: 294
      Description: SP: DAC, Mall-Tintri-2:Data B:2b
      Link-level type: Ethernet, MTU: 1514, Speed: 10Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled, Source filtering: Disabled, Flow control: Disabled, Media type: Fiber
      Device flags   : Present Running
      Interface flags: SNMP-Traps Internal: 0x4000
      Link flags     : None
      CoS queues     : 12 supported, 12 maximum usable queues
      Hold-times     : Up 0 ms, Down 0 ms
      Current address: 10:0e:7e:b2:98:89, Hardware address: 10:0e:7e:b3:4a:8e
      Last flapped   : 2019-07-31 11:16:38 EDT (6w4d 20:53 ago)
      Statistics last cleared: Never
      Traffic statistics:
       Input  bytes  :      294911432805108                    0 bps
       Output bytes  :       38958874043758                  616 bps
       Input  packets:         199333597887                    0 pps
       Output packets:          32227513897                    1 pps
       IPv6 transit statistics:
        Input  bytes  :                   0
        Output bytes  :                   0
        Input  packets:                   0
        Output packets:                   0
      Input errors:
        Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
      Output errors:
        Carrier transitions: 31, Errors: 0, Drops: 0, Collisions: 0, Aged packets: 0, FIFO errors: 0, HS link CRC errors: 0, MTU errors: 0, Resource errors: 0
      Egress queues: 12 supported, 5 in use
      Queue counters:       Queued packets  Transmitted packets      Dropped packets
        0 best-effort                    0          32132032451                    0
        3 fcoe                           0                    0                    0
        4 no-loss                        0                    0                    0
        7 network-cont                   0              4895374                    0
        8 mcast                          0             90586072                    0
      Queue number:         Mapped forwarding classes
        0                   best-effort
        3                   fcoe
        4                   no-loss
        7                   network-control
        8                   mcast
      Active alarms  : None
      Active defects : None
      MAC statistics:                      Receive         Transmit
        Total octets               294911432805108   38958874043758
        Total packets                 199333597887      32227513897
        Unicast packets               199331228092      32133188619
        Broadcast packets                    21972         89429742
        Multicast packets                  2347823          4895536
        CRC/Align errors                         0                0
        FIFO errors                              0                0
        MAC control frames                       0                0
        MAC pause frames                       292                0
        Oversized frames                         0
        Jabber frames                            0
        Fragment frames                          0
        VLAN tagged frames            199331250064
        Code violations                          0
      MAC Priority Flow Control Statistics:
        Priority :  0                             0                0
        Priority :  1                             0                0
        Priority :  2                             0                0
        Priority :  3                             0                0
        Priority :  4                             0                0
        Priority :  5                             0                0
        Priority :  6                             0                0
        Priority :  7                             0                0
      Filter statistics:
        Input packet count                       0
        Input packet rejects                     0
        Input DA rejects                         0
        Input SA rejects                         0
        Output packet count                                       0
        Output packet pad count                                   0
        Output packet error count                                 0
        CAM destination filters: 1, CAM source filters: 0
      Packet Forwarding Engine configuration:
        Destination slot: 0 (0x00)
      CoS information:
        Direction : Output
        CoS transmit queue               Bandwidth               Buffer Priority   Limit
                                  %            bps     %           usec
        0 best-effort             5      500000000     5              0      low    none
        3 fcoe                   35     3500000000    35              0      low    none
        4 no-loss                35     3500000000    35              0      low    none
        7 network-control         5      500000000     5              0      low    none
        8 mcast                  20     2000000000    20              0      low    none
      Interface transmit statistics: Disabled
    
      Logical interface xe-1/0/11.0 (Index 635) (SNMP ifIndex 663) (Generation 391)
        Flags: SNMP-Traps 0x20020000 Encapsulation: Ethernet-Bridge
        Traffic statistics:
         Input  bytes  :                 1612
         Output bytes  :            859471650
         Input  packets:                   13
         Output packets:              2513075
        Local statistics:
         Input  bytes  :                 1612
         Output bytes  :            859471650
         Input  packets:                   13
         Output packets:              2513075
        Transit statistics:
         Input  bytes  :                    0                    0 bps
         Output bytes  :                    0                    0 bps
         Input  packets:                    0                    0 pps
         Output packets:                    0                    0 pps
        Protocol aenet, AE bundle: ae9.0, Generation: 420, Route table: 0
    
    {master:1}
    

     

     

     

     

     



  • 5.  RE: Interface errors on AE, not physical ports

     
    Posted 09-11-2019 23:44

    Hi

     

    This does not look like an issue with physical interface. 

    What you can do is find out if these errors are incrementing due to any single port or if it is being incremented by all the ports. So if this is incrementing cumulatively for all the ports then it is layer 2 framing errors which could be caused by some of the frames received on this ae bundle from the remote side which are being classified as framing errors for those packets. 

     

    Thanks



  • 6.  RE: Interface errors on AE, not physical ports

     
    Posted 09-12-2019 13:20

    Framing errors are typically related to physical issues on the cable or SFPs. See https://kb.juniper.net/InfoCenter/index?page=content&id=KB27597&cat=JUNOS&actp=LIST

     

     As per your output, the port is also reporting 26897 giant packets, which are packets over the size the interface supports. You may want to check the MTU settings on both peers as well.

     

     



  • 7.  RE: Interface errors on AE, not physical ports
    Best Answer

     
    Posted 09-16-2019 05:35

    @lcroce wrote:

    I don't see these errors on the physical ports that make up the ae when I run the same command on them.  Is there a way to diagnose which link is responsible other than just checking every individual cable and component?


     

    Unless I'm misunderstanding the question, member xe-0/0/11 has the exact same error count as the ae. 

     

      Input errors:
        Errors: 28183181, Drops: 0, Framing errors: 28156284, Runts: 0, Policed discards: 0, L3 incompletes: 0, L2 channel errors: 0, L2 mismatch timeouts: 0, FIFO errors: 0, Resource errors: 0
    

     



  • 8.  RE: Interface errors on AE, not physical ports

    Posted 09-16-2019 07:07

    You're right.  I somehow missed that when I was initially looking over this.  Thanks.