Junos OS

Expand all | Collapse all

Drops and Discards on QFX-5100 virtual chassis

  • 1.  Drops and Discards on QFX-5100 virtual chassis

    Posted 08-02-2018 03:11

    Hello,

    I have some issue on my network.

    topology is like that:

    2 QFX- 5100 ROLE: root and backup

    16 Line cards (ex-4300 & qfx 5100)

    The network is running video streaming, start from Source which is video sender, and reciver is Loader.

    Each loader register to multicast group on IRB 71

    Each Source send to multicast group on IRB 70

     

    When the system reaches around 23 g/bps Drop & Discards start to apear heavily.

    just to mention the ports are connected with 40 gb cables.

    No cos/qos implention .

    Recommended version is running on each Switch.

     

    does any one knows where to troubleshoot this? is this normal?

     

    root> show interfaces ae14 extensive
    Physical interface: ae14, Enabled, Physical link is Up
    Interface index: 654, SNMP ifIndex: 671, Generation: 169
    Description: LOADER - 10.71.60.102
    Link-level type: Ethernet, MTU: 1514, Speed: 20Gbps, BPDU Error: None, MAC-REWRITE Error: None, Loopback: Disabled,
    Source filtering: Disabled, Flow control: Disabled, Minimum links needed: 1, Minimum bandwidth needed: 1bps
    Device flags : Present Running
    Interface flags: SNMP-Traps Internal: 0x4000
    Current address: d4:04:ff:94:f6:fe, Hardware address: d4:04:ff:94:f6:fe
    Last flapped : 2018-06-13 10:48:33 IDT (7w1d 01:36 ago)
    Statistics last cleared: 2018-07-29 18:15:39 IDT (3d 18:08 ago)
    Traffic statistics:
    Input bytes : 1515850564345 77838320 bps
    Output bytes : 358613423879547 16235213648 bps
    Input packets: 21889643076 144225 pps
    Output packets: 241592637925 1349066 pps
    IPv6 transit statistics:
    Input bytes : 0
    Output bytes : 0
    Input packets: 0
    Output packets: 0
    Input errors:
    Errors: 0, Drops: 0, Framing errors: 0, Runts: 0, Giants: 0, Bucket drops: 0, Policed discards: 0, Resource errors: 0
    Output errors:
    Carrier transitions: 0, Errors: 0, Drops: 1124985593, MTU errors: 0, Resource errors: 0, Bucket drops: 0
    Egress queues: 12 supported, 5 in use
    Queue counters: Queued packets Transmitted packets Dropped packets
    0 0 241591445231 1124985593
    3 0 0 0
    4 0 0 0
    7 0 215177 0
    8 0 97541 0
    Queue number: Mapped forwarding classes
    0 best-effort
    3 fcoe
    4 no-loss
    7 network-control
    8 mcast

    Logical interface ae14.0 (Index 558) (SNMP ifIndex 1676) (Generation 167)
    Flags: Up SNMP-Traps 0x24024000 Encapsulation: Ethernet-Bridge
    Statistics Packets pps Bytes bps
    Bundle:
    Input : 0 0 0 0
    Output: 0 0 0 0
    Adaptive Statistics:
    Adaptive Adjusts: 0
    Adaptive Scans : 0
    Adaptive Updates: 0
    Link:
    xe-6/0/10.0
    Input : 0 0 0 0
    Output: 11697 0 3824919 0
    xe-6/0/11.0
    Input : 0 0 0 0
    Output: 11692 0 3823284 0
    xe-6/0/11.0
    Input : 0 0 0 0
    Output: 11692 0 3823284 0
    LACP info: Role System System Port Port Port
    priority identifier priority number key
    xe-6/0/10.0 Actor 127 d4:04:ff:94:f6:00 127 34 15
    xe-6/0/10.0 Partner 65535 00:0a:f7:6e:bd:40 255 1 11
    xe-6/0/11.0 Actor 127 d4:04:ff:94:f6:00 127 35 15
    xe-6/0/11.0 Partner 65535 00:0a:f7:6e:bd:40 255 2 11
    xe-6/0/11.0 Actor 127 d4:04:ff:94:f6:00 127 35 15
    xe-6/0/11.0 Partner 65535 00:0a:f7:6e:bd:40 255 2 11
    LACP Statistics: LACP Rx LACP Tx Unknown Rx Illegal Rx
    xe-6/0/10.0 324520 10816 0 0
    xe-6/0/11.0 324519 10816 0 0
    xe-6/0/11.0 324519 10816 0 0
    Marker Statistics: Marker Rx Resp Tx Unknown Rx Illegal Rx
    xe-6/0/10.0 0 0 0 0
    xe-6/0/11.0 0 0 0 0
    xe-6/0/11.0 0 0 0 0
    Protocol eth-switch, MTU: 1514, Generation: 186, Route table: 4

    Attachment(s)

    txt
    show_config.txt   232K 1 version
    txt
    display_set.txt   169K 1 version


  • 2.  RE: Drops and Discards on QFX-5100 virtual chassis

    Posted 08-02-2018 08:21

    Hi,

     

    i think your problem is scaling of the virtual chassis. This technology officially only supports up to 10 members per virtual chassis. You are on 18 members which is way over qualified limits.

     

    I would recommend you to split the setup into two seperate virtual chassis and then interconnect them with at least 2 x 40G bundled in an ae-interface.

     

    From the QFX5100 datasheet:

    Virtual Chassis Technology
    The QFX5100 supports Juniper Networks’ unique Virtual Chassis 
    technology, which enables up to 10 interconnected switches to 
    be managed and operated as a single, logical device with a single 
    IP address.

     



  • 3.  RE: Drops and Discards on QFX-5100 virtual chassis

    Posted 08-02-2018 08:53
    Ty for your comments.
    If it said the virtual chassis can support up to 10 manged switches.. why I can add more then 10?


  • 4.  RE: Drops and Discards on QFX-5100 virtual chassis

    Posted 08-02-2018 11:02

    Well, Juniper and Junos have many scenarios where you can configure over the limits of supported numbers.

    Supported means a design/configuration which has been through proper testing and validation.

     

    BUT: In this case I was wrong.  After more detailed review of your configuration I see that you have a  "Virtual chassis fabric" (VCF) which is a spine/leaf design compared to virtual chassis which usually is a ring topology. VCF supports up to 20 switches so you are within limits.

     

    MDT is also configured on both spines... to be honest I don't have any firm ideas about the issue. I would recommend to get in touch with JTAC for futher debugging.

     

    Note: Only 9 out of your 18 switches are under active support. Your EX4300-48T's and QFX5100-24Q (spines) haven't had their support renewed. My guess is that JTAC won't handle the case before support is in place on all devices in the VCF.

     

     



  • 5.  RE: Drops and Discards on QFX-5100 virtual chassis

     
    Posted 08-05-2018 13:27

    @SaharH wrote:

    No cos/qos implention .

     

    In fact you do:

    class-of-service {
        classifiers {
            ieee-802.1 802_1_P_CLASSIFIER {
                forwarding-class best-effort {
                    loss-priority low code-points [ 000 001 010 011 100 101 ];
                }
                forwarding-class network-control {
                    loss-priority low code-points [ 110 111 ];
                }
            }
        }
    .
    .
    .
    }
    

     

    All of your drops are on the BE queue: 

    Output errors:
    Carrier transitions: 0, Errors: 0, Drops: 1124985593, MTU errors: 0, Resource errors: 0, Bucket drops: 0
    Egress queues: 12 supported, 5 in use
    Queue counters: Queued packets Transmitted packets Dropped packets
    0                            0        241591445231      1124985593
    3                            0                   0               0
    4                            0                   0               0
    7                            0              215177               0
    8                            0               97541               0
    

     

    And you're pushing 16Gb/s on a 20Gb link

    Traffic statistics:
    Input bytes : 1515850564345 77838320 bps
    Output bytes : 358613423879547 16235213648 bps
    

     

    Is it not likely that you're simply bursting over 20Gb from time to time? Or microbursting?