Routing

Expand all | Collapse all

MX-5 High CPU - RPD process...

  • 1.  MX-5 High CPU - RPD process...

    Posted 11-07-2018 10:11

    Hi All,

     

    I have been tshooting high CPU on my MX5-t ... the 'User' process is hogging the CPU...

     

    l.net> show chassis routing-engine | match User                         
          User                      96 percent
          User                      93 percent
          User                      93 percent
          User                      93 percent

     

    net> show system processes extensive | match rpd                      
     1725 root        1  72    0  1455M  1257M RUN    2605.4 90.87% rpd

     

    The culprit task looks to be 'Resolve tree 2'

     

    l.net> show task accounting | except "0.0          0.0          0.0"    
    Task accounting is disabled.

    Task                       Started    User Time  System Time  Longest Run
    Scheduler                    67859          2.1          0.1          0.0
    hakr                             7          0.0            0          0.0
    RT                            5219          3:5          4.3          0.0
    BGP MultiPath                 5507          5:4          0.8          0.1
    BGP_Listen.0.0.0.0+179       11680          17.          0.9          0.1
    Resolve tree 2               13016          19:          2.9          0.1
    KRT                           2877          0.9          1.1          0.0
    SNMP Subagent                  522          0.4          0.0          0.0

     

    It has a large BGP table... along with another MX5-t which has User CPU of 4% ..

    .net> show route summary | match BGP
                     BGP: 2861904 routes, 767947 active
                     BGP: 156861 routes,  56663 active

     

    I don't see any flapping routes or interfaces... ?



  • 2.  RE: MX-5 High CPU - RPD process...

    Posted 11-15-2018 14:15

    anyone managed to fix this? i have tried a reboot with no luck... 



  • 3.  RE: MX-5 High CPU - RPD process...

     
    Posted 11-15-2018 17:54

    The resolve tree is for route rsoluiton, which runs when we see frequent route flaps. Below command should give the route churn information.

     

    Start shell

    rtsockmon -t

     

    Regards,
    Rahul



  • 4.  RE: MX-5 High CPU - RPD process...

    Posted 11-27-2018 02:42

    Interestingly I am still seeing this high CPU but there is no more than usual churn of routes 😞 but can't be an issue with the router as it's neighbor router is doing the same...

     

    Basically we have 2xMX5 in one site with RPD at around 3% and the other site at over 90%.. their configs are similar and they have pretty much the same amount of routes, i.e.

     

    MX5 with low CPU;

    inet.0: 776872 destinations, 1791982 routes (776871 active, 265978 holddown, 3 hidden)
    Direct: 10 routes, 9 active
    Local: 7 routes, 7 active
    OSPF: 10 routes, 8 active
    BGP: 1791950 routes, 776842 active
    Static: 4 routes, 4 active
    Aggregate: 1 routes, 1 active

    inet6.0: 57845 destinations, 105634 routes (57845 active, 16067 holddown, 0 hidden)
    Direct: 8 routes, 8 active
    Local: 12 routes, 12 active
    OSPF3: 6 routes, 6 active
    BGP: 105605 routes, 57816 active
    Static: 2 routes, 2 active
    Aggregate: 1 routes, 1 active

     

     MX5 with high CPU;

    inet.0: 776880 destinations, 2892847 routes (776872 active, 21258 holddown, 2 hidden)
    Direct: 9 routes, 9 active
    Local: 7 routes, 7 active
    OSPF: 8 routes, 6 active
    BGP: 2892819 routes, 776846 active
    Static: 3 routes, 3 active
    Aggregate: 1 routes, 1 active

    inet6.0: 57836 destinations, 160478 routes (57836 active, 171 holddown, 2 hidden)
    Direct: 8 routes, 8 active
    Local: 10 routes, 10 active
    OSPF3: 1 routes, 1 active
    BGP: 160458 routes, 57816 active
    Static: 1 routes, 1 active



  • 5.  RE: MX-5 High CPU - RPD process...

    Posted 12-12-2018 08:42

    Hi, 

     

    It mightbe a multipath issue, I see thta mpath task is also quite active.

    Are your MX'es have the same bgp mpath config? What is the Junos

    version?

     

    Could you try to disable bgp mpath on the problem box? 

    Mpath is local for bgp peer, so disabling/enabling it don't result in bgp

    session re-init (flap).

     

    Thanks,

    Alex



  • 6.  RE: MX-5 High CPU - RPD process...

    Posted 01-07-2019 10:32

    thanks for the reply ... i am running version Junos: 15.1R6.7

     

    Low CPU router;

     

    law@rt1.wg.net> show task accounting | except "0.0          0.0          0.0"    
    Task accounting is enabled.

    Task                       Started    User Time  System Time  Longest Run
    Scheduler                     6829          0.2          0.0          0.0
    hakr                             1            0          0.0          0.0
    RT                            1365          0.8          0.2          0.0
    ICMPv6                           1          0.0            0          0.0
    BGP MultiPath                  495          0.1          0.0          0.0
    BGP_Listen.0.0.0.0+179        1747          0.9          0.2          0.0

     

    High CPU router

    law@rt1.n4.net> show task accounting | except "0.0          0.0          0.0"    
    Task accounting is enabled.
    Task                       Started    User Time  System Time  Longest Run
    Memory                           1          0.0            0          0.0
    hakr                             1          0.0            0          0.0
    RT                             289          15.          0.2          0.0
    BGP MultiPath                  256          17.          0.0          0.1
    PIM I/O./var/run/ppmd_con        7          0.0            0          0.0
    BGP_RT_Background                5          0.0            0          0.0
    BGP_Listen.0.0.0.0+179         651          0.6          0.0          0.0
    Resolve tree 2                 595          54.          0.1          0.1

     

     

    as can be seen the resove tree 2 is huge at 54.... though you are right there is some taken by Multipath also... this is configured the same on both routers... though don't think it is needed as looking at the forwarding table there is no load-balancing anyways././/./ .



  • 7.  RE: MX-5 High CPU - RPD process...

    Posted 01-09-2019 13:34

    tried removing bgp multipath on to see if it reduces the CPU...

    ... made no change to CPU level 😞