Junos OS

Good day,

I have MX960 with 2 RE. ver 10.0R4.7

Yesterday after switchover to backup RE (disk fail was the reason), I see high CPU utilization for fpc0 and fpc2  DPCE-R line cards.

I changed the cards, sfps but it not help. Also I found out if I disable pics xe-2/0/0 and xe-2/1/0 the cpu utilization for fpc2 decrease to normal ~10-15%.

Any ideas?

evgeniy@mx960-slave> show chassis fpc

                     Temp  CPU Utilization (%)   Memory    Utilization (%)

Slot State            (C)  Total  Interrupt      DRAM (MB) Heap     Buffer

  0  Online            29     86          2       1024       46         31

  1  Online            27     10          0       1024       46         31

  2  Online            25    100          3       1024       46         30

  3  Online            24     12          0       1024       46         31

  4  Online            25     12          0       1024       46         31

Regards,

Evgeniy

Hey,

You didn't say much about yout config, unfortunately. Are you using GRES+NSR? Did you see RE CPU utilization spike as well (I guess not as you didn't mention, but worth to check)? Is there anything DPC related in the logs? When you log into DPC (> start shell pfe network fpc2), can you see any errors being thrown in the console?

The interfaces you disabled, do they have any filters applied (if so, can you remove them)? Are these part of VPLS/bridge-domain?

There is number of PRs related to RPD issues during/after switchover, however based on your description there is nothing obvious matching PFE spike.

Thanks,

Hi,
> You didn't say much about yout config, unfortunately
Mainly ipv4 and v6 routing. A lot of bgp sessions.

>Are you using GRES+NSR?
but I tried disable, without result.

>Did you see RE CPU utilization spike as well
no. Just everyday utilization

 PID USERNAME  THR PRI NICE   SIZE    RES STATE    TIME   WCPU COMMAND
   11 root        1 171   52     0K    12K RUN    9293.6 81.25% idle
88095 root        2   8  -88 38580K 16072K nanslp 180:31  6.05% chassisd
   20 root        1 -68 -187     0K    12K RUN     83.6H  1.95% irq10: em0 em1+++*
   12 root        1 -40 -159     0K    12K WAIT    34.5H  0.93% swi2: net
87047 root        1   4    0   968M   948M kqread  58:46  0.49% rpd
 2880 root        1   4    0     0K    12K peer_s   3:01  0.24% peer proxy

 >Is there anything DPC related in the logs?
 No. Nothing strange in log about fpc

 > When you log into DPC (> start shell pfe network fpc2), can you see any errors being thrown in the console?
 No errors in console, but I see the following -

 ADPC2(mx960-slave vty)# show sched
Total uptime 0+07:20:50, (26450900 ms), 183897628 thread dispatches
CPU load is 100% (5 second), 91% (1 minute)
Total network interrupt time 0 (usec)

 CPU       Name      Time(ms)
  0%       Idle      155765
 96%    Threads      25441545
  3%        ISR      853590
  3%    Level 1      818515
  0%    Level 2      31430
  0%    Level 3      5
  0%    Level 5      3640

  Threads the issues of high CPU.

Hey,

I can see you posted some pfe logs on jnsp. What is the utilization (bps but more importantly pps) of the interfaces on those hot DPCs? What is connected from remote side? Any chance you're getting lot of ARP requests or some multicast traffic?

Thanks.

Hi,

the issue was related the bridge-domain.

After removing this portion of configuration the CPU is ok.  But there's not any l2 loops in the topology.

Its seems a software bug...