Junos OS

Expand all | Collapse all

Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

  • 1.  Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

    Posted 12-11-2017 04:48

    Dear all, 

     

    I have several EX2300-T units running junos-arm-32-15.1X53-D56 that sometimes freeze and have the following messages : ( I have licenses for OSPF2 and all other services)

     

    All my units are in Virtual-Chassis of 2 units; 

     

    :08  CAI-EX2300-VSAT1-AB rpd[14775]: RPD_KRT_Q_RETRIES: Route Update

    : No buffer space available

    Oct 12 12:04:11  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 3 err 55, rtsm_id 0:-1, msg type 10

    Oct 12 12:04:16  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 3 err 55, rtsm_id 0:-1, msg type 2

    Oct 12 12:04:18  CAI-EX2300-VSAT1-AB rpd[14775]: RPD_KRT_Q_RETRIES: Route Update

    : No buffer space available

    Oct 12 12:04:21  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 3 err 55, rtsm_id 5:-1, msg type 2

    Oct 12 12:04:26  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 3 err 55, rtsm_id 0:-1, msg type 10

    Oct 12 12:04:28  CAI-EX2300-VSAT1-AB rpd[14775]: RPD_KRT_Q_RETRIES: Route Update

    : No buffer space available

    Oct 12 12:04:31  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 2 err 55, rtsm_id 0:-1, msg type 6

    Oct 12 12:04:36  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 3 err 55, rtsm_id 5:-1, msg type 2

    Oct 12 12:04:38  CAI-EX2300-VSAT1-AB rpd[14775]: RPD_KRT_Q_RETRIES: Route Update

    : No buffer space available

    Oct 12 12:04:41  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 3 err 55, rtsm_id 0:-1, msg type 10

    Oct 12 12:04:46  CAI-EX2300-VSAT1-AB kernel: rt_pfe_veto: Memory over consumed.

    Op 2 err 55, rtsm_id 27:-1, msg type 5

    Oct 12 12:04:49  CAI-EX2300-VSAT1-AB rpd[14775]: RPD_KRT_Q_RETRIES: Route Update

     

    I could not identify what could be the cause. Any idea where it could come from? 

    Failure is random and occurs once every 1-2 weeks depending the amount of units I have on the systems. 

     

    Thank you

     

    Best regards, 

     



  • 2.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

    Posted 12-11-2017 05:09

    how many routes are are you learning via OSPF and have directly connected? The error sounds like there is no more capacity in the PFE for routing entries.

     

    The EX2300 only handles up to 512 prefixes which is seen in the datasheet on page 6: https://www.juniper.net/assets/us/en/local/pdf/datasheets/1000579-en.pdf



  • 3.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

     
    Posted 12-22-2017 01:55

    Hi Folks,

    Just my 2 cents on this…

     

    If the observation is repeating immediately after reboot; then it could be a scaling issue.

     

    Else, if the objections shows up after couple of days; it could be a memory leak and a hardware BUG.



  • 4.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

    Posted 01-12-2018 01:17

    Hi Python, 

     

    this is not just aftrer rebbot, it's usually happenning after few days which makes it quite difficult to troubleshoot. 

    If it's a memory leak or hardware bug, I believe that I should open a case at JTAC?

     

    Thank you

     

    best regards, 



  • 5.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

     
    Posted 01-12-2018 04:42

    Hi Khrysthof,

    A JTAC case should help in case of memory leak or hardware issue.

     

     



  • 6.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

     
    Posted 01-13-2018 05:06

    Memory leaks would be an issue with the version of Junos not the hardware.

     

    I don't see any memory related PR with OSPF active in the public database.  You could try upgrading to the d57 release before opening a ticket to see if the issue is resolved.  But it is not listed as a resolved issue in the release notes.

     



  • 7.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

    Posted 01-29-2018 07:17

    After investigation with JTAC, I was finally advised to upgrade due to memory leak in sysctl net.routetable/sysctl_rtsock().

     



  • 8.  RE: Juniper EX2300-T : kernel: rt_pfe_veto: Memory over consumed

    Posted 01-12-2018 01:15

    Hi,

     

    Thank you for the answer. About 100 routes max. 

    When I do a show route, I believe that I'm far from the limit. 

     

    best regards,