Routing

  • 1.  PEM 0 not OK MX104

    Posted 03-18-2016 07:54

    I have 4 new MX104 routers in the network and all of them generate the above alarm.

    Any one ever encountered this same issue?

     

    integrator@VP_mcRNC_MX-2> show log chassisd | last 50
    Mar 18 17:12:09 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:12:14 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:13:14 send: red alarm clear, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:13:19 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 2)
    Mar 18 17:14:19 tray 0 fan 0 current speed 0x0 requested speed 0x4
    Mar 18 17:24:19 tray 0 fan 0 current speed 0x4 requested speed 0x0
    Mar 18 17:26:59 send: red alarm set, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:26:59 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:27:04 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:27:05 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 6)
    Mar 18 17:27:09 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:27:14 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:28:09 send: red alarm clear, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:28:20 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 2)
    Mar 18 17:31:59 send: red alarm set, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:31:59 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:32:04 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:32:05 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 6)
    Mar 18 17:32:09 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:32:14 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:33:14 send: red alarm clear, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:33:20 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 2)
    Mar 18 17:34:19 tray 0 fan 0 current speed 0x0 requested speed 0x4
    Mar 18 17:44:19 tray 0 fan 0 current speed 0x4 requested speed 0x0
    Mar 18 17:46:59 send: red alarm set, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:46:59 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:47:04 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:47:05 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 6)
    Mar 18 17:47:09 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:47:14 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:48:09 send: red alarm clear, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:48:20 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 2)
    Mar 18 17:52:04 send: red alarm set, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:52:04 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:52:05 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply failed (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 6)
    Mar 18 17:52:09 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:52:14 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:52:19 CHASSISD_PEM_INPUT_BAD: status failure for power supply 0 (status bits: 0x6); check circuit breaker
    Mar 18 17:53:19 send: red alarm clear, device PEM 0, reason PEM 0 Not OK
    Mar 18 17:53:20 CHASSISD_SNMP_TRAP6: SNMP trap generated: Power Supply OK (jnxContentsContainerIndex 2, jnxContentsL1Index 1, jnxContentsL2Index 0, jnxContentsL3Index 0, jnxContentsDescr PEM 0, jnxOperatingState/Temp 2)
    Mar 18 17:54:19 tray 0 fan 0 current speed 0x0 requested speed 0x4

     

    Kind regards,

    Kip.
     



  • 2.  RE: PEM 0 not OK MX104

     
    Posted 03-19-2016 05:50

    If they are using DC power supplies this is a known issue on PR 1064039 where the fans do not detect the temperature correctly and fail to cool the power supply causing the shutdown to occur.

     

    This is fixed in Junos 13.3R7

     

    If you are not in a position to upgrade, JTAC can give you a SLAX script to periodically test the temperature and run the fans.  You may need to adjust the temperature thresholds in this script for your environment to prevent the power supply failures.  We found the settings too high initially but the script does work for systems not yet upgraded.



  • 3.  RE: PEM 0 not OK MX104

    Posted 03-23-2016 06:28

    Hi Steve, Thanks a lot for your feedback. Actually we have version 15.1R2.9 running (Domestic version).

    Is it only resolved on 13.3R7 or on all versions after that including version 15.1? Does domestic version / export version matter?

     

    Kind regards,

    Kip.



  • 4.  RE: PEM 0 not OK MX104

     
    Posted 03-25-2016 11:00

    Looks like they show it resolved in your version.  So yours must be a different issue.

     

    https://prsearch.juniper.net/InfoCenter/index?page=prcontent&id=PR1064039

     

    Resolved In 13.3R7 14.1R6 14.2R5 15.1R2 15.2R1


  • 5.  RE: PEM 0 not OK MX104

    Posted 01-09-2017 12:37

    @spuluka wrote:

    If they are using DC power supplies this is a known issue on PR 1064039 where the fans do not detect the temperature correctly and fail to cool the power supply causing the shutdown to occur.

     

    This is fixed in Junos 13.3R7

     

    If you are not in a position to upgrade, JTAC can give you a SLAX script to periodically test the temperature and run the fans.  You may need to adjust the temperature thresholds in this script for your environment to prevent the power supply failures.  We found the settings too high initially but the script does work for systems not yet upgraded.


    Steve,

     

    What were your preferred temperature thresholds for this script?

     

    Chris



  • 6.  RE: PEM 0 not OK MX104

     
    Posted 01-09-2017 15:56

    I adjusted them experimentally to prevent the PEM error logs.  In the worst case I had to drop the script values by 20 degrees.  This was an enclosed rack setup with front to back cooling while the MX104 is of course side vented.



  • 7.  RE: PEM 0 not OK MX104

    Posted 01-09-2017 18:09

    Hi Chris,

     

    It all depends on adequate air flow and ambient temperature conditions. If you read the PR 1064039, it says following:

    NOTE: In some cases, we may have to use the script after the upgrade. Please make sure to change the thresholds before applying the new script:
    tmp-full: 50
    tmp-int: 45
    tmp-nor: 35

     

    Hope this helps.

     

    Thanks

    Sarathi