Switching

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.



Expand all | Collapse all

EX4500/EX4200 fans

  • 1.  EX4500/EX4200 fans

    Posted 06-30-2021 13:07
    Hello,

    We have a virtual chassis mix of EX4500 and EX4200 models.  Lately we've noticed that the fans have been operating on and off at full speed throughout the day.  Checking the temperatures I see that EX-PFE2 on the devices is operating around 64C/147F which is above the threshold.  I don't see any chassis or system alarms.  The messages log doesn't show anything interesting that I can see.  The switches haven't been physically moved in 7+ years.  Nothing within the controlled environment has changed and if anything it has gotten colder.  The support for these devices ran out a few months ago so I can't open a ticket with JTAC.  New switches are on the way, but with no ETA.  Traffic doesn't appear to be any crazier than usual.  I'm kind of stuck in the troubleshooting steps.  Any suggestions?

    Temp  FPC 0 Rear Left PCB OK 35 degrees C / 95 degrees F
    FPC 0 Rear Middle PCB OK 34 degrees C / 93 degrees F
    FPC 0 Rear Right PCB OK 34 degrees C / 93 degrees F
    FPC 0 PHY0 OK 45 degrees C / 113 degrees F
    FPC 0 PHY6 OK 44 degrees C / 111 degrees F
    FPC 0 PHY12 OK 46 degrees C / 114 degrees F
    FPC 0 PHY19 OK 54 degrees C / 129 degrees F
    FPC 1 CPU OK 28 degrees C / 82 degrees F
    FPC 1 EX-PFE1 OK 45 degrees C / 113 degrees F
    FPC 1 EX-PFE2 OK 65 degrees C / 149 degrees F
    FPC 1 EX-PFE3 OK 51 degrees C / 123 degrees F
    FPC 1 GEPHY Front Left OK 32 degrees C / 89 degrees F
    FPC 1 GEPHY Front Middle OK 40 degrees C / 104 degrees F
    FPC 1 GEPHY Front Right OK 41 degrees C / 105 degrees F
    FPC 1 Uplink Conn OK 39 degrees C / 102 degrees F
    FPC 2 Rear Left PCB OK 37 degrees C / 98 degrees F
    FPC 2 Rear Middle PCB OK 37 degrees C / 98 degrees F
    FPC 2 Rear Right PCB OK 37 degrees C / 98 degrees F
    FPC 2 PHY0 OK 38 degrees C / 100 degrees F
    FPC 2 PHY6 OK 38 degrees C / 100 degrees F
    FPC 2 PHY12 OK 38 degrees C / 100 degrees F
    FPC 2 PHY19 OK 44 degrees C / 111 degrees F
    FPC 3 CPU OK 33 degrees C / 91 degrees F
    FPC 3 EX-PFE1 OK 45 degrees C / 113 degrees F
    FPC 3 EX-PFE2 OK 64 degrees C / 147 degrees F
    FPC 3 EX-PFE3 OK 51 degrees C / 123 degrees F
    FPC 3 GEPHY Front Left OK 32 degrees C / 89 degrees F
    FPC 3 GEPHY Front Middle OK 41 degrees C / 105 degrees F
    FPC 3 GEPHY Front Right OK 39 degrees C / 102 degrees F
    FPC 3 Uplink Conn OK 36 degrees C / 96 degrees F
    FPC 4 CPU OK 39 degrees C / 102 degrees F
    FPC 4 EX-PFE1 OK 47 degrees C / 116 degrees F
    FPC 4 EX-PFE2 OK 65 degrees C / 149 degrees F
    FPC 4 EX-PFE3 OK 52 degrees C / 125 degrees F
    FPC 4 GEPHY Front Left OK 34 degrees C / 93 degrees F
    FPC 4 GEPHY Front Middle OK 44 degrees C / 111 degrees F
    FPC 4 GEPHY Front Right OK 42 degrees C / 107 degrees F
    FPC 4 Uplink Conn OK 38 degrees C / 100 degrees F
    FPC 5 CPU OK 32 degrees C / 89 degrees F
    FPC 5 EX-PFE1 OK 45 degrees C / 113 degrees F
    FPC 5 EX-PFE2 OK 63 degrees C / 145 degrees F
    FPC 5 EX-PFE3 OK 52 degrees C / 125 degrees F
    FPC 5 GEPHY Front Left OK 31 degrees C / 87 degrees F
    FPC 5 GEPHY Front Middle OK 37 degrees C / 98 degrees F
    FPC 5 GEPHY Front Right OK 33 degrees C / 91 degrees F
    FPC 5 Uplink Conn OK 35 degrees C / 95 degrees F

       Fans FPC 0 Fan 1 OK Spinning at normal speed
    FPC 0 Fan 2 OK Spinning at normal speed
    FPC 0 Fan 3 OK Spinning at normal speed
    FPC 0 Fan 4 OK Spinning at normal speed
    FPC 0 Fan 5 OK Spinning at normal speed
    FPC 0 Fan-Tray Airflow OK Back to front
    FPC 1 Fan 1 OK Spinning at full speed
    FPC 1 Fan 2 OK Spinning at full speed
    FPC 1 Fan 3 OK Spinning at full speed
    FPC 2 Fan 1 OK Spinning at normal speed
    FPC 2 Fan 2 OK Spinning at normal speed
    FPC 2 Fan 3 OK Spinning at normal speed
    FPC 2 Fan 4 OK Spinning at normal speed
    FPC 2 Fan 5 OK Spinning at normal speed
    FPC 2 Fan-Tray Airflow OK Back to front
    FPC 3 Fan 1 OK Spinning at normal speed
    FPC 3 Fan 2 OK Spinning at normal speed
    FPC 3 Fan 3 OK Spinning at normal speed
    FPC 4 Fan 1 OK Spinning at full speed
    FPC 4 Fan 2 OK Spinning at full speed
    FPC 4 Fan 3 OK Spinning at full speed
    FPC 5 Fan 1 OK Spinning at normal speed
    FPC 5 Fan 2 OK Spinning at normal speed
    FPC 5 Fan 3 OK Spinning at normal speed


  • 2.  RE: EX4500/EX4200 fans

     
    Posted 06-30-2021 13:50
    I'm not sure about the 4500's, but on the 4200's:

    1. Check that the air intakes on both sides of the chassis are free of  dust or other obstruction (see extreme example below)
    2. Remove the fan tray from the rear (see https://www.juniper.net/documentation/en_US/release-independent/junos/topics/topic-map/ex4200-cooling-maintain.html) and make sure the three fans are also clean and free of dust or debris and turn freely.




  • 3.  RE: EX4500/EX4200 fans

     
    Posted 07-01-2021 01:27
    I just reread the output more carefully--the threshold for a PFE2 yellow alarm is 80ºC, so those temperatures are well within the normal range. It doesn't explain why the fans speeds are cycling however.

    user@switch> show chassis temperature-thresholds
    Fan speed Yellow alarm Red alarm Item Normal High Normal Bad fan Normal Bad fan FPC 0 CPU 60 70 80 70 95 85 FPC 0 EX-PFE1 60 70 80 70 95 85 FPC 0 EX-PFE2 60 70 80 70 95 85 FPC 0 GEPHY Front Left 60 70 80 70 95 85 FPC 0 GEPHY Front Middle 60 70 80 70 95 85 FPC 0 GEPHY Front Right 60 70 80 70 95 85 FPC 0 Uplink Conn 60 70 80 70 95 85



  • 4.  RE: EX4500/EX4200 fans

    Posted 07-30-2021 05:38
    PFE 2 serves interfaces 24-47 and will be located in the middle of the switch. (PFE1 = 0-23, PFE3 = PIC1) I take it that the problem is seen on the EX4200's and not the EX4500's, correct?

    The threshold for increasing the fan speed is 70 degrees. Only when the temperature drops below 60 will the fans settle down. Some of your temps have been at 70 or above, triggering the increase in fan speed, but haven't yet cooled the sensors to 60 degrees. This explains the fact that some fans run high at 65 degrees and some run low on 64.

    As you have no service or warranty on these, I'd recommend you to open them up and clearing the dust inside (using some ESD protection of course). At least inspect the insides of them.

    I also see the increase in temperature in PFE2 (and a little in PHY2, also in the middle):

    FPC 0 EX-PFE1 OK 40 degrees C / 104 degrees F
    FPC 0 EX-PFE2 OK 56 degrees C / 132 degrees F
    FPC 0 EX-PFE3 OK 43 degrees C / 109 degrees F
    FPC 0 GEPHY Front Left OK 28 degrees C / 82 degrees F
    FPC 0 GEPHY Front Middle OK 34 degrees C / 93 degrees F
    FPC 0 GEPHY Front Right OK 32 degrees C / 89 degrees F
    FPC 0 Uplink Conn OK 31 degrees C / 87 degrees F
    FPC 1 CPU OK 41 degrees C / 105 degrees F
    FPC 1 EX-PFE1 OK 45 degrees C / 113 degrees F
    FPC 1 EX-PFE2 OK 56 degrees C / 132 degrees F
    FPC 1 EX-PFE3 OK 48 degrees C / 118 degrees F

    I think your switches are helathy and behaving correctly. If the fan noise bothers you, clean the fans properly (it might give that extra little airflow you need) and use a little bit of silicone spray to remove some friction inside them to make them run smoother. If you notice that some switches always run hotter than the others, swap the fan packages to see if the high temps follow the fan packages. Perhaps your new switches (refurb I suspect) will bring better fan trays so you don't need to replace the switches but only the fans.

    The active master in the VC will have the most CPU usage. Is that an EX4500 or an EX4200? The 4500 will handle the temps better so make one of those the master.


  • 5.  RE: EX4500/EX4200 fans

    Posted 06-30-2021 17:34
    Hey smicker,

    I checked all intakes and fans.  Everything seems to be in working order.  There was little to no dust build-up on any of them.  It's a fairly clean environment they are in.


  • 6.  RE: EX4500/EX4200 fans

    Posted 07-01-2021 17:59
    You may not be able to do much about the fans themselves, but could you set up SNMP with alerts so you know if the temp gets too high?
    It's not a solution, but may bring peace of mind until the new switches are in.