Junos OS

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.



Expand all | Collapse all

Best practice - Upgrade EX4300 non mixed VC (5 Member)

  • 1.  Best practice - Upgrade EX4300 non mixed VC (5 Member)

    Posted 06-03-2021 03:06
    Hello,

    what is the best practice to Upgrade a 5 member non mixed virutal chassis (EX4300)?

    Currently the devices running JunOS 18.3R1.9 and I want to upgrade them to 18.4R2S8 as recommended by JTAC (Junos Software Versions - Suggested Releases to Consider and Evaluate - Juniper Networks).

    I have a planned downtime for round about 4 hours. What are the prerequisites I should consider and what is the safest way to upgrade the entive VC - without breaking it - in one step?

    Thank you for your help and best regards,
    Julian


  • 2.  RE: Best practice - Upgrade EX4300 non mixed VC (5 Member)

     
    Posted 06-03-2021 06:02
    There are two options outlined in these kb articles.

    In the first you upgrade all the devices and reboot all when they are complete
    https://kb.juniper.net/InfoCenter/index?page=content&id=KB23122

    In the second you can configure RE failover so there is no forwarding failure as the blades reboot individually.
    https://kb.juniper.net/InfoCenter/index?page=content&id=KB19987&actp=METADATA

    In both cases you should do a storage clean up and the validate run on the switches prior to the window.  This will make sure there is room on the devices and let you know about any potential errors to correct prior to the window.

    https://www.juniper.net/documentation/us/en/software/junos/junos-install-upgrade/topics/concept/junos-software-installation-disk-space-management.html

    request system software validate /path/file

    When complete be sure to run the snapshot to update the backup partition with the new version


    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 3.  RE: Best practice - Upgrade EX4300 non mixed VC (5 Member)

    Posted 06-03-2021 07:27
    Hello Steve,

    thank you for the information you provided.

    I'll use the approach where all devices will be rebooted (https://kb.juniper.net/InfoCenter/index?page=content&id=KB23122).

    Best regards,
    Julian


  • 4.  RE: Best practice - Upgrade EX4300 non mixed VC (5 Member)

    Posted 06-03-2021 10:32
    If its possible to have full downtime, a reboot for all members is always the preffered choice due to minimal issues.
    NSSU always introduce the risk of split brain etc. been there done that ...but usually works fine.
    NSSU for 5 member would take ~2-3h to complete.
    Full downtime of EX4300 would be 15min tops i think.... didnt find the logs from my last upgrade to confirm it.

    The space thing is a good one. usually a bigger issue on EX3400/2300.

    We usually check stuff like:
    show virtual-chassis status
    show system alarms
    show chassis alarms
    show task replication
    request system configuration rescue save
    request system storage cleanup
    Then... Upload the software to /var/tmp/

    For a full downtime, you can install, and see that all goes well, then reboot all members at once when you are ready.... so the installation can be done hours before the actual maintenance window/reboot.


  • 5.  RE: Best practice - Upgrade EX4300 non mixed VC (5 Member)

    Posted 06-04-2021 02:17
    Hey R_J,

    thank you aswell for your input!

    One last question regarding the upgrade. The VC has a license for MACsec - do I need to "reinstall" that license after the upgrade or will be taken over to the new JunOS-Version.

    Best regards,
    Julian


  • 6.  RE: Best practice - Upgrade EX4300 non mixed VC (5 Member)

     
    Posted 06-04-2021 05:53
    Licenses do carry over during upgrades.

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 7.  RE: Best practice - Upgrade EX4300 non mixed VC (5 Member)

    Posted 07-03-2021 04:49
    Heya,

    just for everyone else who might be in the same situation as I was.

    This was the procedure that worked for me:

    Pre checks
    If possible, reboot the entire VC to have a fresh start (I did this not(!) at first)
       > show virtual-chassis status
       > show system alarms
       > show chassis alarms
       > show conf | display set (copy pasta into .txt just to have the config locally - just in case)
       > show version
       > show system snapshot media internal
       > request system configuration rescue save
       > show system storage
       > request system storage cleanup

    Installation:
    copy the Image via sftp into /var/tmp/Junos.tgz
       > request system software add /var/tmp/Junos.tgz
       > request system reboot all-members

    After the reboot check everything like in the "Pre checks". After a few days and if no errors occur dont forget to copy the primary into the backup via "request system snapshot slice alternate all-members".



    Just to be clear. I had to do the "request system software add" two times. At first I had nasty error messages like the following and in the end it errored out:

    Message from syslogd@vx-core-b6-01 at Jul 2 15:53:18 ...
    vx-core-b6-01 fpc3 CMLC: Going disconnected; Routing engine chassis socket closed abruptly

    Message from syslogd@vx-core-b6-01 at Jul 2 15:53:18 ...
    vx-core-b6-01 fpc4 CMLC: Going disconnected; Routing engine chassis socket closed abruptly

    Message from syslogd@vx-core-b6-01 at Jul 2 15:52:39 ...
    vx-core-b6-01 pfex: SCHED: Thread 34 (PFE Manager) aborted, hogged 3160 ms

    Message from syslogd@vx-core-b6-01 at Jul 2 15:52:39 ...
    vx-core-b6-01 fpc0 SCHED: Thread 34 (PFE Manager) aborted, hogged 3160 ms

    Message from syslogd@vx-core-b6-01 at Jul 2 15:58:57 ...
    vx-core-b6-01 pfex: Frame 00: sp = 0x04b97480, pc = 0x0184da7c

    Message from syslogd@vx-core-b6-01 at Jul 2 15:58:57 ...
    vx-core-b6-01 pfex: Frame 01: sp = 0x04b97498, pc = 0x01810948



    I contacted JTAC and I was hitting the following PRs:
    PR 1421550 
    and
    PR 1545530 

    The solution for a nice and smooth upgrade was to reboot the whole VC and issue the "request system software add" command again.

    I hope someone finds this useful.

    Thank you all again for the input and have a wonderful weekend.

    Julian