Junos OS

Hi guys,

We have recently started working with Juniper ACX 5000 series and have encountered a problem after upgrading to Junos 15.1X54-D50.13

The ACX 5048 is brand new, and the only commands that where runned are to change the root password and to add an IP on a interface plus:

request system software add validate reboot source/jinstall-acx5k-15.1X54-D50.13-domestic-signed.tgz force-host

The upgrade whent fine, but now we can't use the "request system snapshot slice alternate" command:

root@host> show version
fpc0:
--------------------------------------------------------------------------
Hostname: host
Model: acx5048
Junos: 15.1X54-D50.13
JUNOS Base OS boot [15.1X54-D50.13]
JUNOS Base OS Software Suite [15.1X54-D50.13]
JUNOS Online Documentation [15.1X54-D50.13]
JUNOS Crypto Software Suite [15.1X54-D50.13]
JUNOS Kernel Software Suite [15.1X54-D50.13]
JUNOS Packet Forwarding Engine Support (acx5k) [15.1X54-D50.13]
JUNOS Routing Software Suite [15.1X54-D50.13]
JUNOS Enterprise Software Suite [15.1X54-D50.13]
JUNOS py-base-i386 [15.1X54-D50.13]
JUNOS Host Software [15.1X54-D50.13]

root@host> request system snapshot
fpc0:
--------------------------------------------------------------------------
Starting snapshot to usb (/dev/da0)
error: Failed to copy snapshot script to the host

{master:0}

root@host> request system snapshot slice alternate
fpc0:
--------------------------------------------------------------------------
Starting snapshot to usb (/dev/da0)
error: Failed to copy snapshot script to the host

{master:0}

The Junos 15.1X54-D50.13 relase notes are clearly stating:

"On ACX5000 series router, you can take a snapshot of the existing Junos OS by inserting an external USB storage device and executing the "request system snapshot slice alternate" command. This command takes a snapshot of the current running Junos OS on to the external USB storage device."

The USB drive I am using it's new and it's detected(I have also tried with another usb drive):

umass0: SanDisk Cruzer Fit, rev 2.10/1.00, addr 3
da0 at umass-sim0 bus 0 target 0 lun 0
da0: <SanDisk Cruzer Fit 1.00> Removable Direct Access SCSI-6 device
da0: 40.000MB/s transfers
da0: 7632MB (15630336 512 byte sectors: 255H 63S/T 972C)

Can anyone give me some hints on what can I do to fix this?

Thank you

Hi,

Could you check for below commands with the USB storage device connected to see if the USB media is accessible:

show system storage partitions
show system snapshot media external
request system snapshot partition

You could also check below in shell:

start shell
mount -l | grep da0

Cheers,

Ashvin 

Hi Ashvin,

I have already tried these commands but here is the output:

root@host> show system storage partitions
error: command is not valid on the acx5048

{master:0}
root@host> show system snapshot media external
error: command is not valid on the acx5048

{master:0}
root@host> request system snapshot partition
fpc0:
--------------------------------------------------------------------------
Starting snapshot to usb (/dev/da0)
error: Failed to copy snapshot script to the host

{master:0}

root@host> start shell
root@host:RE:0% mount -l | grep da0
root@host:RE:0%
root@host:RE:0% exit

I have tried going deeper into the FreeBSD and search the script that is taking care of the "request system snapshot" and found this one: "snapshot-junos-vm"

Going even deeper into the script I have been able to figure out where the script is giving me the error: "error: Failed to copy snapshot script to the host" and found this:

# Create snapshot on the host
    # Copy the vjunos_install script to host
    ${RCP_CMD} $host_snapshot_script $hypervisor_host_ip:$host_snap_script_path/ >> $LOG_FILE 2>&1
    [ $? -ne 0 ] && error "Failed to copy snapshot script to the host"

 After manually replacing the variables I have now:  "rcp -JU __juniper_private4__ /usr/libexec/ui/vjunos-install.sh 192.168.1.1:/var/run/" which seems to be the problem, because the command fails when runned manually:

root@cr1:RE:0% rcp -JU __juniper_private4__ /usr/libexec/ui/vjunos-install.sh 192.168.1.1:/var/run
rcmd: connection timeout

So I am still not sure what is the cause why the RCP command is failing because the linux hypervisor with IP 192.168.1.1 can be succesfuly accesed via SSH or even ping or RSH:

root@host:RE:0% ping -JU __juniper_private4__ 192.168.1.1
PING 192.168.1.1 (192.168.1.1): 56 data bytes
64 bytes from 192.168.1.1: icmp_seq=0 ttl=64 time=1.341 ms
64 bytes from 192.168.1.1: icmp_seq=1 ttl=64 time=0.314 ms
^C
--- 192.168.1.1 ping statistics ---
2 packets transmitted, 2 packets received, 0% packet loss
round-trip min/avg/max/stddev = 0.314/0.828/1.341/0.513 ms

root@host:RE:0% rsh -JU __juniper_private4__ 192.168.1.1 uname -a
Linux localhost 2.6.32-279.22.1.el6.x86_64 #1 SMP Mon May 2 14:09:57 PDT 2016 x86_64 x86_64 x86_64 GNU/Linux

I am still troubleshooting and still can't figure out where the problem is, but maybe has something to do with the RCP command .. 

Thank you

It seems that something in the JunOS config I have is breaking the snapshot script. After removing everything in the config the snapshot command works fine.

So I will have now to see what in my JunOS config is breaking the snapshot, as this may be a bug.

Thank you 

Hi, 

Good work! I was just wondering whether the USB media was mounted.

If not mounted, would the script be copied?

Could you check if the USB media is mounted when the snapshot was successful.

Cheers,

Ashvin

I have finally found what is the problem and it's seems the following command is breaking the snapshot script:

set system internet-options no-tcp-reset drop-all-tcp

Now we are using this command to protect the RE again DDoS(more information can be found here), but this should not be applied to connection between the JunOS and the linux hypervisor. 

As you can see bellow, with the following ACX 5048 config the snapshot script is failing:

root@host> show configuration | display set
set version 15.1X54-D50.13
set system host-name host
set system internet-options no-tcp-reset drop-all-tcp
set system root-authentication encrypted-password "xxx"
{master:0}

root@host> request system snapshot slice alternate
fpc0:
--------------------------------------------------------------------------
Starting snapshot to usb (/dev/da0)
error: Failed to copy snapshot script to the host
{master:0}

And with the "system internet-options no-tcp-reset drop-all-tcp" option removed everything works fine:

root@host> show configuration | display set
set version 15.1X54-D50.13
set system host-name host
set system root-authentication encrypted-password "xxx"
{master:0}
root@host>

root@host> request system snapshot slice alternate
fpc0:
--------------------------------------------------------------------------
Starting snapshot to usb (/dev/da0)
Creating snapshot on the host ..
Copying bootable disk image from host ..
Writing to usb (/dev/da0) ..
Copying 'Host OS' to '/dev/da0s1' .. (this may take a few minutes)
Copying 'JUNOS' to '/dev/da0s1' .. (this may take a few minutes)
The following filesystems were archived: / /config Host-OS
{master:0}

I am not sure if this is a bug, but this command should NOT stop me from doing snapshot on a USB drive that is directly connected to the ACX 5048.

AshvinO: Thank you for your feedback, but as you can see this has nothing to do with the USB mounting.

Later edit: it seems the snapshot command is useless as the ACX 5048 can't boot from a USB drive even when trying the KB27369 . A Juniper JTAC engineer is looking into this matter right now..

Later edit 2: Juniper JTAC found a way to boot from the USB drive on a ACX 5048 using a install-media image or from a system snapshot and the procedure is the following:

• Write the install media image using win32 into the usb and insert it into the router.
• Once the Usb is detected, power cycle the router.
• Once the router starts rebooting, keep hitting “ESC” till you see the boot menu.
• In the boot menu, select boot manager and under boot manager select select USB.
• Under USB, select “INSTALL JUNOS” and press ENTER. (When booting from USB drive after a system snapshot you should see: Install Junos Snapshot) 
• This will boot from USB and again reboot the router automatically with the image in the USB.

You should get something like this:

System Memory Speed : 1333 MHz

Processor Type : Intel(R) Pentium(R) CPU  @ 1.50GHz


Serial ATA Port 4 : SATA SSD
Serial ATA Port 5 : SATA SSD

Other Device    1 : IBA GE Slot 0101 v1350
Other Device    2 : IBA GE Slot 0102 v1350
0C37
Intel(R) Pentium(R) CPU  @ 1.50GHz                                    1.50 GHz
V0018.7                                                            8192 MB RAM




 Continue                                             This selection will take
 Boot Manager                                         you to the Boot Manager
 Device Manager
 Boot From File
 Setup Utility



lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
x                                Boot Manager                                  x
mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

 Boot Option Menu

 USB : USB     DISK
 SSD0 : SATA SSD
 SSD1 : SATA SSD
 IBA GE Slot 0101 v1350
 IBA GE Slot 0102 v1350
 Internal EFI Shell

 ^ and v to change option, ENTER to select an option, ESC to exit



Now booting : USB : USB     DISK

SYSLINUX 4.02 2010-07-21  Copyright (C) 1994-2010 H. Peter Anvin et al
Rebooting...
Starting Junos Installation .......
          lqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqk
          x       Junos Installer - (c) Juniper Networks 2013        x
          tqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqu
          x Reboot                                                   x
          x Install Junos                                            x
          x Boot to host shell [debug]                               x
          x                                                          x
          x                                                          x
          x                                                          x
          x                                                          x
          x                                                          x
          x                                                          x
          x                                                          x
          x                                                          x
          x                                                          x
          mqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqj

                          Press [Tab] to edit options

After this you should have the ACX5048 with a fresh JunOS installation.

Thank you,

Victor Boglea

thanks for posting this guide Victor!