Security Management

Hi,

 

When tring to backup the NSM device we receive the following error:

 

[admin@Nsm-Mng ~]$  su nsm /usr/netscreen/HaSvr/utils/replicateDb backup
Password:
 Got arguments: backup.  This might take a while to process ...
    Ha/Backup: FAIL

 

Anyone here is familiar with this error message ?

Hi,

Not sure but please become a root user first using sudo su -
And then become nsm user and run again.

But the reason of failure should be in the /usr/netscreen/HaSvr/bin/.backupDoLocal.result file.

I went through that also .. it did not resolve the issue
also there is no file /usr/netscreen/HaSvr/bin/.backupDoLocal.result

[admin@Nsm-Mng ~]$ sudo su
Password:
[root@Nsm-Mng admin]# sudo -u nsm /usr/netscreen/HaSvr/utils/replicateDb backup
Got arguments: backup. This might take a while to process ...
Ha/Backup: FAIL

cd /usr/netscreen/HaSvr/bin/
[root@Nsm-Mng bin]# ls
haSvr.sh haUtilClient highAvail.sh highAvailSvr.sh

I've tried via another method and it pass successfully:

[root@Nsm-Mng ~]# /etc/init.d/haSvr stop
nsm owner is nsm
Stopping apps...
[root@Nsm-Mng ~]# /etc/init.d/devSvr stop
nsm owner is nsm
Stopping apps...
Stopped devSvrManager as nsm (pid 25175)
Stopped devSvrLogWalker as nsm (pid 25338)
Stopped devSvrDataCollector as nsm (pid 25528)
Stopped devSvrDirectiveHandler as nsm (pid 25802)
Stopped devSvrProfilerMgr as nsm (pid 25979)
Stopped devSvrDbSvr (pid 24967)
Stopped devSvrStatusMonitor as nsm (pid 27202)
Stopped devSvrTFTP as nsm (pid 27436)
[root@Nsm-Mng ~]# /etc/init.d/guiSvr stop
nsm owner is nsm
Stopping apps...
Stopped guiSvrManager as nsm (pid 23275)
Stopped guiSvrMasterController as nsm (pid 24016)
Stopped guiSvrDirectiveHandler as nsm (pid 24203)
Stopped guiSvrLicenseManager as nsm (pid 24385)
Stopped guiSvrStatusMonitor as nsm (pid 24539)
Stopped guiSvrWebProxy as nsm (pid 24755)
[root@Nsm-Mng ~]# /usr/netscreen/GuiSvr/utils/tech-support.sh db
Provide the /usr/netscreen/GuiSvr/var/GuiSvrDB201706080016.tar.gz to your Support Engineer
[root@Nsm-Mng ~]# /etc/init.d/guiSvr start
nsm owner is nsm
Starting apps...
Starting guiSvrMasterController as nsm.............OK
Starting guiSvrDirectiveHandler as nsm.............OK
Starting guiSvrLicenseManager as nsm...............OK
Starting guiSvrStatusMonitor as nsm................OK
Starting guiSvrWebProxy as nsm.....................OK
[root@Nsm-Mng ~]# /etc/init.d/devSvr start
nsm owner is nsm
Starting apps...
Starting devSvrProfilerMgr as nsm..................OK
Starting devSvrStatusMonitor as nsm................OK
Starting devSvrTFTP as nsm.........................OK
[root@Nsm-Mng ~]# /etc/init.d/haSvr start
nsm owner is nsm
Starting apps...
Starting highAvail as nsm..........................OK
Starting highAvailSvr as nsm.......................OK

Do you know what is the different between this method and the method I mentioned earlier ?

Hi,

/usr/netscreen/HaSvr/bin/.backupDoLocal.result

Above file is hidden so try ls -al to list this file.

Tech-support.sh and replicateDb both are different scripts to collect database.

tech-support doesnt collect non-db file such as schemas where as replicatDB collects it, you can extract and see the difference in more detail.

For replicatedb, check the size for /var partition if it is enough.

Also refer : https://kb.juniper.net/InfoCenter/index?page=content&id=KB11809&actp=METADATA

[root@Nsm-Mng bin]# cat .backupDoLocal.result
rsync: writefd_unbuffered failed to write 86 bytes: phase "unknown" [generator]: Broken pipe (32)
rsync error: error in rsync protocol data stream (code 12) at io.c(909)
doBackup returned FAILURE

 

[root@Nsm-Mng var]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda2              29G  177M   28G   1% /
none                  4.0G     0  4.0G   0% /dev/shm
/dev/sda11             49G   98M   46G   1% /tmp
/dev/sda10             97G  6.2G   86G   7% /usr
/dev/sda9             353G   99G  237G  30% /var
/dev/sda7             241G   93M  229G   1% /var/cores
/dev/sda5             616G   13G  572G   3% /var/netscreen/DevSvr
/dev/sda6             308G  129M  292G   1% /var/netscreen/DevSvr/profiler_data
/dev/sda8              97G   34G   58G  37% /var/netscreen/GuiSvr

 

[root@Nsm-Mng bin]# rsync --version 
rsync  version 2.6.3  protocol version 28
Copyright (C) 1996-2004 by Andrew Tridgell and others
<http://rsync.samba.org/>
Capabilities: 64-bit files, socketpairs, hard links, symlinks, batchfiles, 
              inplace, IPv6, 64-bit system inums, 64-bit internal inums

rsync comes with ABSOLUTELY NO WARRANTY.  This is free software, and you
are welcome to redistribute it under certain conditions.  See the GNU
General Public Licence for details.

Hi,

 

I tried the following:

 

[root@Nsm-Mng admin]# sudo chown -R nsm /usr/netscreen/HaSvr/utils/*

[root@Nsm-Mng admin]# sudo su - nsm
[nsm@Nsm-Mng ~]$ chmod 777 /usr/netscreen/HaSvr/utils/*
[nsm@Nsm-Mng ~]$ chmod 777 /usr/netscreen/HaSvr/*
[nsm@Nsm-Mng ~]$ chmod 777 /usr/netscreen/*

 

And still getting the error:

 

[admin@Nsm-Mng ~]$ sudo -u nsm /usr/netscreen/HaSvr/utils/replicateDb backup
Password:
 Got arguments: backup.  This might take a while to process ...
     Ha/Backup: FAIL

 

But the error message changed to :

 

[nsm@Nsm-Mng bin]$ cat .backupDoLocal.result
doBackup returned SUCCESS

 

This is wierd ...

It is might getting time out.
Try to Change the timeout value in haSvr.cfg

I've changed it from 1800 to 10800
both :

highAvail.rsyncCommandBackupTimeout 10800
highAvail.rsyncCommandReplicationTimeout 10800

Still getting the same error:
[admin@Nsm-Mng ~]$ su nsm /usr/netscreen/HaSvr/utils/replicateDb backup
Password:
Got arguments: backup. This might take a while to process ...
Ha/Backup: FAIL

I like to use this procedure based on kb122851 to get a full backup of the NSM database for DR purposes.

 

ï SUDO to receive the proper permissions
sudo su -
ï Stop the services server
cd /etc/rc3.d/
./S33haSvr stop
./S32guiSvr stop
./S33devSvr stop

cd /usr/netscreen/GuiSvr/utils

./tech-support.sh db

copy out file from folder and name displayed

cd /usr/netscreen/GuiSvr/var/

cp FILENAME /home/admin

delete file after copy out

rm FILENAME

Transfer file /home/admin here to backup volume and remove from home

start services

1. Start Gui Server -- /etc/init.d/guiSvr start
2. Start Device Server -- /etc/init.d/devSvr start
3. Start HA Server -- /etc/init.d/haSvr start

ï Verify the Dev and GUI Services have started successfully
/etc/init.d/guiSvr status
/etc/init.d/devSvr status

Hi,

But PML said that tech-support is not a full backup .

"tech-support doesnt collect non-db file such as schemas where as replicatDB collects it"

How much it is important it is to backup schemas and non-db files in case we needed to backup the NSM ? How much those files essential ?

If you are planning to restore the db backup on the same server it doesn't matter if you dont take schema backup as server already have it.
If not also we can upgrade schema later on.
Main directory is /usr/netscreen/GuiSvr/var/xdb which contains db.

So, basically, exporting tech-support file and exporting xdb directory should be fine in case we want to restore the files on the same server .
- Is it also correct if the NSM is physical machine (not VM) ?

Yes, you are correct and platform doesn't matter.

Thanks ! Your help is appreciated !!
I'll test it first then we'll drop a script in our Backbox machine to back the NSM up 🙂
Thanks PML , Thank you Steve 🙂

Hi Abed,

 

Please let us know the exact steps you followed to restore also attach the logs using http://kb.juniper.net/InfoCenter/index?page=content&id=KB24982 

 

-PL

We have done some extra-commands and now the NSM Gui in not workig at all.. Only SSH is working ..

When starting the guiSvr is says:

Starting guiSvrStatusMonitor as nsm............failed
Starting guiSvrWebProxy as nsm.................failed

This is the script we have done:

 

connect

mkdir -p /var/BackBox/xdb/

rm -rf /var/netscreen/dbbackup/backup*

rm -rf /var/netscreen/GuiSvr/*.tar.gz

rm -rf /usr/netscreen/GuiSvr/*.tar.gz

rm -rf /usr/netscreen/dbbackup/backup*

cd /etc/rc3.d/

./S33haSvr stop

./S32guiSvr stop

./S33devSvr stop

cd /usr/netscreen/GuiSvr/utils

./tech-support.sh db

mv -f /var/netscreen/dbbackup/backup* /var/BackBox

mv -f /usr/netscreen/GuiSvr/var/* /var/BackBox

cp -Rf /usr/netscreen/GuiSvr/var/xdb/ /var/BackBox

cd /var/BackBox

getlastline

tar -pczf /var/BackBox/NSM_Backup.tar.gz *

tar -tzf /var/BackBox/NSM_Backup.tar.gz

/etc/init.d/guiSvr start

/etc/init.d/devSvr start

/etc/init.d/haSvr start

exit

exit

scp %%USER%%@%%HOST%%:/var/BackBox/NSM_Backup.tar.gz %%DEFAULT_BACKUP_LOCATION%%/

yes

%%PASSWORD%%

connect

inventory

rm -rf /var/BackBox

rm -rf /var/netscreen/dbbackup/backup*

tar -tzf %%DEFAULT_BACKUP_LOCATION%%/NSM_Backup.tar.gz

Hi Abed,

 

When you ran : 

rm -rf /var/netscreen/GuiSvr/*.tar.gz

rm -rf /usr/netscreen/GuiSvr/*.tar.gz

 

You have deleted all NSM application from server so server service won't start.

 

Do you currenly have xdb directory?

 

ls /usr/netscreen/GuiSvr/

ls /var/netscreen/GuiSvr/

 

-PL

[root@Nsm-Mng rc3.d]# ls /usr/netscreen/GuiSvr/
bin doc lib nbiservice README utils var xdbservice
[root@Nsm-Mng rc3.d]# ls /var/netscreen/GuiSvr/
errorLog var webproxy xdb

Hi A'bed,

 

From working server:

 

[root@localhost GuiSvr]# ls /usr/netscreen/GuiSvr/

bin  doc  lib  nbiservice  README  utils  var  xdbservice

[root@localhost GuiSvr]# 

[root@localhost GuiSvr]#  ls /var/netscreen/GuiSvr/

attack   be   ConfigFileVersions  di_engines dmi-schema-stage  fifos     guiSvr.cfg  migration  pids Schemas-GDH  sec-update  svrcli    xdb

auditLog  certDB  data       dmi-schema-backup  errorLog   firmware  license misc     pro.mc.lcf scripts      svnconfig webproxy

[root@localhost GuiSvr]# 

 

So most of the files/directory is deleted from your server as a result services are not starting, you will have to rollback or install new server(same version) and restore xdb directory.

 

-PL

First those two commands :
rm -rf /var/netscreen/GuiSvr/*.tar.gz
rm -rf /usr/netscreen/GuiSvr/*.tar.gz
only deleting tar.gz files .. not all files .. correct me if I'm wrong

Second, we're follwing now this procedure: KB15643

I'll update you when done

I can't believe the issue has been resolved !! NSM back to work ! 🙂
Only one issue is lefted which is this process :
[root@Nsm-Mng ~]# /etc/init.d/devSvr start
nsm owner is nsm
Starting apps...
Starting devSvrStatusMonitor as nsm................OK
Starting devSvrTFTP as nsm.....................failed

The TFTP process is not going and I'm still troubleshooting it

https://kb.juniper.net/InfoCenter/index?page=content&id=KB26880

Thanks again !