show chassis cluster information detail no-forwarding Redundancy mode: Configured mode: active-backup Operational mode: active-backup Cluster configuration: Heartbeat interval: 1000 ms Heartbeat threshold: 3 Control link recovery: Enabled Fabric link down timeout: 66 sec Node health information: Local node health: Healthy Remote node health: Healthy Redundancy group: 0, Threshold: 255, Monitoring failures: none Events: Jul 8 01:09:25.250 : hold->secondary, reason: Hold timer expired Jul 8 01:09:26.908 : secondary->primary, reason: Better priority (200/100) Redundancy group: 1, Threshold: 255, Monitoring failures: none Events: Jul 8 01:09:25.260 : hold->secondary, reason: Hold timer expired Jul 8 01:09:26.951 : secondary->primary, reason: Remote yield (0/0) Jul 8 01:14:42.764 : primary->secondary-hold, reason: Monitor failed: IF CS Jul 8 01:14:43.768 : secondary-hold->secondary, reason: Ready to become secondary Jul 12 13:28:48.040 : secondary->primary, reason: Control link failure Control link statistics: Control link 0: Heartbeat packets sent: 19961196 Heartbeat packets received: 389575 Heartbeat packet errors: 0 Duplicate heartbeat packets received: 0 Control link 1: Heartbeat packets sent: 19961196 Heartbeat packets received: 0 Heartbeat packet errors: 0 Duplicate heartbeat packets received: 0 Control recovery packet count: 0 Sequence number of last heartbeat packet sent: 19961196 Sequence number of last heartbeat packet received: 389575 Fabric link statistics: Child link 0 Probes sent: 39981337 Probes received: 780157 Child link 1 Probes sent: 0 Probes received: 0 Chassis cluster LED information: Current LED color: Red Last LED change reason: Peer node: node1 is not present Cold Synchronization: Status: Cold synchronization completed for: FPC 1 PIC 0 Cold synchronization failed for: N/A Cold synchronization not known for: N/A Current Monitoring Weight: 0 Progress: CS Prereq 1 of 1 SPUs completed 1. if_state sync 1 SPUs completed 2. fabric link 1 SPUs completed 3. policy data sync 1 SPUs completed 4. cp ready 1 SPUs completed 5. VPN data sync 1 SPUs completed 6. Dynamic addr sync 1 SPUs completed CS RTO sync 1 of 1 SPUs completed CS Postreq 1 of 1 SPUs completed Statistics: Number of cold synchronization completed: 1 Number of cold synchronization failed: 0 PFE Cold-Sync complete Cold-Sync failed ---------------------------------------------------------------- FPC 1 PIC 0 1 0 Events: Jul 8 01:14:27.015 : Cold sync for PFE FPC 1 PIC 0 is Pre-req check in process Jul 8 01:14:41.626 : Cold sync for PFE FPC 1 PIC 0 is RTO sync in process Jul 8 01:14:53.245 : Cold sync for PFE FPC 1 PIC 0 is Post-req check in process Jul 8 01:14:55.247 : Cold sync for PFE FPC 1 PIC 0 is Completed Loopback Information: PIC Name Loopback Nexthop Mbuf ------------------------------------------------- FPC 1 PIC 0 Success Success Success SPU monitoring: Status: Active Current monitoring weight: 0 PIC Name State ----------------------------- FPC 1 PIC 0 cp-flow combo PIC Up Statistics: SPU up count: 1 NPC up count: 1 SPU down count: 0 NPC down count: 0 Chassis info processing error count: 0 Events: Jul 8 01:11:39.369 : npc PIC Up in slot FPC 3 PIC 0 Jul 8 01:11:48.646 : ioc PIC Up in slot FPC 2 PIC 0 Jul 8 01:12:12.353 : ioc PIC Up in slot FPC 0 PIC 0 Jul 8 01:14:27.017 : cp-flow combo PIC Up in slot FPC 1 PIC 0 Interface monitoring: Statistics: Monitored interface failure count: 1 Events: Jul 12 13:28:59.587 : Interface ge-4/0/2 monitored by rg 1, changed state from Up to Down Fabric monitoring: Status: Fabric Monitoring: Enabled Activation status: Suspended by local node Fabric Status reported by data plane: Down JSRPD internal fabric status: Up Fabric link events: Dec 8 10:05:34.645 : Fabric link fab0 is up Dec 8 10:07:34.115 : Fabric link fab0 is up Dec 8 10:08:20.450 : Fabric link fab0 is up Dec 8 10:08:56.462 : Fabric link fab0 is up Jan 4 19:20:59.219 : Fabric link fab0 is up Jan 6 13:50:16.380 : Fabric link fab0 is up Jan 19 15:11:50.424 : Fabric link fab0 is up Jan 24 00:35:15.931 : Fabric link fab0 is up Feb 23 09:35:34.762 : Fabric link fab0 is up Feb 23 23:06:06.353 : Fabric link fab0 is up Control link status: Down Server information: Server status : Running Server connected to None Client information: Client status : Inactive Client connected to None Control link events: Dec 8 10:05:34.639 : Control link em1 is up Dec 8 10:07:34.104 : Control link em1 is up Dec 8 10:08:20.445 : Control link em1 is up Dec 8 10:08:56.456 : Control link em1 is up Jan 4 19:20:59.212 : Control link em1 is up Jan 6 13:50:16.375 : Control link em1 is up Jan 19 15:11:50.419 : Control link em1 is up Jan 24 00:35:15.926 : Control link em1 is up Feb 23 09:35:34.748 : Control link em1 is up Feb 23 23:06:06.347 : Control link em1 is up Control link heartbeat and jitter events: Jul 8 01:10:12.047 : Noticed jitter of 3803 ms, between 102-103 Hardware monitoring: Status: Activation status: Enabled Redundancy group 0 failover for hardware faults: Enabled Hardware redundancy group 0 errors: 0 Hardware redundancy group 1 errors: 0 Hardware monitoring events: Jul 8 01:08:25.278 : hw-mon errors read RG-0 errors:0 RG-1 errors:0 Schedule monitoring: Status: Activation status: Disabled Schedule slip detected: None Timer ignored: No Statistics: Total slip detected count: 0 Longest slip duration: 0(s) Configuration Synchronization: Status: Activation status: Enabled Last sync operation: Auto-Sync Last sync result: Not needed Last sync mgd messages: Events: Jul 8 01:09:26.949 : Auto-Sync: Not needed. Cold Synchronization Progress: CS Prereq 1 of 1 SPUs completed 1. if_state sync 1 SPUs completed 2. fabric link 1 SPUs completed 3. policy data sync 1 SPUs completed 4. cp ready 1 SPUs completed 5. VPN data sync 1 SPUs completed 6. Dynamic addr sync 1 SPUs completed CS RTO sync 1 of 1 SPUs completed CS Postreq 1 of 1 SPUs completed {primary:node0} Ajayi@TMOMFW> {primary:node0} Ajayi@TMOMFW> {primary:node0} Ajayi@TMOMFW> {primary:node0} Ajayi@TMOMFW> show chassis cluster interfaces Control link status: Down Control interfaces: Index Interface Monitored-Status Internal-SA 0 em0 Down Disabled 1 em1 Down Disabled Fabric link status: Down Fabric interfaces: Name Child-interface Status (Physical/Monitored) fab0 xe-2/0/1 Up / Down fab0 Redundant-ethernet Information: Name Status Redundancy-group reth0 Up 1 reth1 Down Not configured reth2 Down Not configured reth3 Down Not configured reth4 Down Not configured Redundant-pseudo-interface Information: Name Status Redundancy-group lo0 Up 0 Interface Monitoring: Interface Weight Status Redundancy-group ge-0/0/2 255 Up 1 ge-4/0/2 255 Down 1 show chassis cluster stat ^ 'stat' is ambiguous. Possible completions: statistics Display chassis cluster statistics status Display chassis cluster status {primary:node0} Ajayi@TMOMFW> show chassis cluster status Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 1 node0 200 primary no no None node1 0 lost n/a n/a n/a Redundancy group: 1 , Failover count: 3 node0 200 primary no no None node1 0 lost n/a n/a n/a {primary:node0} ============================================================================================================================= Secondary cluster show chassis cluster information detail no-forwarding Redundancy mode: Configured mode: active-backup Operational mode: active-backup Cluster configuration: Heartbeat interval: 1000 ms Heartbeat threshold: 3 Control link recovery: Enabled Fabric link down timeout: 66 sec Node health information: Local node health: Healthy Remote node health: Healthy Redundancy group: 0, Threshold: 255, Monitoring failures: none Events: Jul 8 01:09:26.905 : hold->secondary, reason: Hold timer expired Jul 12 13:28:48.043 : secondary->ineligible, reason: Control link failure Jul 12 13:31:48.046 : ineligible->disabled, reason: Ineligible timer expired Redundancy group: 1, Threshold: 255, Monitoring failures: none Events: Jul 8 01:09:26.922 : hold->secondary, reason: Hold timer expired Jul 8 01:14:42.767 : secondary->primary, reason: Remote is in secondary hold Jul 12 13:28:48.045 : primary->ineligible, reason: Control link failure Jul 12 13:31:48.064 : ineligible->disabled, reason: Ineligible timer expired Control link statistics: Control link 0: Heartbeat packets sent: 19919840 Heartbeat packets received: 389583 Heartbeat packet errors: 0 Duplicate heartbeat packets received: 0 Control link 1: Heartbeat packets sent: 19919840 Heartbeat packets received: 0 Heartbeat packet errors: 0 Duplicate heartbeat packets received: 0 Control recovery packet count: 0 Sequence number of last heartbeat packet sent: 19919840 Sequence number of last heartbeat packet received: 389584 Fabric link statistics: Child link 0 Probes sent: 780164 Probes received: 780151 Child link 1 Probes sent: 0 Probes received: 0 Chassis cluster LED information: Current LED color: Off Last LED change reason: node1 is disabled Cold Synchronization: Status: Cold synchronization completed for: FPC 1 PIC 0 Cold synchronization failed for: N/A Cold synchronization not known for: N/A Current Monitoring Weight: 0 Progress: CS Prereq 1 of 1 SPUs completed 1. if_state sync 1 SPUs completed 2. fabric link 1 SPUs completed 3. policy data sync 1 SPUs completed 4. cp ready 1 SPUs completed 5. VPN data sync 1 SPUs completed 6. Dynamic addr sync 1 SPUs completed CS RTO sync 1 of 1 SPUs completed CS Postreq 1 of 1 SPUs completed Statistics: Number of cold synchronization completed: 1 Number of cold synchronization failed: 0 PFE Cold-Sync complete Cold-Sync failed ---------------------------------------------------------------- FPC 1 PIC 0 1 0 Events: Jul 8 01:14:22.258 : Cold sync for PFE FPC 1 PIC 0 is Pre-req check in process Jul 8 01:14:41.699 : Cold sync for PFE FPC 1 PIC 0 is RTO sync in process Jul 8 01:14:42.123 : Cold sync for PFE FPC 1 PIC 0 is Completed Loopback Information: PIC Name Loopback Nexthop Mbuf ------------------------------------------------- FPC 1 PIC 0 Success Success Success SPU monitoring: Status: Active Current monitoring weight: 0 PIC Name State ----------------------------- FPC 1 PIC 0 cp-flow combo PIC Up Statistics: SPU up count: 1 NPC up count: 1 SPU down count: 0 NPC down count: 0 Chassis info processing error count: 0 Events: Jul 8 01:11:33.863 : npc PIC Up in slot FPC 3 PIC 0 Jul 8 01:11:43.122 : ioc PIC Up in slot FPC 2 PIC 0 Jul 8 01:12:06.639 : ioc PIC Up in slot FPC 0 PIC 0 Jul 8 01:14:22.259 : cp-flow combo PIC Up in slot FPC 1 PIC 0 Interface monitoring: Statistics: Monitored interface failure count: 0 Fabric monitoring: Status: Fabric Monitoring: Enabled Activation status: Suspended by local node Fabric Status reported by data plane: Up JSRPD internal fabric status: Up Fabric link events: Jul 8 01:12:03.433 : Fabric link fab1 is up Jul 8 01:12:03.438 : Fabric link fab1 is up Jul 8 01:12:03.439 : Child xe-6/0/1 of fab1 is up Jul 8 01:12:09.145 : Fabric link fab0 is up Jul 8 01:12:09.150 : Fabric link fab0 is up Jul 8 01:12:09.151 : Child xe-2/0/1 of fab0 is up Jul 8 01:14:30.321 : Fabric monitoring suspension is revoked by remote node Jul 8 01:14:32.129 : Child link-0 of fab1 is up, pfe notification Jul 8 01:14:33.136 : Fabric link up, link status timer Jul 12 13:33:54.329 : Fabric monitoring is suspended due to USPIPC CONNECTION failure Control link status: Down Server information: Server status : Inactive Server connected to None Client information: Client status : Inactive Client connected to None Control link events: Jul 8 01:08:26.810 : Control link em1 is up Jul 8 01:08:32.294 : Control link em0 is up Jul 8 01:08:32.325 : Control link em1 is up Jul 8 01:09:35.541 : Control link em0 is up Jul 8 01:09:35.545 : Control link em1 is up Jul 8 01:09:36.572 : Control link em0 is up Jul 8 01:09:36.578 : Control link em1 is up Jul 8 01:10:07.032 : Control link em1 is up Jul 12 13:28:48.022 : Control link down, link status timer Feb 23 23:07:47.236 : Control link em1 is up Control link heartbeat and jitter events: Jul 8 01:08:27.236 : Missed heartbeats, between 0-2 Jul 8 01:10:12.049 : Noticed jitter of 4459 ms, between 106-107 Hardware monitoring: Status: Activation status: Enabled Redundancy group 0 failover for hardware faults: Enabled Hardware redundancy group 0 errors: 0 Hardware redundancy group 1 errors: 0 Hardware monitoring events: Jul 8 01:08:26.939 : hw-mon errors read RG-0 errors:0 RG-1 errors:0 Jul 12 13:28:50.170 : hw-mon errors notified RG-0 errors:0 RG-1 errors:0 Schedule monitoring: Status: Activation status: Disabled Schedule slip detected: None Timer ignored: No Statistics: Total slip detected count: 1 Longest slip duration: 4(s) Events: Jul 8 01:10:12.043 : Detected schedule slip Jul 8 01:11:12.194 : Cleared schedule slip Configuration Synchronization: Status: Activation status: Enabled Last sync operation: Auto-Sync Last sync result: Succeeded Last sync mgd messages: mgd: rcp: /config/juniper.conf: No such file or directory mgd: commit complete Events: Jul 8 01:09:33.493 : Auto-Sync: In progress. Attempt: 1 Jul 8 01:10:07.587 : Auto-Sync: Clearing mgd. Attempt: 1 Jul 8 01:10:12.036 : Auto-Sync: Succeeded. Attempt: 1 Cold Synchronization Progress: CS Prereq 1 of 1 SPUs completed 1. if_state sync 1 SPUs completed 2. fabric link 1 SPUs completed 3. policy data sync 1 SPUs completed 4. cp ready 1 SPUs completed 5. VPN data sync 1 SPUs completed 6. Dynamic addr sync 1 SPUs completed CS RTO sync 1 of 1 SPUs completed CS Postreq 1 of 1 SPUs completed {disabled:node1} show chassis cluster interfaces Control link status: Down Control interfaces: Index Interface Monitored-Status Internal-SA 0 em0 Down Disabled 1 em1 Down Disabled Fabric link status: Up Fabric interfaces: Name Child-interface Status (Physical/Monitored) fab1 xe-6/0/1 Up / Up fab1 Redundant-ethernet Information: Name Status Redundancy-group reth0 Up 1 reth1 Down Not configured reth2 Down Not configured reth3 Down Not configured reth4 Down Not configured Redundant-pseudo-interface Information: Name Status Redundancy-group lo0 Up 0 Interface Monitoring: Interface Weight Status Redundancy-group ge-0/0/2 255 Up 1 ge-4/0/2 255 Up 1 {disabled:node1} show chassis cluster status Monitor Failure codes: CS Cold Sync monitoring FL Fabric Connection monitoring GR GRES monitoring HW Hardware monitoring IF Interface monitoring IP IP monitoring LB Loopback monitoring MB Mbuf monitoring NH Nexthop monitoring NP NPC monitoring SP SPU monitoring SM Schedule monitoring CF Config Sync monitoring RE Relinquish monitoring Cluster ID: 1 Node Priority Status Preempt Manual Monitor-failures Redundancy group: 0 , Failover count: 0 node0 0 lost n/a n/a n/a node1 100 disabled no no None Redundancy group: 1 , Failover count: 2 node0 0 lost n/a n/a n/a node1 100 disabled no no None {disabled:node1}