Switching

Expand all | Collapse all

EX4300 - Check-out failed for Chassis control process

  • 1.  EX4300 - Check-out failed for Chassis control process

    Posted 05-10-2019 12:18

    Hi,

     

    I installed a pair of EX4300 a few weeks ago. Very simple setup, and using virtual chassis.

     

    Today I had to make a few changes to the configuration, but commit failed:

     

    root@tpsw01# commit
    fpc0:
    configuration check succeeds
    fpc1:
    error: Check-out failed for Chassis control process (/usr/sbin/chassisd) without details
    error: configuration check-out failed
    fpc0:
    error: remote commit-configuration failed on fpc1
    error: commit failed

    The log doesn't show much interesting:

    tpsw01 mgd[10495]: UI_COMMIT: User 'root' requested 'commit' operation (comment: none)
    tpsw01 mgd[10495]: UI_CHILD_EXITED: Child exited: PID 11979, status 7, command '/usr/sbin/mustd'

    Both switches are running the same version of Junos, 17.3R3-S4.2:

     

    root@tpsw01# run show system snapshot media internal
    fpc0:
    --------------------------------------------------------------------------
    Information for snapshot on       internal (/dev/da0s1a) (primary)
    Creation date: Apr 10 21:10:46 2019
    JUNOS version on snapshot:
      jcrypto-ex: 17.3R3-S4.2
      jdocs-ex: 17.3R3-S4.2
      jsd    : powerpc-17.3R3-S4.2-jet-1
      jsdn-powerpc: 17.3R3-S4.2
      junos  : ex-17.3R3-S4.2
      junos-ex-4300: 17.3R3-S4.2
      jweb-ex: 17.3R3-S4.2
    Information for snapshot on       internal (/dev/da0s2a) (backup)
    Creation date: Apr 17 20:57:12 2019
    JUNOS version on snapshot:
      jcrypto-ex: 17.3R3-S4.2
      jdocs-ex: 17.3R3-S4.2
      jsd    : powerpc-17.3R3-S4.2-jet-1
      jsdn-powerpc: 17.3R3-S4.2
      junos  : ex-17.3R3-S4.2
      junos-ex-4300: 17.3R3-S4.2
      jweb-ex: 17.3R3-S4.2
    
    fpc1:
    --------------------------------------------------------------------------
    Information for snapshot on       internal (/dev/da0s1a) (primary)
    Creation date: Apr 10 21:10:45 2019
    JUNOS version on snapshot:
      jcrypto-ex: 17.3R3-S4.2
      jdocs-ex: 17.3R3-S4.2
      jsd    : powerpc-17.3R3-S4.2-jet-1
      jsdn-powerpc: 17.3R3-S4.2
      junos  : ex-17.3R3-S4.2
      junos-ex-4300: 17.3R3-S4.2
      jweb-ex: 17.3R3-S4.2
    Information for snapshot on       internal (/dev/da0s2a) (backup)
    Creation date: Apr 17 20:58:13 2019
    JUNOS version on snapshot:
      jcrypto-ex: 17.3R3-S4.2
      jdocs-ex: 17.3R3-S4.2
      jsd    : powerpc-17.3R3-S4.2-jet-1
      jsdn-powerpc: 17.3R3-S4.2
      junos  : ex-17.3R3-S4.2
      junos-ex-4300: 17.3R3-S4.2
      jweb-ex: 17.3R3-S4.2

    Any ideas?

     

    I tried to do a "commit full" but with the same error message.



  • 2.  RE: EX4300 - Check-out failed for Chassis control process

    Posted 05-10-2019 14:45

    hi there,

     

    can you show us the output of:

    #commit | display detail
    

    it seems to be FPC1 thats not committing this might give us a better lead



  • 3.  RE: EX4300 - Check-out failed for Chassis control process

    Posted 05-14-2019 04:32

    OK, here's the output:

     

    root@tpsw01# commit | display detail
    2019-04-21 18:43:47 UTC: Obtaining lock for commit
    2019-04-21 18:43:47 UTC: updating commit revision
    2019-04-21 18:43:47 UTC: obtaining db lock on fpc1
    2019-04-21 18:43:48 UTC: re-revision: fpc0-1555534868-23, other-re-revision: fpc0-1555534868-23(0)
    2019-04-21 18:43:48 UTC: UI extensions feature is not configured
    2019-04-21 18:43:48 UTC: Started running translation script
    2019-04-21 18:43:48 UTC: Finished running translation script
    2019-04-21 18:43:48 UTC: start loading commit script changes
    2019-04-21 18:43:48 UTC: no commit script changes
    2019-04-21 18:43:48 UTC: no transient commit script changes
    2019-04-21 18:43:48 UTC: finished loading commit script changes
    2019-04-21 18:43:48 UTC: No translation output from the scripts
    2019-04-21 18:43:48 UTC: Preparing Fast-diff post translation load
    2019-04-21 18:43:48 UTC: copying juniper.db to juniper.data+
    2019-04-21 18:43:48 UTC: finished copying juniper.db to juniper.data+
    2019-04-21 18:43:48 UTC: exporting juniper.conf
    2019-04-21 18:43:48 UTC: expanding interface-ranges
    2019-04-21 18:43:48 UTC: finished expanding interface-ranges
    2019-04-21 18:43:48 UTC: expanding groups
    2019-04-21 18:43:48 UTC: finished expanding groups
    2019-04-21 18:43:48 UTC: setup foreign files
    2019-04-21 18:43:48 UTC: update license counters
    2019-04-21 18:43:48 UTC: finish license counters
    2019-04-21 18:43:48 UTC: propagating foreign files
    2019-04-21 18:43:49 UTC: cdg returns = 7(persist groups is not configured (needed for cdg))
    2019-04-21 18:43:49 UTC: complete foreign files
    2019-04-21 18:43:49 UTC: dropping unchanged foreign files
    2019-04-21 18:43:49 UTC: executing 'ffp propagate'
    2019-04-21 18:43:50 UTC: daemons checking new configuration
    configuration check succeeds
    2019-04-21 18:43:50 UTC: executing 'ffp synchronize'
    2019-04-21 18:43:50 UTC: copying file '//var/etc/stp_port.id' to 'fpc1'
    2019-04-21 18:43:50 UTC: copying file '//var/etc/non_stp_port.id' to 'fpc1'
    2019-04-21 18:43:51 UTC: copying file '//var/etc/stp_all_port.id' to 'fpc1'
    2019-04-21 18:43:51 UTC: copying file '//var/etc/if_alias_map+' to 'fpc1'
    2019-04-21 18:43:51 UTC: sending pull-configuration rpc to fpc1
    2019-04-21 18:43:51 UTC: filename /var/run/db/juniper.db-patch.sync, size 0
    2019-04-21 18:43:52 UTC: pull-configuration success. URL: /var/tmp/juniper.db-patch.sync
    2019-04-21 18:43:52 UTC: sending load-patch rpc to fpc1
    2019-04-21 18:43:52 UTC: remote load-configuration success on fpc1
    2019-04-21 18:43:52 UTC: sending file-delete rpc to fpc1
    2019-04-21 18:43:52 UTC: asking fpc1 to commit
    2019-04-21 18:43:52 UTC: syncing commit db revision to  fpc1
    2019-04-21 18:43:52 UTC: waiting for commit reply from fpc1
    fpc1:
    2019-04-21 18:43:52 UTC: Obtaining lock for commit
    2019-04-21 18:43:52 UTC: updating commit revision
    2019-04-21 18:43:52 UTC: Started running translation script
    2019-04-21 18:43:52 UTC: Finished running translation script
    2019-04-21 18:43:52 UTC: start loading commit script changes
    2019-04-21 18:43:52 UTC: no commit script changes
    2019-04-21 18:43:52 UTC: no transient commit script changes
    2019-04-21 18:43:52 UTC: finished loading commit script changes
    2019-04-21 18:43:52 UTC: No translation output from the scripts
    2019-04-21 18:43:52 UTC: copying juniper.db to juniper.data+
    2019-04-21 18:43:52 UTC: finished copying juniper.db to juniper.data+
    2019-04-21 18:43:52 UTC: exporting juniper.conf
    2019-04-21 18:43:52 UTC: expanding interface-ranges
    2019-04-21 18:43:52 UTC: finished expanding interface-ranges
    2019-04-21 18:43:52 UTC: expanding groups
    2019-04-21 18:43:52 UTC: finished expanding groups
    2019-04-21 18:43:52 UTC: setup foreign files
    2019-04-21 18:43:52 UTC: update license counters
    2019-04-21 18:43:52 UTC: finish license counters
    2019-04-21 18:43:52 UTC: propagating foreign files
    2019-04-21 18:43:53 UTC: cdg returns = 7(persist groups is not configured (needed for cdg))
    2019-04-21 18:43:53 UTC: complete foreign files
    2019-04-21 18:43:53 UTC: dropping unchanged foreign files
    2019-04-21 18:43:53 UTC: executing 'ffp propagate'
    2019-04-21 18:43:54 UTC: daemons checking new configuration
    2019-04-21 18:43:54 UTC: sending commit-check command to Firewall process(1601)
    2019-04-21 18:43:54 UTC: Collecting status of Firewall process
    2019-04-21 18:43:54 UTC: Spawning Init daemon to check new configuration
    2019-04-21 18:43:54 UTC: Collecting status of Init daemon
    2019-04-21 18:43:54 UTC: Spawning Routing protocols process to check new configuration
    2019-04-21 18:43:55 UTC: Collecting status of Routing protocols process
    2019-04-21 18:43:55 UTC: Spawning Chassis control process to check new configuration
    2019-04-21 18:43:55 UTC: Collecting status of Chassis control process
    error: Check-out failed for Chassis control process (/usr/sbin/chassisd) without details
    2019-04-21 18:43:55 UTC: Spawning Booting daemon to check new configuration
    2019-04-21 18:43:55 UTC: Collecting status of Booting daemon
    2019-04-21 18:43:55 UTC: sending commit-check command to Interface control process(1614)
    2019-04-21 18:43:55 UTC: Collecting status of Interface control process
    2019-04-21 18:43:55 UTC: Spawning Simple Network Management Protocol process to check new configuration
    2019-04-21 18:43:55 UTC: Collecting status of Simple Network Management Protocol process
    2019-04-21 18:43:55 UTC: Spawning Management Information Base II process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Management Information Base II process
    2019-04-21 18:43:56 UTC: Spawning Alarm control process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Alarm control process
    2019-04-21 18:43:56 UTC: Spawning Packet Forwarding Engine statistics management process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Packet Forwarding Engine statistics management process
    2019-04-21 18:43:56 UTC: Spawning Link Aggregation Control Protocol process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Link Aggregation Control Protocol process
    2019-04-21 18:43:56 UTC: Spawning Shared memory routing socket message database process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Shared memory routing socket message database process
    2019-04-21 18:43:56 UTC: Spawning Subscriber management process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Subscriber management process
    2019-04-21 18:43:56 UTC: Spawning Logical system multiplexer process to check new configuration
    2019-04-21 18:43:56 UTC: Collecting status of Logical system multiplexer process
    2019-04-21 18:43:56 UTC: Spawning Bidirectional Forwarding Detection process to check new configuration
    2019-04-21 18:43:57 UTC: Collecting status of Bidirectional Forwarding Detection process
    2019-04-21 18:43:57 UTC: Spawning Event processing process to check new configuration
    2019-04-21 18:43:57 UTC: Collecting status of Event processing process
    2019-04-21 18:43:57 UTC: Spawning Layer 2 Control Protocol process to check new configuration
    2019-04-21 18:43:57 UTC: Collecting status of Layer 2 Control Protocol process
    2019-04-21 18:43:57 UTC: Spawning Feature license management process to check new configuration
    2019-04-21 18:43:57 UTC: Collecting status of Feature license management process
    2019-04-21 18:43:57 UTC: Spawning Chassis Manager to check new configuration
    2019-04-21 18:43:57 UTC: Collecting status of Chassis Manager
    2019-04-21 18:43:57 UTC: Spawning Virtual Chassis Control Protocol to check new configuration
    2019-04-21 18:43:57 UTC: Collecting status of Virtual Chassis Control Protocol
    error: configuration check-out failed
    fpc0:
    error: remote commit-configuration failed on fpc1
    error: commit failed

    Any ideas?



  • 4.  RE: EX4300 - Check-out failed for Chassis control process

    Posted 05-14-2019 13:12

    Are you using DAC cables to make the VC connections between FPC0 and FPC1? Can you paste the output of "show chassis hardware"



  • 5.  RE: EX4300 - Check-out failed for Chassis control process

    Posted 05-17-2019 06:29

    Yes, I'm using DAC cables.

     

    This is the requested output:

     

    {master:0}
    root@tpsw01> show chassis hardware
    Hardware inventory:
    Item             Version  Part number  Serial number     Description
    Chassis                                XXXXXXXXXXX      Virtual Chassis
    Routing Engine 0 REV 12   650-044930   XXXXXXXXXXX      EX4300-48P
    Routing Engine 1 REV 12   650-044930   XXXXXXXXXXX      EX4300-48P
    FPC 0            REV 12   650-044930   XXXXXXXXXXX      EX4300-48P
      CPU                     BUILTIN      BUILTIN           FPC CPU
      PIC 0          REV 12   BUILTIN      BUILTIN           48x 10/100/1000 Base-T
      PIC 1          REV 12   BUILTIN      BUILTIN           4x 40GE QSFP+
        Xcvr 0       REV 01   740-046565   QFST005560800020  UNKNOWN
        Xcvr 1       REV 01   740-046565   QFST005560800025  UNKNOWN
      Power Supply 0 REV 01   740-046871   1EDA5110147       JPSU-1100-AC-AFO-A
      Power Supply 1 REV 01   740-046871   1EDA4520961       JPSU-1100-AC-AFO-A
      Fan Tray 0                                             Fan Module, Airflow Out                    (AFO)
      Fan Tray 1                                             Fan Module, Airflow Out                    (AFO)
    FPC 1            REV 12   650-044930   XXXXXXXXXXX      EX4300-48P
      CPU                     BUILTIN      BUILTIN           FPC CPU
      PIC 0          REV 12   BUILTIN      BUILTIN           48x 10/100/1000 Base-T
      PIC 1          REV 12   BUILTIN      BUILTIN           4x 40GE QSFP+
      Power Supply 0 REV 01   740-046871   1EDA4520874       JPSU-1100-AC-AFO-A
      Power Supply 1 REV 01   740-046871   1EDA5110166       JPSU-1100-AC-AFO-A
      Fan Tray 0                                             Fan Module, Airflow Out                    (AFO)
      Fan Tray 1                                             Fan Module, Airflow Out                    (AFO)


  • 6.  RE: EX4300 - Check-out failed for Chassis control process

    Posted 05-17-2019 06:43

    And yes, now I see the problem. The DAC-cables are reported as "UNKNOWN". I'll try changing them.



  • 7.  RE: EX4300 - Check-out failed for Chassis control process

    Posted 05-17-2019 09:40

    Try changing the cables, I have seen this issue and it was a problem with the cable. After replacing the cable I was able to commit. 

     

    I hope that helps in your case as well.