Routing

Hello, 

We have an MX80 that was working fine and from nowhere we lost service and after consoling it turns out the Box cant see any physical interfaces cards even the built-in 4*10G ports. we have an extra 2 port 10G card and its also not seen. as for the routing-engine its seems ok, console to the box works fine, also we can ssh to IP on OOB Mgnt port.

what can we do to make it see the cards again?

root> show version Model: mx80Junos: 18.4R3-S10JUNOS Base OS boot [18.4R3-S10]JUNOS Base OS Software Suite [18.4R3-S10]JUNOS Crypto Software Suite [18.4R3-S10]JUNOS Packet Forwarding Engine Support (MX80) [18.4R3-S10]JUNOS Web Management [18.4R3-S10]JUNOS Online Documentation [18.4R3-S10]JUNOS SDN Software Suite [18.4R3-S10]JUNOS Services Application Level Gateways [18.4R3-S10]JUNOS Services COS [18.4R3-S10]JUNOS Services Jflow Container package [18.4R3-S10]JUNOS Services Stateful Firewall [18.4R3-S10]JUNOS Services NAT [18.4R3-S10]JUNOS Services RPM [18.4R3-S10]JUNOS Services SOFTWIRE [18.4R3-S10]JUNOS Services Captive Portal and Content Delivery Container package [18.4R3-S10]JUNOS Macsec Software Suite [18.4R3-S10]JUNOS Services Crypto [18.4R3-S10]JUNOS Services IPSec [18.4R3-S10]JUNOS DP Crypto Software Software Suite [18.4R3-S10]JUNOS py-base-powerpc [18.4R3-S10]JUNOS py-extensions-powerpc [18.4R3-S10]JUNOS jsd [powerpc-18.4R3-S10-jet-1]JUNOS Kernel Software Suite [18.4R3-S10]JUNOS Routing Software Suite [18.4R3-S10]root> show chassis hardware Hardware inventory:Item             Version  Part number  Serial number     DescriptionChassis                                T8449             MX80Midplane         REV 08   711-031594   ABCG4174          MX80PEM 0            Rev 05   740-028288   XL13218           AC Power Entry ModulePEM 1            Rev 05   740-028288   YE01111           AC Power Entry ModuleRouting Engine            BUILTIN      BUILTIN           Routing EngineTFEB 0                    BUILTIN      BUILTIN           Forwarding Engine ProcessorFan Tray                                                 Fan Trayroot> 

Regards, 

Lish

Seems likely this will be a hardware failure issue since the built in fpc is missing.  Check the logs for the time period when the device went offline to see if there are any helpful messages.

show log messages

If this file has rolled over there are generally several back ones you can see the names of using

show log messages?

Have you already shutdown and started the box or a basic restart?

You could try to force offline offline per fpc as well.

https://www.juniper.net/documentation/us/en/software/junos/chassis/topics/ref/command/request-chassis-fpc.html

Hello, 

We have an MX80 that was working fine and from nowhere we lost service and after consoling it turns out the Box cant see any physical interfaces cards even the built-in 4*10G ports. we have an extra 2 port 10G card and its also not seen. as for the routing-engine its seems ok, console to the box works fine, also we can ssh to IP on OOB Mgnt port.

what can we do to make it see the cards again?

root> show version Model: mx80Junos: 18.4R3-S10JUNOS Base OS boot [18.4R3-S10]JUNOS Base OS Software Suite [18.4R3-S10]JUNOS Crypto Software Suite [18.4R3-S10]JUNOS Packet Forwarding Engine Support (MX80) [18.4R3-S10]JUNOS Web Management [18.4R3-S10]JUNOS Online Documentation [18.4R3-S10]JUNOS SDN Software Suite [18.4R3-S10]JUNOS Services Application Level Gateways [18.4R3-S10]JUNOS Services COS [18.4R3-S10]JUNOS Services Jflow Container package [18.4R3-S10]JUNOS Services Stateful Firewall [18.4R3-S10]JUNOS Services NAT [18.4R3-S10]JUNOS Services RPM [18.4R3-S10]JUNOS Services SOFTWIRE [18.4R3-S10]JUNOS Services Captive Portal and Content Delivery Container package [18.4R3-S10]JUNOS Macsec Software Suite [18.4R3-S10]JUNOS Services Crypto [18.4R3-S10]JUNOS Services IPSec [18.4R3-S10]JUNOS DP Crypto Software Software Suite [18.4R3-S10]JUNOS py-base-powerpc [18.4R3-S10]JUNOS py-extensions-powerpc [18.4R3-S10]JUNOS jsd [powerpc-18.4R3-S10-jet-1]JUNOS Kernel Software Suite [18.4R3-S10]JUNOS Routing Software Suite [18.4R3-S10]root> show chassis hardware Hardware inventory:Item             Version  Part number  Serial number     DescriptionChassis                                T8449             MX80Midplane         REV 08   711-031594   ABCG4174          MX80PEM 0            Rev 05   740-028288   XL13218           AC Power Entry ModulePEM 1            Rev 05   740-028288   YE01111           AC Power Entry ModuleRouting Engine            BUILTIN      BUILTIN           Routing EngineTFEB 0                    BUILTIN      BUILTIN           Forwarding Engine ProcessorFan Tray                                                 Fan Trayroot> 

Regards, 

Lish

I shut down the box and loaded the factory default even.

this is the chassisd logs at the moment of the crash: 

Jul 25 11:39:46  Receive SIGHUPJul 25 11:39:46  sighup_handler do new_configJul 25 11:39:46  parse_configuration ddlJul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x267d000, fn 0x0, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x26d0000, fn 0x1e4e148, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x26d8000, fn 0x1e4e148, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x26e0000, fn 0x1e4e148, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x270a000, fn 0x0, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x2765000, fn 0x1e4e148, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x276d000, fn 0x1e4e148, i2c-validity: 0 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x27a0000, fn 0x0, i2c-validity: 1 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x27a3000, fn 0x0, i2c-validity: 1 Jul 25 11:39:46  fpc_is_xm_based_fpc: fru pointer is 0x27aa000, fn 0x0, i2c-validity: 1 Jul 25 11:39:46  ch_info_dynamic_power_mgmt_state_blob_get: dynamic power mgmt state not set in kernelJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {chassis, aggregated-devices}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Object {aggregated-devices,  }Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {aggregated-devices, ethernet}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Object {ethernet,  }Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {ethernet, device-count}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Object {ethernet, device-count}Jul 25 11:39:46  configured aggregated ethernet device count 5Jul 25 11:39:46  aggregated-device ethernet device-countJul 25 11:39:46  configured aggregated ethernet stateJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {chassis, tfeb}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Object {tfeb,  }Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {tfeb, slot}: Container Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Container {slot,  }Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {tfeb, slot}: Element Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Parsing Element {slot,  }Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {slot, slotnumber}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {slot, sampling-instance}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Parsing Attribute {sampling-instance,  }Jul 25 11:39:46  tfeb 0: sampling instance IRP-INSTANCEJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {chassis, alarm}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Object {alarm,  }Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Found {alarm, management-ethernet}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Walking Object {management-ethernet, link-down}Jul 25 11:39:46  fxp0-link-downJul 25 11:39:46  core group access set to:1Jul 25 11:39:46  Calling chasd_config_read_pfe_irb_options with check_only 0Jul 25 11:39:46  No routing-options source_routing configuration options setJul 25 11:39:46  No virtual-chassis options setJul 25 11:39:46  No HW FIB bulk options setJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {forwarding-options, sampling}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {forwarding-options, load-balance-label-capability}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  protocol-id queue-depth delete-flagJul 25 11:39:46  Total Queue Allocation: 0/1024Jul 25 11:39:46  ch_info_pfe_hyper_mode_get: hw.chassis.pfe_hyper_mode sysctl doesn't exist in current kernelJul 25 11:39:46  Entry chasd_config_read_ir_mode check 0Jul 25 11:39:46  Entry chasd_config_read_ir_mode_internal: init 1 ir 0 r 0 en 0Jul 25 11:39:46  Exit chasd_config_read_ir_mode_internal: ret 0 ir 0 r 0 en 0 chk 0Jul 25 11:39:46  Exit chasd_config_read_ir_mode: ret 0 ir 0 r 0Jul 25 11:39:46  Entry chasd_config_read_license_mode r_mode fpcs 0 ir_mode fpcs 0Jul 25 11:39:46  Exit chasd_config_read_license_mode r_mode fpcs 0 ir_mode fpcs 0Jul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, commit}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, configuration-database}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, login}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, root-authentication}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, services}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, host-name}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, time-zone}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, no-multicast-echo}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, no-redirects}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, no-ping-record-route}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, no-ping-time-stamp}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, ports}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, name-server}: Container Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, syslog}: Object Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  cfg_ddl_chasd_handle_config_option: Did not find {system, compress-configuration-files}: Attribute Config action: DAX_ITEM_UNCHANGEDJul 25 11:39:46  Read ifname xe-0/0/0Jul 25 11:39:46  Read ifname xe-0/0/1Jul 25 11:39:46  Read ifname xe-0/0/2Jul 25 11:39:46  Read ifname xe-0/0/3Jul 25 11:39:46  Read ifname xe-1/0/0Jul 25 11:39:46  Read ifname xe-1/1/0Jul 25 11:39:46  Read ifname ae1Jul 25 11:39:46  Read ifname fxp0Jul 25 11:39:46  Read ifname lo0Jul 25 11:39:46  Read ifname me0Jul 25 11:39:46  pic_parse_ifname: parse_interface of xe-0/0/0 found fpc 0, pic 0, port 0, channel -1Jul 25 11:39:46  pic_parse_ifname: parse_interface of xe-0/0/1 found fpc 0, pic 0, port 1, channel -1Jul 25 11:39:46  pic_parse_ifname: parse_interface of xe-0/0/2 found fpc 0, pic 0, port 2, channel -1Jul 25 11:39:46  pic_parse_ifname: parse_interface of xe-0/0/3 found fpc 0, pic 0, port 3, channel -1Jul 25 11:39:46  pic_parse_ifname: parse_interface of xe-1/0/0 found fpc 1, pic 0, port 0, channel -1Jul 25 11:39:46  pic_parse_ifname: parse_interface of xe-1/1/0 found fpc 1, pic 1, port 0, channel -1Jul 25 11:39:46  chasd_cfg_read_l2_bridge: l2_bridge_mode will be updated only in boot mode Jul 25 11:39:46  ch_create_auto_install_tmp_file AIU: Created /tmp/qfxautoconfig_tmp and enabling auto-installJul 25 11:39:46  cfg_ddl_parse_mxseries_misc_conf_options AIU: Auto-install not triggeredJul 25 11:39:46  completed chassis state from ddlJul 25 11:39:46  ch_set_non_stop_forwarding_cfg:Setting non-stop-forwarding to Disabled, source DDLJul 25 11:39:46  ch_do_multichassis_overrides: Setting multichassis replication to DisabledJul 25 11:39:46  fpc_post_process_config_generic1: FCOE fabric hash prev 0x1 new 0x1 Jul 25 11:39:46  fpc_post_process_config_generic1: FCOE eth hash prev 0x1 new 0x1 Jul 25 11:39:46  fpc_post_process_config_generic1: UFT mode prev 0x0 new 0x0 Jul 25 11:39:46  fpc_post_process_config_generic1: UFT num prefix prev 0x0 new 0x0 Jul 25 11:39:46  fpc_post_process_config_generic1: fpc is enhanced hashkey capableJul 25 11:39:46  fpc_post_process_config_generic1: fru->fru_old_config.cfg_ucode_rebalance TRUE new TRUE Jul 25 11:39:46  fpc_post_process_config_generic, new setting : albflag(0) tol(0)old setting:albflag(0) tol(0) Jul 25 11:39:46  fpc_do_config... Jul 25 11:39:46  fpc_pic_post_process_config_generic:  FPC 0 PIC 0 pic type:2f9Jul 25 11:39:46  pic_should_bounce fpc/pic (0/0): PIC  - new speed 0 old speed 0 Jul 25 11:39:46  pic_port_bounce port bounce port 0 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 1 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 2 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 3 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 0 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 1 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 2 base 0Jul 25 11:39:46  pic_port_bounce port bounce port 3 base 0Jul 25 11:39:46  [0,0] in pic_do_trinity_configJul 25 11:39:46  fpc_pic_process_rcvrd_clk_config() fpc 0 pic 0 Jul 25 11:39:46  fpc_do_config... Jul 25 11:39:46  fpc_do_config... Jul 25 11:39:47  fpc_do_config... Jul 25 11:39:47  fpc_error_post_process_config: cfg_changed =0 Jul 25 11:39:47  fpc_post_process_config_generic1: FCOE fabric hash prev 0x1 new 0x1 Jul 25 11:39:47  fpc_post_process_config_generic1: FCOE eth hash prev 0x1 new 0x1 Jul 25 11:39:47  fpc_post_process_config_generic1: UFT mode prev 0x0 new 0x0 Jul 25 11:39:47  fpc_post_process_config_generic1: UFT num prefix prev 0x0 new 0x0 Jul 25 11:39:47  fpc_post_process_config_generic1: fpc is enhanced hashkey capableJul 25 11:39:47  fpc_post_process_config_generic1: fru->fru_old_config.cfg_ucode_rebalance TRUE new TRUE Jul 25 11:39:47  fpc_post_process_config_generic, new setting : albflag(0) tol(0)old setting:albflag(0) tol(0) Jul 25 11:39:47  fpc_do_config... Jul 25 11:39:47  fpc_pic_post_process_config_generic:  FPC 1 PIC 0 pic type:2f0Jul 25 11:39:47  pic_should_bounce fpc/pic (1/0): PIC  - new speed 0 old speed 0 Jul 25 11:39:47  pic_port_bounce port bounce port 0 base 0Jul 25 11:39:47  pic_port_bounce port bounce port 0 base 0Jul 25 11:39:47  [1,0] in pic_do_trinity_configJul 25 11:39:47  fpc_pic_process_rcvrd_clk_config() fpc 1 pic 0 Jul 25 11:39:47  fpc_do_config... Jul 25 11:39:47  fpc_pic_post_process_config_generic:  FPC 1 PIC 1 pic type:2f0Jul 25 11:39:47  pic_should_bounce fpc/pic (1/1): PIC  - new speed 0 old speed 0 Jul 25 11:39:47  pic_port_bounce port bounce port 0 base 0Jul 25 11:39:47  pic_port_bounce port bounce port 0 base 0Jul 25 11:39:47  [1,1] in pic_do_trinity_configJul 25 11:39:47  fpc_pic_process_rcvrd_clk_config() fpc 1 pic 1 Jul 25 11:39:47  fpc_do_config... Jul 25 11:39:47  fpc_do_config... Jul 25 11:39:47  fpc_error_post_process_config: cfg_changed =0 Jul 25 11:39:47  pseudo_update_devs_redundancy_group: Redundancy-group type RLT has new count 0, old count 0 Jul 25 11:39:47  pseudo_update_devs_redundancy_group: Redundancy-group type RVT has new count 0, old count 0 Jul 25 11:39:47 CHASSISD_PARSE_COMPLETE: Using new configuration

Seems likely this will be a hardware failure issue since the built in fpc is missing.  Check the logs for the time period when the device went offline to see if there are any helpful messages.

show log messages

If this file has rolled over there are generally several back ones you can see the names of using

show log messages?

Have you already shutdown and started the box or a basic restart?

You could try to force offline offline per fpc as well.

https://www.juniper.net/documentation/us/en/software/junos/chassis/topics/ref/command/request-chassis-fpc.html

Hello, 

We have an MX80 that was working fine and from nowhere we lost service and after consoling it turns out the Box cant see any physical interfaces cards even the built-in 4*10G ports. we have an extra 2 port 10G card and its also not seen. as for the routing-engine its seems ok, console to the box works fine, also we can ssh to IP on OOB Mgnt port.

what can we do to make it see the cards again?

root> show version Model: mx80Junos: 18.4R3-S10JUNOS Base OS boot [18.4R3-S10]JUNOS Base OS Software Suite [18.4R3-S10]JUNOS Crypto Software Suite [18.4R3-S10]JUNOS Packet Forwarding Engine Support (MX80) [18.4R3-S10]JUNOS Web Management [18.4R3-S10]JUNOS Online Documentation [18.4R3-S10]JUNOS SDN Software Suite [18.4R3-S10]JUNOS Services Application Level Gateways [18.4R3-S10]JUNOS Services COS [18.4R3-S10]JUNOS Services Jflow Container package [18.4R3-S10]JUNOS Services Stateful Firewall [18.4R3-S10]JUNOS Services NAT [18.4R3-S10]JUNOS Services RPM [18.4R3-S10]JUNOS Services SOFTWIRE [18.4R3-S10]JUNOS Services Captive Portal and Content Delivery Container package [18.4R3-S10]JUNOS Macsec Software Suite [18.4R3-S10]JUNOS Services Crypto [18.4R3-S10]JUNOS Services IPSec [18.4R3-S10]JUNOS DP Crypto Software Software Suite [18.4R3-S10]JUNOS py-base-powerpc [18.4R3-S10]JUNOS py-extensions-powerpc [18.4R3-S10]JUNOS jsd [powerpc-18.4R3-S10-jet-1]JUNOS Kernel Software Suite [18.4R3-S10]JUNOS Routing Software Suite [18.4R3-S10]root> show chassis hardware Hardware inventory:Item             Version  Part number  Serial number     DescriptionChassis                                T8449             MX80Midplane         REV 08   711-031594   ABCG4174          MX80PEM 0            Rev 05   740-028288   XL13218           AC Power Entry ModulePEM 1            Rev 05   740-028288   YE01111           AC Power Entry ModuleRouting Engine            BUILTIN      BUILTIN           Routing EngineTFEB 0                    BUILTIN      BUILTIN           Forwarding Engine ProcessorFan Tray                                                 Fan Trayroot> 

Regards, 

Lish