Junos OS

Expand all | Collapse all

Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

  • 1.  Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

    Posted 6 days ago

    Hello, have a  "broken" SRX300
    Log from CLI

    DRAM: 4 GiB
    Clearing DRAM...... done
    SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB


    U-Boot 2013.07-JNPR-3.10 (Build time: May 26 2020 - 21:24:42)

    Octeon unique ID: 0a4000204519f31e038d
    Using DRAM size from environment: 4096 MBytes
    SRX_300 board revision major:1, minor:8, serial #: CV1317AF0729
    OCTEON CN7020-AAP pass 1.2, Core clock: 1200 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR)
    DRAM: 4 GiB
    Clearing DRAM...... done
    SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB
    SATA0: not available
    SATA1: not available
    PCIe: Port 0 link active, 1 lanes, speed gen2
    PCIe: Link timeout on port 1, probably the slot is empty
    PCIe: Port 2 not in PCIe mode, skipping
    Net: octeth0
    Node 0 Interface 0 has 1 ports (SGMII)
    Boot Media: eUSB usb
    WARNING: Running from backup u-boot
    SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB
    0 bytes written, 2097152 bytes skipped in 1.631s, speed 1285918 B/s
    Recovered primary U-Boot from secondary, please upgrade it to the latest version.
    Found TPM SLB9660 TT 1.2 by Infineon
    TPM initialized
    USB1: Starting the controller
    USB XHCI 1.00
    scanning bus 1 for devices... 2 USB Device(s) found
    USB0: Starting the controller
    USB XHCI 1.00
    scanning bus 0 for devices... 1 USB Device(s) found
    scanning usb for storage devices... 1 Storage Device(s) found
    Type the command 'usb start' to scan for USB storage devices.

    Press SPACE to stop autoboot: 0
    SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB
    SF: 1048512 bytes Read: OK
    SF: 1048576 bytes Read: OK
    ## Starting application ...
    [0]Booting from eUSB slice 1
    Consoles: U-Boot console
    Found compatible API, ver. 3.10

    FreeBSD/MIPS U-Boot bootstrap loader, Revision 2.11
    (builder@monarth, Sun Jul 26 18:59:02 PDT 2020)
    Memory: 4096MB
    [0]Booting from eUSB slice 1
    Loading /boot/defaults/loader.conf
    /boot/init.4th loaded.
    /kernel data=0x101599c+0x1d3e60 |


    Hit [Enter] to boot immediately, or space bar for command prompt.
    Booting [/kernel]...
    Kernel entry at 0x801000c0 ...
    init regular console
    Primary ICache: Sets 16 Size 128 Asso 39
    Primary DCache: Sets 8 Size 128 Asso 32
    Secondary DCache: Sets 1024 Size 128 Asso 4
    CIU_FUSE 0x5/0x3
    GDB: debug ports: uart
    GDB: current port: uart
    KDB: debugger backends: ddb gdb
    KDB: current backend: ddb
    kld_map_v: 0x8ff80000, kld_map_p: 0x0
    Running in PARTITIONED TLB MODE
    Copyright (c) 1996-2021, Juniper Networks, Inc.
    All rights reserved.
    Copyright (c) 1992-2007 The FreeBSD Project.
    Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
    The Regents of the University of California. All rights reserved.
    FreeBSD is a registered trademark of The FreeBSD Foundation.
    JUNOS 21.1R1.11 #0: 2021-03-18 20:48:46 UTC
    builder@qnc-jre-emake1t.juniper.net:/volume/build/junos/21.1/release/21.1R1.11/obj/octeon/junos/bsd/kernels/JSRXNLE/kernel
    can't re-use a leaf (perf_mon)!
    can't re-use a leaf (threshold)!
    can't re-use a leaf (debug)!
    JUNOS 21.1R1.11 #0: 2021-03-18 20:48:46 UTC
    builder@qnc-jre-emake1t.juniper.net:/volume/build/junos/21.1/release/21.1R1.11/obj/octeon/junos/bsd/kernels/JSRXNLE/kernel
    real memory = 4294967296 (4194304K bytes)
    avail memory = 2303045632 (2196MB)
    FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
    Security policy loaded: Junos MAC/veriexec (mac_veriexec)
    Security policy loaded: JUNOS MAC/pcap (mac_pcap)
    Security policy loaded: JUNOS MAC/runasnonroot (mac_runasnonroot)
    MAC/veriexec fingerprint module loaded: SHA1
    MAC/veriexec fingerprint module loaded: SHA256
    netisr_init: forcing maxthreads from 4 to 2
    random: <Software, Yarrow> initialized
    cpu0 on motherboard
    : CAVIUM's OCTEON 70XX/71XX CPU Rev. 0.2 with no FPU implemented
    L1 Cache: I size 78kb(128 line), D size 32kb(128 line), thirty two way.
    L2 Cache: Size 512kb, 4 way
    obio0 on motherboard
    uart0: <Octeon-16550 channel 0> on obio0
    uart0: console (9600,n,8,1)
    twsi0 on obio0
    set clock 0x49
    xhci0: <Cavium Octeon 7xxx xHCI Host Driver> on obio0
    usb0: <USB bus for xHCI Controller> on xhci0
    usb0: USB revision 3.0
    uhub0: vendor 0x0000 XHCI root hub, class 9/0, rev 3.00/1.00, addr 1
    uhub0: 2 ports with 2 removable, self powered
    xhci1: <Cavium Octeon 7xxx xHCI Host Driver> on obio0
    usb1: <USB bus for xHCI Controller> on xhci1
    usb1: USB revision 3.0
    uhub1: vendor 0x0000 XHCI root hub, class 9/0, rev 3.00/1.00, addr 1
    uhub1: 2 ports with 2 removable, self powered
    cpld0 on obio0
    pcib0: <Cavium on-chip PCIe HOST bridge> on obio0
    Disabling Octeon big bar support
    pcib0: Initialized controller
    pci0: <PCI bus> on pcib0
    pci0: <network, ethernet> at device 0.0 (no driver attached)
    pci0: <network, ethernet> at device 0.1 (no driver attached)
    gblmem0 on obio0
    octpkt0: <Octeon RGMII> on obio0
    boot_bus0 on obio0
    cfi0: <Macronix MX25L64 - 8MB> on boot_bus0
    cfi1: <Macronix MX25L64 - 8MB> on boot_bus0
    umass0: ATP Electronics ATP CG eUSB, rev 2.00/11.00, addr 2
    Timecounter "mips" frequency 1200000000 Hz quality 0
    da0 at umass-sim0 bus 0 target 0 lun 0
    da0: <ATP ATP CG eUSB 1100> Fixed Direct Access SCSI-4 device
    da0: 40.000MB/s transfers
    da0: 7672MB (15712256 512 byte sectors: 255H 63S/T 978C)
    random: unblocking device.
    hwpmc: OCTEON/4/64/0x1ff<INT,USR,SYS,EDG,THR,REA,WRI,INV,QUA>
    Trying to mount root from ufs:/dev/da0s1a
    WARNING: / was not properly dismounted
    MFSINIT: Initialising MFSROOT
    WARNING: / was not properly dismounted
    Process-1 beginning MFSROOT initialization...
    Creating MFSROOT...
    /dev/md0: 20.0MB (40956 sectors) block size 16384, fragment size 2048
    using 4 cylinder groups of 5.00MB, 320 blks, 640 inodes.
    super-block backups (for fsck -b #) at:
    32, 10272, 20512, 30752
    Populating MFSROOT...
    Creating symlinks...
    Setting up mounts...
    Continuing boot from MFSROOT...
    Attaching /cf/packages/junos via /dev/mdctl...
    Mounted junos package on /dev/md1...
    J
    WARNING: R/W mount of /cf/var denied. Filesystem is not clean - run fsck
    mount: /dev/bo0s3f : Operation not permitted
    ** /dev/bo0s3f
    ** Last Mounted on /cf/var
    ** Phase 1 - Check Blocks and Sizes
    ** Phase 2 - Check Pathnames
    ** Phase 3 - Check Connectivity
    ** Phase 4 - Check Reference Counts
    UNREF FILE I=188558 OWNER=0 MODE=100644
    SIZE=0 MTIME=Jun 8 23:53 2021
    RECONNECT? yes

    LINK COUNT FILE I=188558 OWNER=0 MODE=100644
    SIZE=0 MTIME=Jun 8 23:53 2021 COUNT 2 SHOULD BE 1
    ADJUST? yes

    ** Phase 5 - Check Cyl groups
    FREE BLK COUNT(S) WRONG IN SUPERBLK
    SALVAGE? yes

    SUMMARY INFORMATION BAD
    SALVAGE? yes

    BLK(S) MISSING IN BIT MAPS
    SALVAGE? yes

    772 files, 8388 used, 1100825 free (281 frags, 137568 blocks, 0.0% fragmentation)

    ***** FILE SYSTEM MARKED CLEAN *****

    ***** FILE SYSTEM WAS MODIFIED *****
    Automatic reboot in progress...
    Verified jboot signed by PackageProductionECP256_2021 method ECDSA256+SHA256
    Verified junos signed by PackageProductionECP256_2021 method ECDSA256+SHA256
    Verified junos-21.1R1.11 signed by PackageProductionECP256_2021 method ECDSA256+SHA256
    Checking integrity of BSD labels:
    s1: Passed
    s2: Passed
    s3: Passed
    s4: Passed
    ** /dev/bo0s3e
    ** Last Mounted on /config
    ** Phase 1 - Check Blocks and Sizes
    ** Phase 2 - Check Pathnames
    ** Phase 3 - Check Connectivity
    ** Phase 4 - Check Reference Counts
    ** Phase 5 - Check Cyl groups
    13 files, 13 used, 94763 free (27 frags, 11842 blocks, 0.0% fragmentation)

    ***** FILE SYSTEM MARKED CLEAN *****
    ** /dev/bo0s3f
    FILE SYSTEM CLEAN; SKIPPING CHECKS
    clean, 1100825 free (281 frags, 137568 blocks, 0.0% fragmentation)
    Checking integrity of licenses:
    Checking integrity of configuration:
    rescue.conf.gz: Passed

    LPC bus driver
    lpcbus0 on cpld0
    tpm0: <Trusted Platform Module> on lpcbus0
    tpm: IFX SLB 9660 TT 1.2 rev 0x10
    Loading configuration ...
    Time and ticks drifted too much, resetting synchronization...
    mgd: commit complete
    Setup /jail/var/cache dir only for srx300
    [: 16: unexpected operator
    Setting initial options: .
    Starting optional daemons: ln: /opt/telemetry/na-grpcd/na-ocpaths.json: File exists
    ln: /opt/telemetry/na-grpcd/na-agent.json: File exists
    ln: /opt/telemetry/na-grpcd/na-grpc-server.ini: File exists
    ln: /opt/telemetry/na-mqttd/na-mqtt.conf: File exists
    usbd.
    Doing initial network setup:
    .
    Initial interface configuration:
    additional daemons: eventd.
    Enhanced arp scale is disabled
    Additional routing options:kern.module_path: /boot//kernel;/boot/modules -> /boot/modules;/cryptosoft0: <software crypto> on motherboard
    modules/ifpfe_drv;/modules;
    kld netpfe drv: ifpfed_dialer pvid_IPsec: Initialized Security Association Processing.
    db kld ipsec.
    Doing additional network setup:.
    Starting final network daemons:.
    setting ldconfig path: /usr/lib /opt/lib
    starting standard daemons: cron.
    Initial rc.mips initialization:.
    Local package initialization:.
    starting local daemons:set cores for group access
    .
    kern.securelevel: -1 -> 1
    Creating JAIL MFS partition...
    JAIL MFS partition created
    Boot media /dev/da0 has dual root support
    ** /dev/da0s2a
    FILE SYSTEM CLEAN; SKIPPING CHECKS
    clean, 1049900 free (84 frags, 131227 blocks, 0.0% fragmentation)
    chassis.ko loaded Loading JUNOS chassis module
    chassis_init_hw_chassis_startup_time: chassis startup time 0.000000
    Wed Jun 9 05:17:51 UTC 2021


    SPI stage 1 bootloader (Build time: May 26 2020 - 21:22:59)


    U-Boot 2013.07-JNPR-3.10 (Build time: May 26 2020 - 21:23:00)

    Octeon unique ID: 0a4000204519f31e038d
    N0.LMC0 Configuration Completed: 4096 MB
    SRX_300 board revision major:1, minor:8, serial #: CV1317AF0729
    OCTEON CN7020-AAP pass 1.2, Core clock: 1200 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR)
    DRAM: 4 GiB
    Clearing DRAM...... done
    SF: Detected SF with page size 256 Bytes, erase size 64 KiB, total 8 MiB


    U-Boot 2013.07-JNPR-3.10 (Build time: May 26 2020 - 21:24:42)

    Octeon unique ID: 0a4000204519f31e038d
    Using DRAM size from environment: 4096 MBytes
    SRX_300 board revision major:1, minor:8, serial #: CV1317AF0729
    OCTEON CN7020-AAP pass 1.2, Core clock: 1200 MHz, IO clock: 600 MHz, DDR clock: 667 MHz (1334 Mhz DDR)
    DRAM: 4 GiB
    Clearing DRAM......


    Sometimes  Clearing Dram is ok and i can load junos. From USB flash i can boot junos normally - thats mean that u-boot is ok?

    So, questions is what is broken - eUSB SSD Flash Memory or one of the 10s DDR3 memory chip?​



  • 2.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

     
    Posted 6 days ago
    Since you can successfully boot from external media, a logical next step would be to try reinstalling Junos on the internal media with the format drive option.

    Add --format to the normal installation command 

    Then if it is the disk any bad sectors will be marked and not used so you may be able to recover.

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 3.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

    Posted 6 days ago

    I already boot from usb and install new firmware from usb to internal media but without format install and after this - still same stuck on Clearing DRAM.
    From loader> prompt " --format" option  is not available, only install.

    Question #2 - same srx100 i will successfuly resurrected with nand_format from db mode =>

    How i can boot to db=> prompt ?




  • 4.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

     
    Posted 5 days ago
    Sounds like there are boot changes with this series.  Maybe one these two procedures for disk repair might work.

    https://kb.juniper.net/InfoCenter/index?page=content&id=KB20570

    https://kb.juniper.net/InfoCenter/index?page=content&id=KB35814

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 5.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

    Posted 5 days ago

    So, is there a high probability that this is a flash or is it one of the memory banks? For repair, of course, it is easier to rewire a SSD flash drive than 10s of DDR3 BGA: D chips.
    I know that in EX switches, such as EX2300|4300  i can choose OAM menu with Ctrl+C before autoboot. On srx devices OAM menu is availiable too or not?

    "user> start shell
    % su
    Password:
    root@% nand-mediack -C
    root@% nand-mediack
    root@% nand-format
    "
    As I understand it, a flash drive without broken sectors and with it everything should be ok?

    Ok, thx for your reply. Will try to format and install disk with various methods. :D




  • 6.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

     
    Posted 5 days ago
    I am leaning towards the disk being the issue only because you can successfully launch from usb boot.

    But I guess we should also say that these disk procedures are all things you can do without violating any warranty.  So don't replace RAM is you have any kind of support option.  

    And naturally if you have support you can open a JTAC case and have them confirm if it is a RAM hardware issue and get an RMA replacement in that case.

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 7.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

    Posted 4 days ago

    I dont have any JTAC support cuz im buying this SRX on ebay with 50$ with mark "for parts" )

    Some times after few reboots or just waiting few hours status Clearing DRAM can finish and after that i have normall boot from internal storage.




  • 8.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

     
    Posted 4 days ago
    Since you can normal boot and operation on a usb drive, I would still run all the supported disk related repairs to see if internal boot can be restored.

    If these do not work I am pretty sure the internal drive is integrated to the motherboard on these branch units but I have never opened up an SRX340 to confirm.

    Likewise other small branch SRX I have seen have integrated and not replaceable memory so I assume that is the case here too but don't know for sure.

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 9.  RE: Corrupt flash or DDR3 memory chip? Stuck on Clearing DRAM...

    Posted 4 days ago
      |   view attached
    Yes, u r right. RAM is intergrated. But, i can remove DDR3L chips from random standart DDR3 4RAM PC and replace to SRX ).

    It's a pity that so far it is not possible to do nand_format to eliminate the problem in flash memory

    So,, after replace 100% normal DDR3L chip to SRX - I will write off the results