I do believe this is a memory leak. But who knows where ...
Without going into the weeds, the easiest thing to do, in my opinion, is to restart components one by one (see restart ? in the CLI) and then check vmstat -m | grep temp from a root shell to see at what point the temp memory bucket gets freed up.
Original Message:
Sent: 05-08-2025 13:46
From: Nikolay Semov
Subject: High routing memory utilization
Yeah ... Now we just have to figure out what this "temp" bucket is and why it's taking up 2 GB of memory...
temp 14375344 2020898K - 123160135 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768
------------------------------
Nikolay Semov
Original Message:
Sent: 05-08-2025 03:11
From: DARIO PULIERO
Subject: High routing memory utilization
Hi Nikolay,
I have any active traceoptions:
@srx# show | display set | match traceoptions{primary:node0}[edit]
Below the output of show system memory command:
@srx> show system memorynode0:--------------------------------------------------------------------------System memory usage distribution: Total memory: 3193324 Kbytes (100%) Reserved memory: 81736 Kbytes ( 2%) Wired memory: 2513524 Kbytes ( 78%) Active memory: 87748 Kbytes ( 2%) Inactive memory: 203068 Kbytes ( 6%) Cache memory: 0 Kbytes ( 0%) Free memory: 65520 Kbytes ( 2%)Pid VM-Kbytes( % ) Resident( % ) Process-name 0 0(00.00) 0(00.00) [kernel] 1 0(00.00) 0(00.00) /sbin/init -- 2 0(00.00) 0(00.00) [jfe_job_0_0] 3 0(00.00) 0(00.00) [jfe_job_1_0] 4 0(00.00) 0(00.00) [jfe_job_1_1] 5 0(00.00) 0(00.00) [jfe_job_2_0] 6 0(00.00) 0(00.00) [jfe_job_3_0] 7 0(00.00) 0(00.00) [jfe_job_3_1] 8 0(00.00) 0(00.00) [jfe_job_4_0] 9 0(00.00) 0(00.00) [jfe_job_4_1] 10 0(00.00) 0(00.00) [audit] 11 0(00.00) 0(00.00) [idle] 12 0(00.00) 0(00.00) [intr] 13 0(00.00) 0(00.00) [geom] 14 0(00.00) 0(00.00) [md0] 15 0(00.00) 0(00.00) [crypto] 16 0(00.00) 0(00.00) [crypto returns 0] 17 0(00.00) 0(00.00) [crypto returns 1] 18 0(00.00) 0(00.00) [md0.uzip] 19 0(00.00) 0(00.00) [cam] 20 0(00.00) 0(00.00) [usb] 21 0(00.00) 0(00.00) [sctp_iterator] 22 0(00.00) 0(00.00) [rand_harvestq] 23 0(00.00) 0(00.00) [kern_dump_proc] 24 0(00.00) 0(00.00) [pagedaemon] 25 0(00.00) 0(00.00) [vmdaemon] 26 0(00.00) 0(00.00) [bufdaemon] 27 0(00.00) 0(00.00) [syncer] 28 0(00.00) 0(00.00) [vnlru] 29 0(00.00) 0(00.00) [netdaemon] 30 0(00.00) 0(00.00) [wkupdaemon] 31 0(00.00) 0(00.00) [jsr_kkcm] 32 0(00.00) 0(00.00) [jsr_jsm] 33 0(00.00) 0(00.00) [dfree_resourced] 34 0(00.00) 0(00.00) [if_pfe_listen] 35 0(00.00) 0(00.00) [ifsclientclosed] 36 0(00.00) 0(00.00) [kern_pir_proc] 37 0(00.00) 0(00.00) [if_pic_listen0] 38 0(00.00) 0(00.00) [tnparpproc] 48 0(00.00) 0(00.00) [soaiod1] 49 0(00.00) 0(00.00) [soaiod2] 50 0(00.00) 0(00.00) [soaiod3] 51 0(00.00) 0(00.00) [soaiod4] 168 0(00.00) 0(00.00) [md1] 169 0(00.00) 0(00.00) [md1.uzip] 503 0(00.00) 0(00.00) [md2] 504 0(00.00) 0(00.00) [md2.uzip] 702 0(00.00) 0(00.00) [md3] 703 0(00.00) 0(00.00) [md3.uzip] 1226 0(00.00) 0(00.00) [md4] 1227 0(00.00) 0(00.00) [md4.uzip] 1284 0(00.00) 0(00.00) [md5] 1285 0(00.00) 0(00.00) [md5.uzip] 1345 0(00.00) 0(00.00) [md6] 1346 0(00.00) 0(00.00) [md6.uzip] 1404 0(00.00) 0(00.00) [md7] 1405 0(00.00) 0(00.00) [md7.uzip] 1465 0(00.00) 0(00.00) [md8] 1466 0(00.00) 0(00.00) [md8.uzip] 1526 0(00.00) 0(00.00) [md9] 1527 0(00.00) 0(00.00) [md9.uzip] 1627 0(00.00) 0(00.00) [md10] 1628 0(00.00) 0(00.00) [md10.uzip] 1715 0(00.00) 0(00.00) [md11] 1716 0(00.00) 0(00.00) [md11.uzip] 1774 0(00.00) 0(00.00) [md12] 1775 0(00.00) 0(00.00) [md12.uzip] 1833 0(00.00) 0(00.00) [md13] 1834 0(00.00) 0(00.00) [md13.uzip] 2379 0(00.00) 0(00.00) [md14] 2380 0(00.00) 0(00.00) [md14.uzip] 2487 0(00.00) 0(00.00) [md15] 2488 0(00.00) 0(00.00) [md15.uzip] 2622 0(00.00) 0(00.00) [md16] 2623 0(00.00) 0(00.00) [md16.uzip] 2722 0(00.00) 0(00.00) [md17] 2723 0(00.00) 0(00.00) [md17.uzip] 2788 0(00.00) 0(00.00) [md18] 2789 0(00.00) 0(00.00) [md18.uzip] 2859 0(00.00) 0(00.00) [md19] 2860 0(00.00) 0(00.00) [md19.uzip] 2918 0(00.00) 0(00.00) [md20] 2919 0(00.00) 0(00.00) [md20.uzip] 2977 0(00.00) 0(00.00) [md21] 2978 0(00.00) 0(00.00) [md21.uzip] 3036 0(00.00) 0(00.00) [md22] 3037 0(00.00) 0(00.00) [md22.uzip] 3095 0(00.00) 0(00.00) [md23] 3096 0(00.00) 0(00.00) [md23.uzip] 3154 0(00.00) 0(00.00) [md24] 3155 0(00.00) 0(00.00) [md24.uzip] 3215 0(00.00) 0(00.00) [md25] 3216 0(00.00) 0(00.00) [md25.uzip] 3276 0(00.00) 0(00.00) [md26] 3277 0(00.00) 0(00.00) [md26.uzip] 3337 0(00.00) 0(00.00) [md27] 3338 0(00.00) 0(00.00) [md27.uzip] 3398 0(00.00) 0(00.00) [md28] 3399 0(00.00) 0(00.00) [md28.uzip] 3460 0(00.00) 0(00.00) [md29] 3461 0(00.00) 0(00.00) [md29.uzip] 3521 0(00.00) 0(00.00) [md30] 3522 0(00.00) 0(00.00) [md30.uzip] 3582 0(00.00) 0(00.00) [md31] 3583 0(00.00) 0(00.00) [md31.uzip] 3643 0(00.00) 0(00.00) [md32] 3644 0(00.00) 0(00.00) [md32.uzip] 3704 0(00.00) 0(00.00) [md33] 3705 0(00.00) 0(00.00) [md33.uzip] 3808 0(00.00) 0(00.00) [md34] 3809 0(00.00) 0(00.00) [md34.uzip] 3869 0(00.00) 0(00.00) [md35] 3870 0(00.00) 0(00.00) [md35.uzip] 3930 0(00.00) 0(00.00) [md36] 3931 0(00.00) 0(00.00) [md36.uzip] 3989 0(00.00) 0(00.00) [md37] 3990 0(00.00) 0(00.00) [md37.uzip] 4048 0(00.00) 0(00.00) [md38] 4049 0(00.00) 0(00.00) [md38.uzip] 4109 0(00.00) 0(00.00) [md39] 4110 0(00.00) 0(00.00) [md39.uzip] 4170 0(00.00) 0(00.00) [md40] 4171 0(00.00) 0(00.00) [md40.uzip] 4231 0(00.00) 0(00.00) [md41] 4232 0(00.00) 0(00.00) [md41.uzip] 4292 0(00.00) 0(00.00) [md42] 4293 0(00.00) 0(00.00) [md42.uzip] 4359 0(00.00) 0(00.00) [md43] 4360 0(00.00) 0(00.00) [md43.uzip] 4420 0(00.00) 0(00.00) [md44] 4421 0(00.00) 0(00.00) [md44.uzip] 4481 0(00.00) 0(00.00) [md45] 4482 0(00.00) 0(00.00) [md45.uzip] 4542 0(00.00) 0(00.00) [md46] 4543 0(00.00) 0(00.00) [md46.uzip] 4691 0(00.00) 0(00.00) [md47] 4692 0(00.00) 0(00.00) [md47.uzip] 4808 0(00.00) 0(00.00) [md48] 4809 0(00.00) 0(00.00) [md48.uzip] 4875 0(00.00) 0(00.00) [md49] 4876 0(00.00) 0(00.00) [md49.uzip] 4942 0(00.00) 0(00.00) [md50] 4943 0(00.00) 0(00.00) [md50.uzip] 5012 0(00.00) 0(00.00) [md51] 5013 0(00.00) 0(00.00) [md51.uzip] 5079 0(00.00) 0(00.00) [md52] 5080 0(00.00) 0(00.00) [md52.uzip] 7319 0(00.00) 0(00.00) [md53] 7320 0(00.00) 0(00.00) [md53.uzip] 7519 0(00.00) 0(00.00) [md54] 7520 0(00.00) 0(00.00) [md54.uzip] 7591 0(00.00) 0(00.00) [md55] 7592 0(00.00) 0(00.00) [md55.uzip] 8648 0(00.00) 0(00.00) [ftcleanup] 11499 3092(55.17) 3076(00.10) /sbin/jlaunchhelperd 11500 7476(42.79) 4884(00.15) /sbin/jlaunchd -N 11515 1884(91.63) 1296(00.04) /usr/libexec/bslockd -mp -N 11516 6576(71.05) 3808(00.12) /usr/sbin/tnetd -N -l 11519 20808(89.08) 11988(00.38) /usr/sbin/sdk-vmmd -N 11520 15596(00.00) 7164(00.22) /usr/sbin/cron -s 11523 16660(91.96) 10556(00.33) /usr/sbin/craftd -N 11525 64036(01.53) 42536(01.33) /usr/sbin/mgd -N 11531 13044(79.61) 6900(00.22) /usr/sbin/inetd -N -w -R 512 11536 10092(82.16) 5856(00.18) /usr/sbin/shm-rtsdbd -N 11537 7548(71.72) 4884(00.15) /usr/sbin/gstatd -N 11541 24064(56.35) 12244(00.38) /usr/sbin/agentd -N 11542 20380(91.55) 13528(00.42) /usr/sbin/jsrpd -N 11562 30204(44.02) 12372(00.39) /usr/sbin/jsd -N 11563 14480(92.65) 6736(00.21) /usr/sbin/xmlproxyd -N 11564 33316(72.15) 12880(00.40) /usr/sbin/dot1xd -N 11566 23652(00.56) 12552(00.39) /usr/sbin/utmd -N 11567 11844(91.96) 6784(00.21) /usr/sbin/sysctlrelayd -N 11568 28356(50.34) 3312(00.10) /usr/sbin/na-grpcd -c /opt/telemetry/na-grpcd/na-grpc-server.ini 11569 19108(79.33) 10944(00.34) /usr/sbin/pkid -N 11642 3096(55.33) 3064(00.10) /usr/libexec/getty.junos 3wire ttyu0 11646 62772(01.83) 41316(01.29) mgd: (mgd) /packages/mnt/junos-runtime/usr/sbin/mgd (root) 11650 2080(93.02) 1516(00.05) /sbin/watchdog -t-1 11734 17508(91.92) 10876(00.34) /usr/sbin/ppmd -N 11738 21080(81.64) 11432(00.36) /usr/sbin/bfdd -N 11739 24160(91.02) 12476(00.39) /usr/sbin/lacpd -N 11740 15616(80.07) 9408(00.29) /usr/sbin/lfmd -N 11742 20168(53.62) 9008(00.28) /usr/libexec32/smid -N 11743 6780(42.05) 6728(00.21) /usr/sbin/rpcbind -N -I -h 128.0.0.1 11744 2628(52.64) 2344(00.07) /bin/sh /etc/srx_sntpd.sh 11745 20484(34.49) 11488(00.36) /usr/sbin/jsqlsyncd -N 11748 2916(86.48) 2292(00.07) /usr/sbin/tnp.sntpd -JU __juniper_private4__ -N 11756 7276(45.10) 7128(00.22) /usr/sbin/na-mqttd -c /opt/telemetry/na-mqttd/na-mqtt.conf 11763 9416(92.57) 5524(00.17) /usr/sbin/fipsd -N 11855 0(00.00) 0(00.00) [ppt_0a_82100010] 11961 0(00.00) 0(00.00) /usr/sbin/license-check -U -M -p 10 -i 10 11999 0(00.00) 0(00.00) [ppt_0a_81100010] 27042 36736(00.88) 19028(00.60) /usr/sbin/aamwd -N 36202 24324(00.00) 11760(00.37) sshd: dario [priv] 36204 24764(00.00) 11848(00.37) sshd: dario@0 36205 92060(02.20) 31088(00.97) -cli 36206 64860(01.55) 42984(01.35) mgd: (mgd) (dario)/dev/pts/0 54612 21748(00.52) 8376(00.26) /usr/sbin/ntf-agent -N -c /var/etc/ntf-agent.conf 54769 15632(00.37) 7568(00.24) /usr/sbin/mgd-api -N 54850 24060(00.57) 13608(00.43) /usr/sbin/appsecured -N 54917 34696(00.83) 20552(00.64) /usr/sbin/eventd -N -r -s -A 54952 44160(01.05) 20768(00.65) /usr/sbin/idpd -N 56460 737304(17.60) 9468(00.30) /usr/sbin/httpd-gk -N 56469 23280(00.56) 10468(00.33) /usr/sbin/httpd --config /jail/etc/httpd.conf 56994 21988(00.00) 21984(00.69) /usr/sbin/xntpd -j -N -g -e 58251 95100(02.27) 65496(02.05) /usr/sbin/appidd -N 71763 36840(49.21) 12452(00.39) /usr/sbin/l2ald -N 71764 22556(00.54) 11188(00.35) /usr/sbin/sampled -N 71765 25880(00.62) 12488(00.39) /usr/sbin/rmopd -N 71766 34264(81.37) 13360(00.42) /usr/libexec32/cosd 71770 9448(76.22) 5704(00.18) /usr/sbin/jflowd -N 71771 10640(76.68) 8124(00.25) /usr/sbin/srrd -N 71772 16336(94.91) 8296(00.26) /usr/sbin/dood -N 71773 9816(92.01) 5584(00.17) /usr/sbin/pppd -N 71774 31924(71.14) 14160(00.44) /usr/sbin/l2cpd -N 71775 16244(90.47) 10432(00.33) /usr/sbin/oamd -N 71776 11160(52.62) 6128(00.19) /usr/sbin/mplsoamd -N 71777 12392(92.84) 7032(00.22) /usr/sbin/sendd -N 71778 16788(72.68) 10516(00.33) /usr/sbin/wwand -N 71779 6840(00.00) 6720(00.21) nfsd: master 71780 13524(77.37) 8008(00.25) /usr/sbin/commit-syncd -N 71782 6728(40.89) 6680(00.21) /usr/sbin/mountd -N -s -h 128.0.0.1 -p 6666 71783 9432(76.53) 5556(00.17) /usr/sbin/datapath-traced -N 71784 21864(71.27) 12068(00.38) /usr/sbin/smihelperd -N 71785 20280(94.50) 8516(00.27) /usr/sbin/grpc-tunnel -N 71787 6840(00.00) 6720(00.21) nfsd: server 71788 13632(91.51) 6768(00.21) /usr/sbin/nstraced -N 71789 12256(61.10) 6820(00.21) /usr/sbin/gprsd -N 71790 15340(74.90) 7320(00.23) /usr/sbin/syshmd -N 71792 9420(76.29) 5556(00.17) /usr/sbin/smtpd -N 71793 21528(96.33) 2564(00.08) /usr/sbin/wmic -N 71794 11820(77.34) 6392(00.20) webapid: master process /usr/sbin/webapid -N 71805 67804(55.64) 16328(00.51) /usr/sbin/chassisd -N 71806 25180(82.85) 13252(00.41) /usr/sbin/alarmd -N 71807 744448(17.77) 35332(01.11) /sbin/dcd -N 71810 17100(94.18) 10932(00.34) /usr/sbin/snmpd -N 71826 29620(00.71) 14240(00.45) /usr/sbin/mib2d -N 71828 27080(00.65) 14936(00.47) /usr/libexec32/pfed -N 71830 29528(74.08) 12500(00.39) /usr/sbin/kmd -N 71831 29040(60.34) 14884(00.47) /usr/libexec32/dfwd -N 71832 9524(76.61) 5512(00.17) /usr/sbin/irsd -N 71833 17216(86.70) 8196(00.26) /usr/sbin/bdbrepd -N 71834 59820(01.43) 37952(01.19) /usr/sbin/nsd -N 71835 21148(81.19) 10900(00.34) /usr/sbin/lsysd -N 71836 20136(74.56) 10988(00.34) /usr/libexec32/repd -N 71842 18884(71.04) 9744(00.31) /usr/sbin/dyn-filterd -N 71844 32964(36.58) 14140(00.44) /usr/sbin/ipfd -N 71845 19404(00.46) 11012(00.34) /usr/sbin/rtlogd -N 71846 11196(91.59) 6724(00.21) /usr/sbin/jkm -N 71850 22488(77.85) 8660(00.27) /usr/sbin/sub-brokerd -N 71851 22488(77.49) 8580(00.27) /usr/sbin/pub-brokerd -N 71884 11832(77.41) 6400(00.20) webapid: worker process 71935 0(00.00) 0(00.00) [ppt_07_82100001] 78368 24324(00.00) 11760(00.37) sshd: dario [priv] 78376 24328(00.00) 11772(00.37) sshd: dario@1 78377 92064(02.20) 31008(00.97) -cli 78378 64404(01.54) 42912(01.34) mgd: (mgd) (dario)/dev/pts/1 78451 897684(21.42) 107540(03.37) /usr/sbin/rpd -N 78673 64100(01.53) 42696(01.34) mgd: (mgd) (root)node1:--------------------------------------------------------------------------System memory usage distribution: Total memory: 3193324 Kbytes (100%) Reserved memory: 81736 Kbytes ( 2%) Wired memory: 2209252 Kbytes ( 69%) Active memory: 54220 Kbytes ( 1%) Inactive memory: 600576 Kbytes ( 18%) Cache memory: 0 Kbytes ( 0%) Free memory: 244860 Kbytes ( 7%)Pid VM-Kbytes( % ) Resident( % ) Process-name 0 0(00.00) 0(00.00) [kernel] 1 0(00.00) 0(00.00) /sbin/init -- 2 0(00.00) 0(00.00) [jfe_job_0_0] 3 0(00.00) 0(00.00) [jfe_job_1_0] 4 0(00.00) 0(00.00) [jfe_job_1_1] 5 0(00.00) 0(00.00) [jfe_job_2_0] 6 0(00.00) 0(00.00) [jfe_job_3_0] 7 0(00.00) 0(00.00) [jfe_job_3_1] 8 0(00.00) 0(00.00) [jfe_job_4_0] 9 0(00.00) 0(00.00) [jfe_job_4_1] 10 0(00.00) 0(00.00) [audit] 11 0(00.00) 0(00.00) [idle] 12 0(00.00) 0(00.00) [intr] 13 0(00.00) 0(00.00) [geom] 14 0(00.00) 0(00.00) [md0] 15 0(00.00) 0(00.00) [crypto] 16 0(00.00) 0(00.00) [crypto returns 0] 17 0(00.00) 0(00.00) [crypto returns 1] 18 0(00.00) 0(00.00) [md0.uzip] 19 0(00.00) 0(00.00) [cam] 20 0(00.00) 0(00.00) [usb] 21 0(00.00) 0(00.00) [sctp_iterator] 22 0(00.00) 0(00.00) [rand_harvestq] 23 0(00.00) 0(00.00) [kern_dump_proc] 24 0(00.00) 0(00.00) [pagedaemon]{primary:node0}
And this is the output of vmstat -m:
# vmstat -m Type InUse MemUse HighUse Requests Size(s) CAM queue 6 2K - 39 16,32,512 USB 20 23K - 22 16,128,512,4096 USBdev 18 4K - 24 32,64,128,256,512 vtbuf 24 1968K - 46 4096 vt 11 6K - 11 512 DEVFS2 231 15K - 845 16,32,64 DEVFS3 990 248K - 1174 256 DEVFS1 231 116K - 295 512 DEVFS 41 2K - 42 16,128 DEVFSP 0 0K - 6 64 msdosfs_node 7 2K - 468 256msdosfs_mount 1 1K - 1 512 msdosfs_fat 1 8K - 1 8192 nullfs_hash 1 512K - 1 nullfs_node 44 3K - 225 64 nullfs_mount 4 1K - 4 32 pfs_nodes 35 18K - 35 512 pfs_vncache 739 47K - 2985 64 tmpfs mount 2 1K - 2 128 tmpfs name 492 9K - 8200 16,32,64,128 GEOM 1272 178K - 7295 16,32,64,128,256,512,1024,16384 geom_uzip 224 3188K - 72684 128,256,512,1024,2048,4096,8192,16384,32768,65536 isadev 1 1K - 1 128 isofs_mount 56 14K - 56 256 isofs_node 12642 3161K - 12642 256CAM dev queue 3 1K - 3 64 cdev 4 1K - 4 256 filedesc 42 169K - 3491 16,4096,8192 sigio 2 1K - 640 64 filecaps 0 0K - 886 16,32,64,128 kdtrace 688 124K - 12174792 64,256 kenv 76 12K - 78 16,32,64,128,512,1024,8192 kqueue 318 69K - 3967386 64,128,256,512,2048 proc-args 120 6K - 3751504 16,32,64,128,256 hhook 16 3K - 24 16,32,256 ithread 202 30K - 202 32,128,256 prison 12 1K - 12 32 KTR 1 11264K - 1 KTRACE 100 13K - 100 128 ddb_capture 1 64K - 1 65536 linker 1276 10481K - 1775 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 acpiintr 1 1K - 1 64 lockf 372 40K - 82115451 64,128 loginclass 4 1K - 1177 128 devbuf 4278 3109K - 45057 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 temp 14375344 2020898K - 123160135 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 acpica 930 99K - 39673 16,32,64,128,256,512,2048,8192 module 396 50K - 396 128 mtx_pool 2 72K - 2 8192,65536 netstack 1 1K - 1 16 osd 978 17K - 23536461 16,32,64,128,256 pmchooks 1 1K - 1 128 pmc 1 1K - 1 64 pgrp 13 2K - 159315 128 session 10 3K - 159189 256 proc 2 2K - 2 1024 subproc 550 1189K - 3967357 512,4096 cred 139 35K - 1303168 256 acpitask 1 64K - 1 65536 acpisem 23 3K - 23 128 acpidev 21 2K - 21 64 plimit 47 12K - 2385539 256 uidinfo 4 1K - 400 128,256 dumper 1 1K - 1 512 CAM SIM 3 1K - 3 256 sysctl 1 1K - 864632618 32,64,128 sysctloid 10272 530K - 10348 16,32,64,128 sysctltmp 0 0K - 4891235 16,32,64,128,256,512,1024 CAM XPT 12 1K - 25 32,128,512,1024 CAM DEV 3 6K - 14 2048 tidhash 1 2K - 1 2048 callout 3 1672K - 3 umtx 940 118K - 940 128 p1003.1b 1 1K - 1 16 SWAP 2 132K - 2 65536 bus 1832 193K - 4378 16,32,64,128,256,1024 bus-sc 66 208K - 1927 16,32,64,128,256,512,1024,2048,4096,8192,16384,65536 CAM CCB 0 0K - 26 2048 devstat 58 117K - 58 32,4096 eventhandler 138 12K - 138 64,128 gtaskqueue 18 26K - 18 16,32,256,8192 kobj 189 756K - 550 4096 Per-cpu 1 1K - 1 32 CAM path 3 1K - 39 32 rman 150 18K - 469 16,32,128 sbuf 1 1K - 44336 16,32,64,128,256,512,1024,2048,4096,8192,32768,65536 sglist 21 14K - 21 32,512,1024 toponodes 8 1K - 8 128 CAM periph 2 1K - 15 16,32,64,128,256 taskqueue 54 6K - 54 16,32,256 terminal 11 3K - 11 256 Unitno 37 3K - 64743175 32,64 vmem 3 276K - 11 2048,4096,8192,16384,32768 ioctlops 0 0K - 10939 512,2048 select 172 22K - 172 128 iov 0 0K - 219075467 16,32,64,128,256,512,1024,2048 msg 4 30K - 4 2048,4096,8192,16384 sem 4 106K - 4 2048,4096 shm 21 224K - 2260 8192,65536 tty 15 15K - 79 1024 pts 2 1K - 66 256 mbuf_tag 76 4K - 16790309525 32,64,128 shmfd 121 61K - 345 64,256,1024,8192 soname 241 28K - 222948171 16,32,64,128 pcb 832 762K - 18294679 16,32,64,128,256,512,1024,2048,4096,8192 acl 0 0K - 13 4096 vfscache 4 1025K - 4 256,512 cl_savebuf 0 0K - 2233 64 vfs_hash 1 512K - 1 vnodes 5 1K - 8 32,256 mount 1127 47K - 1222 16,32,64,128,256,4096 statfs 0 0K - 6315993 4096 vnodemarker 0 0K - 11813919 512chacha20random 1 1K - 1 512 zlib 56 448K - 56 8192 crypto 2 2K - 2 256,1024audit_evclass 230 8K - 287 32 veriexec 10842 1347K - 11298 16,32,128,4096 pagedep 4 385K - 3605 256 inodedep 6 1538K - 25622 512 bmsafemap 4 25K - 15652 256,8192 newblk 3 3072K - 103333 256 indirdep 0 0K - 1122 128,32768 freefrag 0 0K - 6284 128 freeblks 0 0K - 12880 128 freefile 0 0K - 10613 64 diradd 0 0K - 16968 128 mkdir 0 0K - 82 128 dirrem 3 1K - 16988 128 newdirblk 0 0K - 41 64 freework 3 1K - 21475 64,128 sbdep 0 0K - 8671 64 savedino 0 0K - 3864 128,256 softdep 3 2K - 3 512 ufs_mount 9 46K - 13 512,2048,4096,8192 vm_pgdata 2 1K - 2 128 UMAHash 5 39K - 10 512,1024,2048,16384,32768 memdesc 1 4K - 1 4096 pci_link 10 1K - 10 16,128 atkbddev 1 1K - 1 64 ata_pci 1 1K - 1 64 apmdev 1 1K - 1 128 madt_table 0 0K - 2 16,4096 kbdmux 6 22K - 6 16,512,1024,2048,16384 md_disk 56 224K - 56 4096 entropy 2 1K - 14719 16,32,64,128,256,512,2048 intr 4 220K - 4 32768,65536 io_apic 1 2K - 1 2048 local_apic 1 2K - 1 2048 UART 4 12K - 4 16,1024,2048,8192 MCA 10 2K - 10 128 cpus 2 1K - 2 16 msi 39 5K - 39 128 nexusdev 6 1K - 6 16 hmac_drbg 0 0K - 2 128,256 lr 1 4K - 1 4096 pic 7 4K - 7 32,128,1024 pfestat 193 111K - 29126640 16,32,64,4096,8192 gencfg 1922 3157K - 839823845 16,32,64,128,512,1024,2048,4096,8192,16384,32768 jsr 4 1K - 88 32 idl 64 980K - 60916544 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768 rtsmsg 1 4K - 709396726 4096,8192,16384,32768 rtstat 166 5312K - 8694724 32768 mesh-group 4 1K - 4 ifl_qstats 52 208K - 56 4096 sample 9 8K - 9 32,1024,2048vpls_lc_instance 1 8K - 1 8192 ifa_list 41 1K - 51 16ifstateSLLNode 0 0K - 256 16lag_list_node 24 1K - 54 32gresstatevarlog 1 96K - 1 JFE job 1 1K - 1 1024 JFE PROFILE 45 51K - 45 16,64,2048JFE statistics 18 1387K - 18 8192,32768sysctl policing 899 43K - 899 jlist 1 1K - 1 schedhist 1 704K - 1 jlock 433 6185K - 51580 64,2048 gtable 1264 71K - 82864 64 DDI 6 1K - 6 in6ifmulti 3 1K - 6 64 ifl_idx_mgr 1 1K - 1 in6grentry 6 1K - 12 64 iflogical 190 123K - 205 16,128,1024,2048,4096 iffamily 204 73K - 228 16,32,64,2048 rtnexthop 932 375K - 644747216 16,256,512,1024,2048 metrics 16 4K - 47827 inifmulti 2 1K - 3 64 ingrentry 3 1K - 5 64 rnode 995 48K - 197470595 32,64,256 STP 73 17K - 76 16,128,512 rcache 4 16K - 4 4096 syncache 1 16K - 1 16384 fragment 0 0K - 73 ip6opt 1 1K - 1477061 32,128 tlv_stat 0 0K - 3452035 16384,32768,65536nhack_info_entries 4 1K - 141075022 32 ifdevice 46 12K - 64 16,32,64,128,2048 ifstat 429 427K - 1097953 16,32,64,1024,4096,8192 rtdata 1 1K - 1 64shared_bitvector 3 1K - 3 rts_osd_slot 57 1K - 837 16Bridge Domain 4 1K - 4 32 iflib 16 103K - 20 16,64,128,512,1024,4096,8192,16384,32768 ipfw 48 26K - 148 16,32,64,128,256,512,1024,2048,4096 ifmaddr 283 9K - 361 32 rtable 1040 129K - 197527015 16,32,128,512,1024 sysctl 0 0K - 89 256,512,1024,2048 ifaddr 112 12K - 132 32 mkey 847 27K - 23484333 32,128 pfe_ipc 3 1K - 11626247 16,32,64,4096,8192,16384,32768 ifstate 10497 796K - 1213550749 16,32,64,128,512,2048,16384,32768virtfs_mount_tag 10 1K - 10 16,32 p9_client 10 2K - 10 128 virtfs_mount 10 5K - 10 512 ifstrace 100129 8337K - 3968418849 64,128,256 itable16 1156 427K - 822710436 256,512 itable32 551 44K - 552 itable64 1 1K - 1 128 Export Host 2 2K - 2 1024 inpcbpolicy 257 9K - 553768 32 ipsecpolicy 514 129K - 1107536 256 solaris 397093 20229K - 397138 16,32,64,128,256,512,1024,2048,4096,8192,16384,32768,65536 fbt 90810 11608K - 90810 128 rpc 7 5K - 7 32,64,256,512,4096NFSD V4client 1 1K - 1 256 NFSD lckfile 1 1K - 1 256 NFSD session 1 1K - 1 1024 SDT 58 3K - 58 16,64
Do you see anything strange?
Thanks :)
------------------------------
DARIO PULIERO
Original Message:
Sent: 05-07-2025 14:11
From: Nikolay Semov
Subject: High routing memory utilization
Interesting ...
So, the total for all processes comes to about 1.8 GB. show system memory may be an even better command for this.
You do have 2.4 GB wired memory though; that seems excessive. Maybe something's leaking in the kernel, or there's lots of cached I/O. Do you have any active traceoptions statements in your config?
You can also start shell user root and use vmstat -m to take a peek at some kernel memory allocations to see if anything would pop out there...
------------------------------
Nikolay Semov
Original Message:
Sent: 05-07-2025 08:32
From: DARIO PULIERO
Subject: High routing memory utilization
Hi Nikolay,
in attach the output of "run show system processes extensive".
everything looks okay to me.
------------------------------
DARIO PULIERO
Original Message:
Sent: 05-06-2025 10:41
From: Nikolay Semov
Subject: High routing memory utilization
Did you ever run show system processes extensive over time to see which process is taking up more and more memory? Having no additional information beyond what you mentioned last time is unlikely to result in any revelations.
------------------------------
Nikolay Semov
Original Message:
Sent: 05-06-2025 03:05
From: DARIO PULIERO
Subject: High routing memory utilization
Hi all,
the routing engine memory utilization (min 1.3.6.1.4.1.2636.3.1.13.1.11.9.1.0.0) is 91% ...Can anyone help me understand why?
as in the past, if I don't restart the device, it will get very close to 100% and stop working properly
Thanks

------------------------------
DARIO PULIERO
Original Message:
Sent: 04-11-2025 04:18
From: DARIO PULIERO
Subject: High routing memory utilization
Hi,
I've already done this command but it cleans the storage but not the buffers
------------------------------
DARIO PULIERO
Original Message:
Sent: 04-10-2025 22:40
From: eugene1973
Subject: High routing memory utilization
Perhaps
request system storage cleanup
https://community.juniper.net/discussion/srx550-is-showing-100-cpu-for-many-days
------------------------------
Adrian Aguinaga
B.S.C.M. I.T.T. Tech
(Construction Management)
A.A.S. I.T.T. Tech
(Drafting & Design)
Original Message:
Sent: 04-09-2025 02:51
From: DARIO PULIERO
Subject: High routing memory utilization
Thank you, I'm reading the PRs and hope to find the cause of the problem.
------------------------------
DARIO PULIERO
Original Message:
Sent: 04-08-2025 14:54
From: spuluka
Subject: High routing memory utilization
There are 14 PR where memory leaks are associated with RPD. You could have a look at the details of each and see if the configuration and related processes are applicable to your situation and any of these potential are related.
https://prsearch.juniper.net/home?stype=affectingthis&ostype=junos&sid=SRXSeries-junos&bv=22.4R3-S2&uv=&qt=memory%20rpd
------------------------------
Steve Puluka BSEET - Juniper Ambassador
IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP - Retired)
http://puluka.com/home
Original Message:
Sent: 04-08-2025 03:08
From: DARIO PULIERO
Subject: High routing memory utilization
Hi Spuluka,
The currently installed version is 22.4 R3-S2 but the problem also occurred in previous versions. In the past, JTAC asked me to update the devices to this version which they recommended. Every time JTAC has not been able to find the cause of the problem and there seems to be no clue in the logs. The only solution is to turn the devices off and on again.
------------------------------
DARIO PULIERO
Original Message:
Sent: 04-07-2025 16:24
From: spuluka
Subject: High routing memory utilization
I agree this is potentially a memory leak which would be a bug in the version of Junos you are currently running. What is your current version?
There is a public bug search site, Juniper calls the PR (problem reports). If you find the one that describes your symptoms it will list the version of Junos you will need to upgrade for the bug fix.
https://prsearch.juniper.net/home
------------------------------
Steve Puluka BSEET - Juniper Ambassador
IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP - Retired)
http://puluka.com/home
Original Message:
Sent: 04-07-2025 05:54
From: DARIO PULIERO
Subject: High routing memory utilization
Hi all,
I have a doubt about the behavior of an SRX4100 cluster regarding the use of the routing engine memory.
Specifically, the memory is never cleaned, reaching almost 100%. In the past, I noticed that when memory usage approaches 100%, the device seems to not work correctly (it does not process policies correctly) and, moreover, it responds very slowly from the CLI.
Attached you can see the screenshot of the graph related to the use of the routing engine memory (or routing engine buffer pool).
Does anyone experience the same behavior?
Regards
------------------------------
Dario Puliero
------------------------------