SRX

 View Only
last person joined: 20 hours ago 

Ask questions and share experiences about the SRX Series, vSRX, and cSRX.
  • 1.  How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-29-2020 13:13

    Hi Folks,

    Got an SRX300 (single fw) with pretty basic setup - trust/untrust and Src NAT with the egress interface public IP. One other  zone for testing called labnet zone. Same policy as trust....labnet -> untrust allow and use egress IP as the source NAT IP.

    Simple policy, allow all out from trust to untrust. Nothing from untrust to trust. And allow all out from labnet to untrust.

    At any rate, what I'm seeing is some packet loss when traversing the firewall. In other words, I can ping all day long from any host on the network to the trust interface (in this case 10.0.0.1) and get ZERO packet loss (just testing with ICMP).

    But when I ping either the untrust interface IP or the next hop gateway (ISP) or anything on the otherside of the firewall; I'll get some packet loss. I get the fact, I can't control any packet loss once it hits the ISP gateway. But I should NOT see any packet loss just doing a normal 64byte ping to my untrust interface IP.  Network traffic is pretty minimal. All Gig attached interfaces...and most of the time the interfaces are well under 100Mbps of traffic.

    Looking at my MRTG graphs of my interfaces (granted its only 5min snapshots...but the history shows pretty much the same)

    Max Average Current
    In 24.3 Mb/s (2.4%) 172.3 kb/s (0.0%) 65.4 kb/s (0.0%)
    Out 20.9 Mb/s (2.1%) 472.2 kb/s (0.0%) 645.9 kb/s (0.1%)

    So no heavy load.

    Any thoughts on how best to track this down?

    If it was a wiring or downstream switch issue...I'd see packet loss on both sides of the firewall. But I'm not...its ONLY when traversing the firewall.

    The two inside zones are on IRB interfaces (RVI)  so I'm using 'family ethernet-switching' on each of the inside interfaces so my downstream switch I can do .1Q tagging.

    ge-0/0/1 {
    description "Link to downstream switch";
    unit 0 {
    family ethernet-switching {
    interface-mode trunk;
    vlan {
    members all;
    }
    }
    }

    protocols {

    l2-learning {
    global-mode switching;

    }

    I have not tried routed/logical/sub-interfaces so I do all the switching on the downstream switch. The idea was to use global-mode switching so I could use the switch chip on the front panel ports of the SRX to switch between VLANs if needed.

    Currently running Junos 18.4R3.3. Was running 18.2 and same behavior is experienced.

    Would you recommend just turning on security flow packet tracing, such as below?

    security{

    flow {
    traceoptions {
    file mycap.txt size 1m files 2 world-readable;
    flag packet-drops;
    flag basic-datapath;
    packet-filter c2s {
    source-prefix 10.0.0.54/32;
    }
    }
    }

    Thanks!

    -J



  • 2.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-29-2020 16:30

    So, as an additional step of isolating where the packet loss is occurring....I went ahead and setup an RPM probe to do similar testing.

    ICMP probe test every 5 seconds to my ISP gateway.

    &

    HTTP GET from google.com

    So far, ZERO packet loss to each of these two endpoints. So this is leading me to believe that it is indeed flowd that is the culprit here.  I guess I'll try datapath debug to see if I can find the drops; unless you all have another suggestion here?

    Thanks.

    -J




  • 3.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-30-2020 02:31
    Edited by Hector Fuentes 12-30-2020 02:32

    One  common thing  is to have a set of screens configured on the untrust zone. Perhaps you are hitting those protection rails. There are a few types of screens, particularly, you may want to look at the Statistics-Based Screens (ICMP-based flood protection). 

    One quick way to check your active screens on each security zone is:

    user@host> show security zones



    ------------------------------
    Hector Fuentes
    ------------------------------



  • 4.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-30-2020 02:55

    Dear

     

    Best Regards,

    Swan Htet

    Senior Network Engineer

    ATG SYSTEMS CO,.LTD

    Contact: +959972509508

    Email: swanhtet@atgsys.com

     

     

     

     






  • 5.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-30-2020 07:28
      |   view attached

    Unfortunately, it doesn't appear to be hitting any screens. Counters are clean.

     #show security screen statistics zone untrust
    Screen statistics:

    IDS attack type Statistics
    ICMP flood 0
    UDP flood 0
    TCP winnuke 0
    TCP port scan 0
    UDP port scan 0
    ICMP address sweep 0
    TCP sweep 0
    UDP sweep 0
    IP tear drop 0
    TCP SYN flood 0
    SYN flood source 0
    SYN flood destination 0
    IP spoofing 4
    ICMP ping of death 0
    IP source route option 0
    TCP land attack 0
    TCP SYN fragment 0
    TCP no flag 0
    IP unknown protocol 0
    IP bad options 0
    IP record route option 0
    IP timestamp option 0
    IP security option 0
    IP loose source route option 0
    IP strict source route option 0
    IP stream option 0
    ICMP fragment 0
    ICMP large packet 0
    TCP SYN FIN 0
    TCP FIN no ACK 0
    Source session limit 0
    TCP SYN-ACK-ACK proxy 0
    IP block fragment 0
    Destination session limit 0
    IPv6 extension header 0
    IPv6 extension hop by hop option 0
    IPv6 extension destination option 0
    IPv6 extension header limit 0
    IPv6 malformed header 0
    ICMPv6 malformed packet 0
    IP tunnel summary 0

    Interestingly, when I turn on flow tracing, I do see the packet drop, which is weird because all the other similar flows (just ICMP ping with same SRC/DST) are not dropped. Attached is an excerpt of the trace with the first flow being ok and then the last one being dropped.

    -J


    Attachment(s)



  • 6.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-30-2020 07:48

    Turn off source nat for traffic from 10.254.12.2  to  75.145.78.9




  • 7.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-30-2020 12:33

    Why would I want to turn off source nat to the public address?

    In any case, this is just an example of the drops. Any public address (it could be google.com) I'll see the random drops and its flowd that is doing this. With the RPM probes I get zero packet loss. Its only when going through flowd that I see these random drops. As you can see in the trace debug,  it appears to be translating to the wrong incoming address. Its translating to 10.0.0.32 and not to 10.254.12.2

    Not sure why that is the case. Looks like a bug to me....

    I'll try and capture the random packet loss to google or other public IP.




  • 8.  RE: How to isolate random packet drops on SRX (SRX300 / branch)?

    Posted 12-30-2020 12:49

    As per my understanding from flow trace, the ip 75.145.78.9 is configured on ge-0/0/5 (untrust) interface. The traffic from 10.254.12.2 to 75.145.78.9  is getting source natted to ge-0/0/5 interface ip. i.e 75.145.78.9.  That is why , I suggested to turn off source nat for this specific traffic.

    Dec 30 04:41:49 04:41:49.094908:CID-0:RT:flow_first_src_xlate: src nat returns status: 1, rule/pool id: 9/2, pst_nat: False, nat_eim: False.

    Dec 30 04:41:49 04:41:49.094908:CID-0:RT: dip id = 2/0, 10.254.12.2/1->75.145.78.9/1252 protocol 1