SRX

I've been struggling to convert a configuration from 12.3/21.4 to 23.4.

The configuration appears to be valid but the issue is I can't run a speedtest (Ookla cli version) and get a vague cannot read error. When I go to certain, but not all, websites they time out. If I use the default 23.4 version it works but its default version is different from 12.3's. The 23.4 default configuration is the same as 21.4.

Basically my configuration has several address-assignment pools that point to a router IP. The router IP is defined in interfaces irb. I have vlans that associate the ID with l3-interface irb.n. WAN is defined in zones security-zone untrust interfaces. Finally I have system services dhcp-local-server that point to irb.n. My ethernet interfaces have family ethernet-switching where they reference vlan members.

In 21.4/23.4, the default configuration have interfaces with family inet with a router IP and there is only 1 address-assignment pool (192.168.2.0/24). It has a dhcp-attributes propagate-settings ge-0/0/0.

My configuration works under 21.4 but not 23.4.

What am I doing wrong?

Here's my config that works under 12.3 and 21.4. Instead of including all my vlans, I just include 1. Here xe-0/0/19 is the WAN and xe-0/0/17 is where a workstation can get an IP from 192.168.3.0/24.

system {
    services {
        dns {
            dns-proxy {
                interface {
                    irb.0;
                }
            default-domain * {
                forwarders {
                    1.1.1.1;
                }
            }
        }
        dhcp-local-server {
            group jdhcp-group {
                interface irb.0;
            }
        }
    }
}
security {
    nat {
        source {
            rule-set trust-to-untrust {
                from zone trust;
                to zone untrust;
                rule source-nat-rule {
                    match {
                        source-address 0.0.0.0/0;
                    }
                    then {
                        source-nat {
                            interface;
                        }
                    }
                }
            }
        }
    policies {
        from-zone trust to-zone untrust {
            policy default-permit {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
    }
    zones {
        security-zone trust {
            host-inbound-traffic {
                system-services {
                    all;
                }
                protocols {
                    all;
                }
            }
            interfaces {
                irb.0;
            }
        }
        security-zone untrust {
            screen untrust-screen;
            interfaces {
                xe-0/0/19.0 {
                    host-inbound-traffic {
                        system-services {
                            dhcp;
                            ping;
                            ntp;
                        }
                    }
                }
            }
        }
    }
interfaces {
    xe-0/0/17 {
        unit 0 {
            family ethernet-switching {
                vlan {
                    members vlan-trust;
                }
            }
        }
    }
    xe-0/0/19 {
        unit 0 {
            family inet {
                dhcp {
                    update-server;
                }
            }
        }
    }
    irb {
        unit 0 {
            family inet {
                address 192.168.3.254/24;
            }
        }
    }
}
access {
    address-assignment {
        pool DefaultPool {
            family inet {
                network 192.168.3.0/24;
            range 1 {
                low 192.168.3.100;
                high 192.168.3.199;
            }
            dhcp-attributes {
                router {
                    192.168.3.254;
                }
            }
        }
    }
}
vlans {
    vlan-trust {
        vlan-id 3;
        l3-interface irb.0;
    }
}

Here's the config that won't work under 23.4. xe-0/0/19 and xe-0/0/17 mirror the working 23.4 default configuration and that works. But xe-0/0/18 and xe-0/0/16 are converted from my original configuration and that doesn't work. In this current configuration xe-0/0/18 does get an IP (it's actually connected to my SRX running 21.3) but when I connect my workstation to xe-0/0/16 I get a 192.168.2.2 IP and there's no route to the internet. I tried adding propagate-settings xe-0/0/18 but that doesn't make any difference. If I reconfigure xe-0/0/16 into family inet with the appropriate router IP and place the interface to jdhcp-group then it works. But I want to define a trunk so I could pass all my VLANs to my switch.

system {
    services {
        dhcp-local-server {
            group jdhcp-group {
                interface ge-0/0/1.0;
                interface xe-0/0/17.0;
                interface irb.4;
            }
        }
    }
    name-server {
        8.8.8.8;
        8.8.4.4;
    }
}
security {
    screen {
        ids-option untrust-screen {
            icmp {
                ping-death;
            }
            ip {
                source-route-option;
                tear-drop;
            }
            tcp {
                syn-flood {
                    alarm-threshold 1024;
                    attack-threshold 200;
                    source-threshold 1024;
                    destination-threshold 2048;
                    timeout 20;
                }
                land;
            }
        }
    }
    nat {
        source {
            rule-set trust-to-untrust {
                from zone trust;
                to zone untrust;
                rule source-nat-rule {
                    match {
                        source-address 0.0.0.0/0;
                    }
                    then {
                        source-nat {
                            interface;
                        }
                    }
                }
            }
        }
    }
    policies {
        from-zone trust to-zone trust {
            policy default-permit {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        from-zone trust to-zone untrust {
            policy default-permit {
                match {
                    source-address any;
                    destination-address any;
                    application any;
                }
                then {
                    permit;
                }
            }
        }
        pre-id-default-policy {
            then {
                log {
                    session-close;
                }
            }
        }
    }
    zones {
        security-zone trust {
            host-inbound-traffic {
                system-services {
                    all;
                }
                protocols {
                    all;
                }
            }
            interfaces {
                xe-0/0/17.0;
                irb.4;
            }
        }
        security-zone untrust {
            screen untrust-screen;
            interfaces {
                xe-0/0/18.0 {
                    host-inbound-traffic {
                        system-services {
                            dhcp;
                            ntp;
                            ping;
                        }
                    }
                }
                xe-0/0/19.0 {
                    host-inbound-traffic {
                        system-services {
                            dhcp;
                            ntp;
                            ping;
                        }
                    }
                }
            }
        }
    }
}
interfaces {
    xe-0/0/16 {
        unit 0 {
            family ethernet-switching {
                vlan {
                    members vlan-trust;
                }
            }
        }
    }
    xe-0/0/17 {
        unit 0 {
            family inet {
                address 192.168.2.1/24;
            }
        }
    }
    xe-0/0/18 {
        unit 0 {
            family inet {
                dhcp {
                    update-server;
                }
            }
        }
    }
    xe-0/0/19 {
        unit 0 {
            family inet {
                dhcp {
                    update-server;
                }
            }
        }
    }
    irb {
        unit 4 {
            family inet {
                address 192.168.4.254/24;
            }
        }
    }
}
access {
    address-assignment {
        pool junosDHCPPool {
            family inet {
                network 192.168.2.0/24;
                range junosRange {
                    low 192.168.2.2;
                    high 192.168.2.254;
                }
                dhcp-attributes {
                    router {
                        192.168.2.1;
                    }
                    propagate-settings xe-0/0/19.0;
                }
            }
        }
        pool DefaultPool {
            family inet {
                network 192.168.4.0/24;
                range junosRange {
                    low 192.168.4.100;
                    high 192.168.4.199;
                }
                dhcp-attributes {
                    name-server {
                        192.168.4.254;
                    }
                    router {
                        192.168.4.254;
                    }
                }
            }
        }
    }
}
vlans {
    vlan-trust {
        vlan-id 4;
        l3-interface irb.4;
    }
}

The configuration is not the problem but either lies in the base drivers or network parameters/optimizations in 23.4.

Because there were errors associated with specific websites (the same sites) or programs (Ookla's speedtest cli) I inspected interface statistics using ethtool. There were some errors, but not a whole lot. Several hundred rx_long_length_errors while all other error counters were 0. They increased by around 5 each time I performed the speed test. I have no idea what this statistic means, so decided to use the onboard NIC to compare. The onboard works but downloads were slow (200 Mbps) compared to upload (900 Mbps).

The problematic interface is an Intel X540-AT2 while the onboard is an i350 in a Supermicro X9DRi.

I swapped the X540 with another i350-based card and it works without issue. Speeds of 900 Mbps upload/download. Websites don't timeout. Error counters all 0. This NIC is a newer one without any heatsink on the controller.

I decided to try an Intel 82599es-based NIC and it has the same problem as the X540. Error reading when trying to perform a download test. Using the open source speedtest-cli, the download test fails (0 Mbps) while the upload is capped to 2 Gbps (normal limitation for the program).

Finally I tested using a Mellanox ConnectX-3 and that worked without any issues.

The X540 doesn't have flow control enabled and it can't be changed. The 82599es has flow control enabled.

My test system is running Ubuntu 20.04 LTS.

I initially encountered this problem when I fresh-installed 23.4 with my current configuration and noticed I was getting errors from my test system. As soon as I downgraded to 21.4 keeping the configuration, everything worked.

Anyone have any insights they could share regarding what tuning could be performed or if what driver tweaks were committed between 21.4 and 23.4?

JunOS is pretty monolithic in the sense that you're unlikely to get much details as to what changes were made under the hood from 21.4 to 23.4. I suspect you would, however, find fingers pointed at the SFP+ module you're using on the SRX.

I used Arista modules but will try it with some Juniper ones. Personally I am leaning towards the firmware since the onboard i350s in the deprecated MB has issues while the newer card does not and they are copper.

I was able to follow up the issue I was experiencing by obtaining new firmware for one of my NICs. Not the PCIe 82599es one but for the i350 onboard NIC. After the upgrade, I no longer have the slow (200 Mbps) download transfer.

It's possible the issue only occurs if I'm directly connecting to the SRX but I haven't had the time to test if I still experience the problem by connecting to a switch.

I just tried an NIC based on an Intel X550 with copper ports and while it exhibited the same symptoms, I replaced the inbox drivers of Ubuntu 20.04 (5.15.0) with the latest from Intel (5.20.3) and that fixed the issue. But 5.20.3 doesn't fix the problem with the 82599es-based NIC.