SRX

Hi All,

   I know this is a sore point for many users but I would just like to try and iron out the best practises for this port and have this as a point of reference for anyone trying to work with this god forsaken port.

If an enterprise had SRX (whats the plural of SRX SRXs?) distributed across the country and wanted to use these fxp0 ports to manage the box out of band does this essentially mean that enterprise would need one big flat Layer 2 management vlan network spanning the country  in order to connect every fxp0 interface into since the port cannot be used as a transit port to route traffic?

How have you guys implement this port or SRX management in general?

Thanks!

If you have a real out of band network, that can be used to manage the devices, but few people do. If there is a management network, its often just another VLAN on the switches. So what I usually do is a setup with two routing instances. I think I have already described the setup here before, but its something like this:

      ISP

       |

--------------

| DEFAULT-VR |  fxp0 (1.1.1.10/24) ---|

|            |  fxp0 (1.1.1.20/24) ---|

--------------                        |

                                  MGMT VLAN

---------------                       |

| INTERNAL-VR |  reth0 (1.1.1.1/24) --|

|             | 

---------------

  |        |

 WAN      LAN

All routes are exchanged between the two VRs, except for routes to the MGMT vlan. If I need to manage this device over a WAN connection, that is plugged in on the internal VR, traffic is routed out through reth0 so I can access each fxp0 address (with the necessary backup-router configs of course).

If I need to manage the devices through a VPN connection (scary, but it happens), place the st0.X interface in the internal-vr and everything works in the same manner.

I have used this setup and slight varions several times and it works pretty well. The advantage is that I get logs from both devices in NSM (if traffic levels aren't too high). The alternative is placing the two SRXes in VC-mode for management but then you loose the logs from the backup node, which might not seem like a big problem but if RG0 and RG1 are active on different nodes, that means you no longer get traffic logs.

And as soon as we can terminate VPN connections in non-default routing instances, the ISP link can be moved to the internal instance and all the special route-distribution config can be deleted 🙂

Hi MOTD,

    Thanks for the great response I just have a question about this design.

If we take the WAN example, we have a user coming from the WAN with a desitination IP of fxp0 1.1.1.10 the traffic will enter the SRX, go out the reth0 interface and hit the fxp0 interface.  Now the return traffic will have to use the default-VR to get back into reth0 and back out the WAN.  How does the return route work?

The one to which I currently have access doesn't in fact exchange any routes between the two instances, except for the default route so that client can access the internet.

In the default-vr there is indeed a static route for the internal networks pointing to the reth0 address.

I have a second cluster where I do exchange all routes as I described previously, but I think there I have created a static route in the default instances for the IPs of my NSM servers. But I can't access that device to make sure. The problem with not distributing all routes is that some ALGs (like FTP) stop functioning correctly.

The main problem with this setup is that it is way too complicated for most people to understand. I can't even remember sometimes 🙂

@motd: Yes, ingenious, and hard to understand, I couldn't agree more. In addition, services that are not supported in a VR will be unavailable, such as DHCP. Not every environment may need those services, of course.

What I have been doing is use a reth interface for management, using the cluster support in NSM. Alternatively, create a management network that is not routed to the Internet.

That leaves NTP as an issue to be solved. One reasonably easy way to solve it is to give the NTP server a second IP in the fxp0 range. While you're at it, you can also give NSM a leg in that range, and maybe a server or two that will be used for ssh/https access. That way, there is an additional management option in addition to the inline reth management.

It's not perfect. I think I understand why things work this way - without a routing engine active, fxp0 can't be placed into its own VR like we can do in ScreenOS. That doesn't mean I like it.

@tbehrens Yes, I forgot about DHCP. Its not supported apparently, which came as a surprise to me because I've been using relays since 9.6. It does work in certain situations but I'm told it depends on the behavior of the DHCP servers (not windows). I need to find some detailed information about what is and what isn't supported as its not even mentioned in the release notes.

VC-mode is indeed by far the easiest, but you loose access to the logs of the backup device. I don't care that much about the event logs but if you have a failover and RG0/RG1 are running on different nodes, that means no more traffic logs either. There is an easy fix for that though, make NSM receive traffic logs via SYSLOG instead of the **** way logging is done now. It would also be more stable and faster, but that fix seems to be to obvious for product management to accept 😉

The separate management network is indeed the best approach. That will work for a datacenter, but not when the clusters are in remote sites, which I believe was the original question. Imagine having two SRX clusters in two different locations without Layer-2 connectivity between them, trying to find a way to manage them with a single NSM server. I would love to learn about an easier solution though, but I've almost given up wishing for miracles 🙂

Hi Guys,

   Are you saying that DHCP relay wont work across a VR.

ex)  A host (192.168.1.10) lives in Main-VR and my server (10.10.10.10) lives in Server-VR.  I have a route from Main-VR to Server-VR for 10.10.10.0/24 and setup a bootp helper address of 10.10.10.10 it wont work?

DHCP relay will only work on interfaces that are bound to the default inet.0 routing instance.

If you have an interface or interfaces bound to a different VR, then DHCP relay will not work for them.  I have first-hand experience on this.

keithr@srx> show system services dhcp relay-statistics 
  Received packets: 110
  Forwarded packets: 0
  Dropped packets: 220
    Due to missing interface in relay database: 0
    Due to missing matching routing instance: 0
    Due to an error during packet read: 0
    Due to an error during packet send: 220
    Due to invalid server address: 0
    Due to missing valid local address: 0
    Due to missing route to server/client: 0

I have a case open, 2010-1213-0658.  There is an associated PR for this problem listed in my case, PR571662.

I'm being told that Juniper has no ETA on when it will be fixed, which to me translates to "they don't consider it a high priority."  Meanwhile, this extremely basic functionality worked perfectly well for years and years on the old ScreenOS platforms, even the crusty old NS500 and NS5GT.

Perhaps if a few people started putting a little more pressure on Juniper for these types of things...

Thanks Keith.  This is a major issue.  Just to clarify.  Would the DHCP server need to live on the default inet.0 or the host requesting the DHCP.

Do you have any work around you have come up with ( asides from having a DHCP server in each VR.

The DHCP server can live anywhere.

The interfaces that "intercept" the DHCPDISCOVER broadcasts and perform the relay must be in the inet.0 routing instance.

So, if you have a subnet 192.168.1.0/24 and you use interface vlan.1 as the gateway interface (for example, 192.168.1.1), and you have your DHCP relay set like this:

forwarding-options {
    helpers {
        bootp {
            relay-agent-option;
            description "DHCP Server";
            server 10.10.10.1;
            interface {
                vlan.1;
            }
        }
    }
}

Since interface vlan.1 is performing the relay, vlan.1 must be in inet.0.

As a workaround for this, if you have subnets that must live in a different VR and need DHCP services, the only thing I've come up with to this point is to have another switch in that VLAN do the DHCP relay rather than rely on the SRX.

Do you know if this issue is isolated to SRX.  Would i have this issue with en EX-4200?  My point is that I have an EX-4200 with 3 VR's.  Nothing actually lives in the main VR all my interfaces are a member of the created VR's.

I don't know, we don't use the EX switches so I've never had a chance to test it.

There are Juniper employees lurking around here, I'm sure one of them will chime in.  

This is the list of features that need to stay in inet.0 as per my notes. May not be complete, or still accurate - it's from Q1/2 last year.

The following features will only work in the default routing instance

• IKE termination
• DHCP client
• DHCP server
• DHCP relay
• Signature updates (i.e. for IDP/AV updates, the update server must be reachable through the default vr)
• In general any device-outbound traffic will only be sent through the default instance (i.e. NTP, syslog, etc)
• Management traffic (fxp0)

Supported in a VR since 10.4:

• IPSec/VPN termination - st0 interface, point-to-point and point-to-multipoint

Hi keithr,

we have a dhcp relay on a interface bound to a vr, dhcp server is although reachable via the same vr. This works without any problems:

set forwarding-options helpers bootp relay-agent-option
set forwarding-options helpers bootp server ip1 routing-instance prod-vr
set forwarding-options helpers bootp server ip2 routing-instance prod-vr
set forwarding-options helpers bootp interface reth1.2870
set forwarding-options helpers bootp interface reth1.2872

...

set routing-instances prod-vr interface reth1.2870
set routing-instances prod-vr interface reth1.2872

show system services dhcp relay-statistics
node1:
--------------------------------------------------------------------------
  Received packets: 20306
  Forwarded packets: 37718
  Dropped packets: 0
    Due to missing interface in relay database: 0
    Due to missing matching routing instance: 0
    Due to an error during packet read: 0
    Due to an error during packet send: 0
    Due to invalid server address: 0
    Due to missing valid local address: 0
    Due to missing route to server/client: 0

This works on

Model: srx5800
JUNOS Software Release [10.2R2.11]

/sok

Hi All,

   The reason it does not work is because the relay gets sent out using the master.inet.0 table.  If you import the route of the server into your main table it works fine.

Thanks

Ugh, just ran into this show-stopping issue configuring virtual routers with dhcp.  

Juniper I am begging you for an easy configuration of basic dhcp services within virtual router instances.

Magraw, Can you give an example please?  This DHCP issue throws a major snag at me.