Switching

My company bought 7 Juniper EX3200 series switches to be deployed at our new DR site. Our company policy is to completely isolate management traffic from regular data traffic. In this regard, a dedicated out-of-band management interface is the preferred way to remotely mange any device.

My ordeal began as soon as I started setting up the switches. The advertised "out-of-band management" interface was simply not working as it should. In fact, there was no way to set up a separate default-gateway to the management interface. As I started looking into it, I was taken back when I realized that the management interface was in fact sharing the same routing instance as the regular copper interfaces.

JTAC was of no support in this regard. They kept telling me that it is not possible to route between the management interface and the copper interfaces. I don't have a problem with it. In fact you should NEVER be able to pass traffic between the management interface and the copper interfaces. In this regard, the routing instance should NOT be shared between the management interface and the copper interfaces as it negates the whole framework. I spoke with two JTAC personnel on this issue, who did not seem to understand the concept of out-of-band management in the first place.

So, has anyone successfully implemented true out-of-band management on EX series switches? Can the Juniper switches support this feature at all?

I have never used the EX switches but I have used many SRXs which support full out-of-band management (which is sometimes a pain with clusters). I would be surprised if the EX didn't support it too.

The EX switch is indeed doing routing. Infant, the switches are participating in OSPF.

A little background - From our HQ we have two IPSec VPNs to the DR site. One VPN tunnel carries all data traffic and the other VPN tunnel is used to access the OOB network. There are lot of devices on the OOB network including other network hardware and servers accessible via IPMI (iLOMs, iDRACs etc.). We can access all the other devices EXCEPT the EX3200 switches.

Because the routing instance is being shared, and the switch is participating in OSPF, the front routing interfaces know the path back to the HQ via the copper ports. So, I try to SSH into the switch's Management interface, the switch receives my initial SYN packet, and responds with a SYN, ACK  but sends this packet out via the front copper interfaces with a source address of the management interface.

I have verified this behavior with wireshark packet captures and my firewall logs. (firewall logs this behavior as suspicious and drops the SYN, ACK as the initial SYN packet was not handled by it)

Contacting JTAC for assistance on this issue was a complete waste of time. The person I spoke with had no idea about OOB framework and why it is important. In fact he was down right condescending and told me I did not understand the concept of a dedicated management interface. He wouldn't listen to me and everytime I tried to explain what I was trying to do, he just responded back with canned responses.  I have a feeling he was just reading back from some sort of technical document about what the management interface can and cannot be used for.

At this point the best solution seems to be from "wimclend", that is to assign one of the copper interfaces to a separate routing instance and use that to manage the switch. But is is surprising that Juniper is not supporting this feature which has been around for quite some time now.

Having a dedicated OOB interface was one of the reasons we choose the EX3200. As of writing this post, I am using a policy based NAT to manage the EX3200 to make it seem like I am managing them from the same subnet as the OOB network.

This has been a setback for us and we had to push back our live dates by a couple of days. However, I don't think Juniper is going to be on the table the next time we make a purchase or an upgrade.

Hi rtalipinani,

My name is Alper and I'm with the EX support team. I would like to take a look at the cases you opened with JTAC. Can you please let me know the case numbers?

Thanks,

-Alper

Don't forget there still is a big difference between using the management port and using a regular switch port for management. Using one of the regular switch ports you'll have to explicitly disable it every time you configure a protocol (like STP or whatever). We tried simulating out-of-band that way on another vendors switch which didn't even have a management port, it was a major PITA.

I'd just stick with NAT - its the easiest solution and prevents a lot of other potential issues.

Sure, everyone would like a separate management routing instance, thats probably one of the oldest feature requests. But they do have a point, the reason we run into this issue is because our OOB networks are not truly separate.

Yes I agree that the me interface been to be by default in a seperate routing instance.

We have also had to deploy NAT on the gateway routers facing the me interfaces.

As we have these boxes dotted all over the place this a pain in the ***.

Specifically as you dont want you internal mgmt network routes inside the inet.0 table.

the only other thing we do is set static routes to point to the management subnet out of the me interface and mark these statics with the no-advertise flag so they are not propogated to neighbors.

3 years down the line and I'm facing the same issue.

I almost went the other way around put all my produciton L3 ports into a routing instance, but then i have to rely on my OOB network for management.

In my case, the OOB network is unreliable in that we use DSL to connect our sites' OOB networks back to the HQ. As far as I'm concerned, OOB is an alternate access method for when there's a brown coloured fan on the horizon 

Starting with Junos OS Release 17.3R1 it looks like Junper now has a way to seperate the management traffic 

https://www.juniper.net/documentation/en_US/junos/topics/topic-map/management-interface-in-non-default-instance.html

Not with EX unfortunately. Just MX and some SRX.

Yes it is supported only inMX/PTX/QFX/SRX boxes

Management Ethernet interface (fxp0) is confined in a non-default virtual routing and forwarding table Junos OS can confine the management interface in a dedicated management instance by setting the CLI configuration statement management-instance at the [edit system] hierarchy level. By doing so, operators can ensure that management traffic no longer has to share a routing table (that is, default.inet.0 table) with other control or protocol traffic in the system. Instead, there is a mgmt_junos routing instance introduced for management traffic. This feature is supported on following products/applications: Product Introduced Release Prerequisites MX Series PTX Series QFX Series SRX Series