Data Center

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.



Expand all | Collapse all

DC into MPLS Core with L3 VPN

  • 1.  DC into MPLS Core with L3 VPN

     
    Posted 04-01-2021 06:45
      |   view attached
    Good Day,

    I need some design guidance. Currently the client is using legacy deployment of L3VPN services separation on the MPLS edge with VRRP running inside the VRF with L2 all the way to the servers through the data center switches.Pretty easy and straight forward. It is a pure L3 network all the way through from one end to the next. Please see drawing for detail. We are now modernizing the core and data center with the idea to use VXLAN-EVPN at the DC and therefore looking at some design tips. We have QFX5200 as spine so i wondering what would be the best way to connect the DC tenants to the MX  LER into the existing VRF's. Should i enable routing on the spine for each service with anycast GW to remove the need for VRRP and use EBGP then between the spine service VRF and the MX l3VPN to exchange routing information and keep redundancy in place, kinda service chaining or what would be the best recommend for this type of deployment. Please consider the need for the L3VPN in the MPLS edge needs to remains it is just tenants now on the new VXLAN VNI needs to tie into these VRF services.

    Thanks,


  • 2.  RE: DC into MPLS Core with L3 VPN

    Posted 04-02-2021 17:33

    hello 
    If there are existing VRFs running for IPVPN services between the MX's and the spines qfx5200 are connected to leaf qfx5120, the easiest way is to enable the EVPN-VXLAN type-5 capabilities inside the same IPVPN VRF (at the MX) to receive the type-5 prefixes that are originated at the leaf devices. 

    Please, can you clarify what type of leaf devices will be used and if you interested in deploying distributed (aka edge routed) the first-hop IP gateway for SERVICE A? 

    Many thanks, 

    Michal




  • 3.  RE: DC into MPLS Core with L3 VPN

     
    Posted 04-04-2021 16:26
    Good Day, thanks for your reply.  The spine will consist of 2x 5200 and leafs  4 x 5110.


  • 4.  RE: DC into MPLS Core with L3 VPN

    Posted 04-05-2021 18:24

    hello MFB
    In this case, the spines qfx5200 would be deployed in a lean spine role and only leaf qfx5110 and MX'es would be the NVE speaking EVPN-VXLAN RT-5 to each other.
    Your lean spines qfx5200 would connect to MXes  via eBGP underlay to deliver 

    ## Example lean (no vtep termination) spine qfx5200-32c configuration: 

    set protocols bgp group underlay type external
    set protocols bgp group underlay export underlay_export
    set protocols bgp group underlay local-as 65011
    set protocols bgp group underlay neighbor 192.168.1.1 peer-as 65013
    set protocols bgp group underlay neighbor 192.168.2.1 peer-as 65031
    set protocols bgp group overlay type internal
    set protocols bgp group overlay local-address 172.16.7.11
    set protocols bgp group overlay family evpn signaling
    set protocols bgp group overlay cluster 172.16.7.11
    set protocols bgp group overlay local-as 64512
    set protocols bgp group overlay neighbor 172.16.7.13
    set protocols bgp group overlay neighbor 172.16.7.31

    ## here's an example MX configuration where the same routing-instance is exposed to IPVPN as well as Type-5 domain: 

    set routing-instances EVPN-VXLAN-to-L3VPN instance-type vrf
    set routing-instances EVPN-VXLAN-to-L3VPN interface lo0.1
    set routing-instances EVPN-VXLAN-to-L3VPN route-distinguisher 100.0.0.100:1115
    set routing-instances EVPN-VXLAN-to-L3VPN vrf-import VRF-1-import
    set routing-instances EVPN-VXLAN-to-L3VPN vrf-export VRF-1-export
    set routing-instances EVPN-VXLAN-to-L3VPN vrf-table-label
    set routing-instances EVPN-VXLAN-to-L3VPN routing-options multipath
    set routing-instances EVPN-VXLAN-to-L3VPN protocols evpn ip-prefix-routes advertise direct-nexthop
    set routing-instances EVPN-VXLAN-to-L3VPN protocols evpn ip-prefix-routes encapsulation vxlan
    set routing-instances EVPN-VXLAN-to-L3VPN protocols evpn ip-prefix-routes vni 1115
    set routing-instances EVPN-VXLAN-to-L3VPN protocols evpn ip-prefix-routes export VRF-1-Type-5-export

    set policy-options policy-statement VRF-1-Type-5-export term dc-subnets from route-filter 40.40.115.0/24 exact
    set policy-options policy-statement VRF-1-Type-5-export term dc-subnets then accept
    set policy-options policy-statement VRF-1-Type-5-export term remote-subnets from route-filter 10.10.9.0/24 exact
    set policy-options policy-statement VRF-1-Type-5-export term remote-subnets from route-filter 30.30.37.0/24 exact
    set policy-options policy-statement VRF-1-Type-5-export term remote-subnets from route-filter 50.50.74.0/24 exact
    set policy-options policy-statement VRF-1-Type-5-export term remote-subnets then accept
    set policy-options policy-statement VRF-1-export term 1 then community add COM-VXLAN
    set policy-options policy-statement VRF-1-export term 1 then community add COM-L3VPN
    set policy-options policy-statement VRF-1-export term 1 then accept
    set policy-options policy-statement VRF-1-import term vxlan-dc from community COM-VXLAN
    set policy-options policy-statement VRF-1-import term vxlan-dc then accept
    set policy-options policy-statement VRF-1-import term mpls-core from community COM-L3VPN
    set policy-options policy-statement VRF-1-import term mpls-core then accept
    set policy-options policy-statement my_underlay_export term term1 from route-filter 100.0.0.100/32 exact
    set policy-options policy-statement my_underlay_export term term1 then accept
    set policy-options community COM-L3VPN members target:3000:3000
    set policy-options community COM-VXLAN members target:1115:1115

    set interfaces et-0/0/0 vlan-tagging
    set interfaces et-0/0/0 mtu 9216
    set interfaces et-0/0/0 unit 3176 vlan-id 3176
    set interfaces et-0/0/0 unit 3176 family inet address 20.16.100.100/24
    set interfaces et-0/0/0 unit 3176 family iso
    set interfaces et-0/0/0 unit 3176 family mpls
    set interfaces et-0/0/0 unit 3177 vlan-id 3177
    set interfaces et-0/0/0 unit 3177 family inet address 20.17.100.100/24
    set interfaces et-0/0/0 unit 3177 family iso
    set interfaces et-0/0/0 unit 3177 family mpls
    set interfaces et-0/0/1 unit 0 family inet address 20.115.115.100/24
    set interfaces xe-0/1/0 description to-Juniper-107-RR
    set interfaces xe-0/1/0 mtu 9216
    set interfaces xe-0/1/0 unit 0 family inet address 20.100.107.100/24
    set interfaces xe-0/1/0 unit 0 family iso
    set interfaces xe-0/1/0 unit 0 family mpls
    set interfaces xe-0/1/1 mtu 9216
    set interfaces xe-0/1/1 unit 0 family inet address 20.8.100.100/24
    set interfaces lo0 unit 0 family inet address 100.0.0.100/32 preferred
    set interfaces lo0 unit 0 family iso address 49.1000.0000.0100.00
    set interfaces lo0 unit 1 family inet address 100.0.1.100/32


    #######​ the leaf qfx5110 edge routed configuration ####


    set forwarding-options vxlan-routing next-hop 32768
    set forwarding-options vxlan-routing interface-num 8192
    set forwarding-options vxlan-routing overlay-ecmp
    set routing-options forwarding-table export LB
    set routing-options forwarding-table chained-composite-next-hop ingress evpn
    set routing-options router-id 172.16.7.8

    set routing-instances T5-VRF1 routing-options static route 10.10.100.8/32 discard
    set routing-instances T5-VRF1 routing-options multipath
    set routing-instances T5-VRF1 protocols evpn ip-prefix-routes advertise direct-nexthop
    set routing-instances T5-VRF1 protocols evpn ip-prefix-routes encapsulation vxlan
    set routing-instances T5-VRF1 protocols evpn ip-prefix-routes vni 1100
    set routing-instances T5-VRF1 protocols evpn ip-prefix-routes export my-t5-export
    set routing-instances T5-VRF1 instance-type vrf
    set routing-instances T5-VRF1 interface irb.101
    set routing-instances T5-VRF1 interface lo0.1
    set routing-instances T5-VRF1 route-distinguisher 172.16.7.8:100
    set routing-instances T5-VRF1 vrf-target target:1115:1115
    set routing-instances T5-VRF1 vrf-table-label

    set policy-options policy-statement my-t5-export term term1 from route-filter 172.16.100.8/32 exact
    set policy-options policy-statement my-t5-export term term1 from route-filter 10.10.100.8/32 exact
    set policy-options policy-statement my-t5-export term term1 then accept
    set policy-options policy-statement my-t5-export term term2 from route-filter 10.10.101.0/24 orlonger
    set policy-options policy-statement my-t5-export term term2 then accept

    ** first-hop IP anycast gateway for the server/tenant connected to leaf qfx5110
    set interfaces irb unit 101 family inet address 10.10.101.254/24
    set interfaces irb unit 101 mac 00:00:5e:00:53:aa
    set interfaces lo0 unit 1 family inet address 172.16.100.8/32

    ** EVI default-switch for leaf qfx5110 to leaf qfx5110 L2 communication ** 

    set switch-options vtep-source-interface lo0.0
    set switch-options route-distinguisher 172.16.7.8:1
    set switch-options vrf-target target:1:7779

    set protocols evpn vni-options vni 50101 vrf-target target:1:7779
    set protocols evpn encapsulation vxlan
    set protocols evpn multicast-mode ingress-replication
    set protocols evpn default-gateway no-gateway-community
    set protocols evpn extended-vni-list 50101

    set vlans v101 vlan-id 101
    set vlans v101 l3-interface irb.101
    set vlans v101 vxlan vni 50101


    ​** leaf qfx5110 pe-ce port connected to a server/tenant  

    set interfaces ae0 mtu 9100
    set interfaces ae0 esi 00:02:02:02:02:02:02:02:00:02
    set interfaces ae0 esi all-active
    set interfaces ae0 aggregated-ether-options lacp active
    set interfaces ae0 aggregated-ether-options lacp system-id 00:00:00:02:00:02
    set interfaces ae0 unit 0 family ethernet-switching interface-mode trunk
    set interfaces ae0 unit 0 family ethernet-switching vlan members 101

    set interfaces et-0/0/4 ether-options 802.3ad ae0

    set chassis aggregated-devices ethernet device-count 18

    HTH,
    Michal




  • 5.  RE: DC into MPLS Core with L3 VPN

     
    Posted 04-06-2021 07:43
    Thanks for the feedback will review and revert back.


  • 6.  RE: DC into MPLS Core with L3 VPN

     
    Posted 04-06-2021 08:23
    Thanks for the information provided. What options will i have if the L3 Gateway needs exist on the MX and be redundant however the fabric is not extended to the MX?


  • 7.  RE: DC into MPLS Core with L3 VPN

    Posted 04-06-2021 10:01
    hello ,

    In case you'd prefer conserving your existing L3 gateways and outside of the fabric, we can consider enabling the "bridged overlay" EVPN-VxLAN architecture with an ESI-LAG delivering an L2 handoff from selected leaf nodes. 

    However,  you can also enable the ESI-LAG from the qfx5200 spines and also perform the L2 handoff to the MX'es which will continue running VRRP in this case. Make sure in this case the back-to-back link between the MX'es from your diagram is a pure L3 link and is not having any L2 trunking enabled, to ensure we don't create any backdoor ethernet loop.  

    Regarding the bridged-overlay EVPN-VXLAN here's the example document: 

    https://www.juniper.net/documentation/en_US/release-independent/solutions/topics/task/configuration/bridged-overlay-cloud-dc-configuring.html

    Thanks 
    Michal



  • 8.  RE: DC into MPLS Core with L3 VPN

    Posted 04-02-2021 17:35
    Hi,

    Depending on where EVPN (VTEP) is terminated  there are a number of options.
    Firstly - VRRP is the First Hop Redundancy Protocol and requires L2 to work, in routed networks VRRP isn't necessary, common way to provide resiliency towards external networks is to inject external prefixes (or default route). Server resiliency is achieved by bu dual-homing the server to 2 leafs and using either ESI-LAG or routed ECMP.

    If hand-off is IP and per VRF (similar to Inter-AS option A) - external routers would connect to leafs and have unicast (inet.0/inet6)BGP session per VRF (VLAN/IRB) associated with it
    If hand-off is VXLAN and VXLAN is terminated outside of the fabric (e.g. on MX LER), external routers could be physically connected to either leafs or spines and exchange:
    EVPN routes, so MX cant either terminate them or perform interworking - EVPN VXLAN to either EVPN MPLS (if L2 stretching is needed) or L3VPN (routing only)
    unicast BGP to exchange loopback/VTEP range so BGP session could be establish/BGP next-hop could be resolved. This is similar to Inter-AS option C.

    Please let me know if you need more details.
    Jeff

    ------------------------------
    Jeff Tantsura
    ------------------------------