CN2

 View Only
last person joined: one month ago 

Ask questions and share experiences with Juniper’s Cloud-Native Contrail Networking (CN2).
Back to discussions
Expand all | Collapse all

CN2-Apstra Integration: A Kubernetes CNI For Enabling Communication Between Underlay-Based Pods, Overlay-Based Pods and Bare Metal Servers

  • 1.  CN2-Apstra Integration: A Kubernetes CNI For Enabling Communication Between Underlay-Based Pods, Overlay-Based Pods and Bare Metal Servers

    Posted 03-06-2023 10:22
    Edited by Juniper Community Admin 03-07-2023 10:04

    In the previous blog in this series, I examined enabling communication between SRIOV-based pods. But the reality is that data centers (especially ones needed for 5G deployment) have a mixture of workloads/pods with different network connectivity:

    ·      Pods that have interfaces attached to a Virtual-Function(VF) of a SRIOV-enabled NIC which provides connectivity directly via the fabric underlay. We call these SRIOV pods. An example of such a pod is the User-Plane Function (UPF) in 5G where SRIOV can essentially take performance to line rate. While there are advantages, there are certain disadvantages as well like the need for fabric provisioning and a limited number of VFs in an SRIOV-enabled NIC

    ·    Pods that have interfaces attached to virtualized router in the compute node (eg: CN2 vRouter) which provide connectivity via overlays (over a fabric underlay). We call these non-SRIOV pods

    ·    Bare Metal Servers (BMS) that are not part of the Kubernetes clusters. The customer could be running applications directly on the native OS or the customer might have created VMs (eg: on ESXi server) or containers for running applications

    Irrespective of the network connectivity to the workloads, we need to enable communication across such a heterogeneous mix. Also, there are use cases where the heterogeneous workloads can belong to the same Virtual-Network/Subnet or to a different Virtual-Network/Subnet. For example, the UPF can be an SRIOV pod, the Intermediate-UPF (I-UPF) and Gi-LAN services can be non-SRIOV pods all belonging to the same Virtual-Network/Subnet.

    Given the above complex networking requirements especially with 5G deployments gaining traction, there is a need to address these requirements in order to simplify things for the customer. That is why we have provided user-friendly Kubernetes constructs and a CNI that hides all the above complexities and manual fabric provisioning. As part of CN2  release 23.1, Juniper now offers such Kubernetes-based constructs to allow Intra-VN and Inter-VN communication across a heterogeneous mix of SRIOV pods, non-SRIOV pods and BMS workloads.



    Intra-VN Communication Between SRIOV Pods, non-SRIOV Pods and BMS


    All steps below are done in Kubernetes unless specified.
    One-time setup steps:

    ·       Change CN2's encapsulation priority to vxlan in global-vrouter-config

    ·       In Apstra, create a Remote EVPN Gateway referencing CN2 networking controllers

    ·     Create a CN2 BGPRouter, referencing the Remote EVPN Gateway from the previous step. It will exchange EVPN Type-2 (ie MAC-IP) routes needed for intra-VN communication

    Intra-VN provisioning steps:

    ·         Create a common CN2 VN and add Apstra plugin label to extend the VN to the fabric

    ·         Create SRIOV NAD and reference the common VN

    ·         Create SRIOV pods and attach to the SRIOV NAD

    ·         Create non-SRIOV NAD and reference the common VN

    ·         Create non-SRIOV pods and attach to the non-SRIOV NAD

    ·         In Apstra, attach fabric ports (connecting to BMS) to the VN created in Apstra via CN2

    Inter-VN Communication Between SRIOV Pods, non-SRIOV Pods and BMS



    All steps below are done in Kubernetes unless specified.

    One-time setup steps:

    ·         Change CN2's encapsulation priority to vxlan in global-vrouter-config

    ·         In Apstra, create a Remote EVPN Gateway referencing CN2 networking controllers

    ·         Create a CN2 BGPRouter, referencing the Remote EVPN Gateway from the previous step. It will exchange EVPN Type-5 (ie IP-Prefix) routes needed for inter-VN communication

    Inter-VN provisioning steps:

    ·     Create a CN2 VNR (with a vn selector label of your choosing) with routingType set to evpn to enable routing between all the VNs attached to this VNR (using a label selector)

    ·       Create SRIOV NAD (and add Apstra plugin label and VNR selector label to match)

    ·       Create SRIOV pods and attach to the SRIOV NAD

    ·       Create non-SRIOV NAD (and add VNR selector label to match)

    ·       Create non-SRIOV pods and attach to the non-SRIOV NAD

    ·      Create reference NAD (and add VNR selector label to match) in CN2 with ssor label (Sytem Source Of Record) equal to Asptra to alias the BMS VN in Apstra into CN2

    Sample Topology to Validate Intra/Inter-VN Communication Between SRIOV Pods, non-SRIOV Pods and BMS



    ------------------------------
    Pradeep H Krishnamurthy
    ------------------------------



  • 2.  RE: CN2-Apstra Integration: A Kubernetes CNI For Enabling Communication Between Underlay-Based Pods, Overlay-Based Pods and Bare Metal Servers

     
    Posted 03-08-2023 10:56

    Thanks for writing this article, Pradeep. It's a perfect complement to our annoucement blog that is coming out soon. I'll drop a link to that when available. In the meantime, I think people would appreciate the demo video series. The first of six videos is just an intro, but the others are based on short demos. Credit to Pradeep for those too.



    ------------------------------
    James Kelly
    Sr. Dir PLM, Cloud-Ready Data Center business
    ------------------------------

    Original Message