View Only
last person joined: 6 days ago 

Ask questions and share experiences with Juniper’s Cloud-Native Contrail Networking (CN2).

CN2-Apstra Integration: A Kubernetes CNI For End-To-End Provisioning Of SRIOV Virtual Networks With DataCenter Fabric Underlay

  • 1.  CN2-Apstra Integration: A Kubernetes CNI For End-To-End Provisioning Of SRIOV Virtual Networks With DataCenter Fabric Underlay

    Posted 10-20-2022 18:44
    Edited by Michael Pappas 10-21-2022 11:52
    With the advent of 5G technology, telcos are gearing up for a massive surge in the data traffic in DataCenters. A good example is the UPF (User Plane Function) that offers a high-performance forwarding engine for user traffic and will typically be run as a Kubernetes Pod. At the same time, there has been a constant evolution of high-speed packet processing technologies to increase the I/O throughput of servers. One such technology is SR-IOV (Single Root I/O Virtualization) which allows the Network Interface Card (NIC) to present itself as several virtual NICs or Virtual Functions (VF) which can transmit and receive packets directly. These VFs can be attached to different Pods thereby making them highly efficient in terms of packet processing.

    The Kubernetes ecosystem too has kept up to leverage this new technology via the SR-IOV Network Device Plugin which discovers and advertises the networking resources in the form of SR-IOV virtual functions (VFs) to be used by the CNI (ie Container Networking Interface) component of Kubernetes.

    While each of these address specific needs, there are certain gaps/shortcomings when you try to come up with a holistic solution by integrating these
    • DataCenters have IP Fabrics deployed and typically leverage IP overlay technology. But to leverage the SR-IOV VF capability, there is a need to dynamically configure the underlay ie the fabric ports connecting to the servers (which have SRIOV-enabled NICs) as and when Pods (that use SR-IOV VFs) are spawned
    • The SR-IOV based Virtual Network (to which Pods are attached) needs an IPAM to allocate IPs to the Pods. The IPAMs that are available online have their shortcomings. For example - consider using host-local when there are multiple Kubernetes worker nodes (which would typically be the case). The IPs that are allocated are local only to that worker node and in case of multiple worker nodes, we will end up with duplicate IPs being allocated to the Pods across different worker nodes

    These gaps/shortcomings create a bad user-experience for customers who are also looking at automating the solution for operational efficiency.
    This is where Juniper has taken a lead in addressing these shortcomings. As part of its CEFI (Customer Experience First Initiative) initiative, there is a new feature being introduced in the upcoming release of Juniper Cloud-Native Contrail Networking (CN2). CN2 is a SDN (Software-Defined Solution) which also serves as Kubernetes CNI to provide feature-rich networking capabilities. This feature leverages Apstra which is Juniper's intent-based networking software which automates and validates the design, deployment, and operation of data center networks.

    Deployment depicting the components in CN2-Apstra integration

    As seen in the above deployment diagram, there is an Apstra Plugin that is installed in the CN2 workspace which listens for CN2/Kubernetes events related to user-facing intents like NAD/VN/VNR and then translates those into relevant underlay Fabric networking by leveraging Apstra


    • Create NADs (Network Attachment Definition)
      Specify annotation to extend the Virtual-Network to fabric
      Specify a common label(if needed) for inter-VN routing
      Specify CN2-IPAM to allocate Pod IP from a central IP Pool
    • Create Pods and attach them to different Virtual Networks
    • Create VNR to enable routing between VNs having common label
    • CN2 Apstra Plugin listens for changes and provisions the fabric via Apstra SDK

    Sample Topology (for  Intra-VN Pod Communication)
    Sample topology for communication between SRIOV Pods attached to same Virtual Network

    Pradeep H Krishnamurthy