With the advent of 5G technology, telcos are gearing up for a massive surge in the data traffic in DataCenters. A good example is the UPF (User Plane Function) that offers a high-performance forwarding engine for user traffic and will typically be run as a Kubernetes Pod. At the same time, there has been a constant evolution of high-speed packet processing technologies to increase the I/O throughput of servers. One such technology is
SR-IOV (Single Root I/O Virtualization) which allows the Network Interface Card (NIC) to present itself as several virtual NICs or Virtual Functions (VF) which can transmit and receive packets directly. These VFs can be attached to different Pods thereby making them highly efficient in terms of packet processing.
The Kubernetes ecosystem too has kept up to leverage this new technology via the SR-IOV Network Device Plugin which discovers and advertises the networking resources in the form of SR-IOV virtual functions (VFs) to be used by the CNI (ie Container Networking Interface) component of Kubernetes.
While each of these address specific needs, there are certain gaps/shortcomings when you try to come up with a holistic solution by integrating these
- DataCenters have IP Fabrics deployed and typically leverage IP overlay technology. But to leverage the SR-IOV VF capability, there is a need to dynamically configure the underlay ie the fabric ports connecting to the servers (which have SRIOV-enabled NICs) as and when Pods (that use SR-IOV VFs) are spawned
- The SR-IOV based Virtual Network (to which Pods are attached) needs an IPAM to allocate IPs to the Pods. The IPAMs that are available online have their shortcomings. For example - consider using host-local when there are multiple Kubernetes worker nodes (which would typically be the case). The IPs that are allocated are local only to that worker node and in case of multiple worker nodes, we will end up with duplicate IPs being allocated to the Pods across different worker nodes
These gaps/shortcomings create a bad user-experience for customers who are also looking at automating the solution for operational efficiency.
This is where Juniper has taken a lead in addressing these shortcomings. As part of its CEFI (Customer Experience First Initiative) initiative, there is a new feature being introduced in the upcoming release of Juniper Cloud-Native Contrail Networking (CN2). CN2 is a SDN (Software-Defined Solution) which also serves as Kubernetes CNI to provide feature-rich networking capabilities. This feature leverages Apstra which is Juniper's intent-based networking software which automates and validates the design, deployment, and operation of data center networks.
Solution
Workflow
- Create NADs (Network Attachment Definition)
Specify annotation to extend the Virtual-Network to fabric
Specify a common label(if needed) for inter-VN routing
Specify CN2-IPAM to allocate Pod IP from a central IP Pool
- Create Pods and attach them to different Virtual Networks
- Create VNR to enable routing between VNs having common label
- CN2 Apstra Plugin listens for changes and provisions the fabric via Apstra SDK
Sample Topology (for Intra-VN Pod Communication)
------------------------------
Pradeep H Krishnamurthy
------------------------------