Blog Viewer

L2VPN Validation on ACX7000

By Suneesh Babu posted 17 days ago

  

ACX7000 platform is tested with 8,000 Layer2 VPN Routing-instances for 99.9% line rate traffic.                                                                

Introduction

This is the third article in the ACX7k Metro Validation Series:

In this article, we validate the L2VPN feature scale support on the ACX7100-32C platform with 22.2R1 Junos-EVO image.

L2VPN is a BGP or LDP MPLS Enabled Solution to connect Customer Edge Devices (CEs) across a Provider Edge Network. RFC-6624 supports L2VPN using BGP for auto-discovery and Signalling. The PEs provide layer 2 bridged connectivity between CEs. The PEs of the network are connected via MPLS technology with the help of LDP, RSVP, SR-MPLS etc.

JUNOS-EVO supports L2VPN functionality with instance type “l2vpn”. 8,000 L2VPN Instances were tested on ACX7100-32C. The same scale is tested on ACX7100-48L as well as on ACX7509. The ACX7024 Scale is not covered in this article (same features but different scale are expected on this platform).

Please note that despite sharing the same ACX moniker, the ACX7000 products are different products than ACX500/710/1000/1100/2100/2200/4000/5000/5400/6000. They are powered by different Packet Forwarding Engines (PFE), and support different feature sets and scales.

The Key Performance Indicator (KPI) tested are:

L2VPN KPI Scale
Number of L2VPN Instances 8,000
Number of Interfaces 8,000

Metro L2VPN Topology

Test topology consists of three routers, PE1, P1 and PE2, where ACX7100-32C is the PE1 and ACX7100-48L is the PE2 and P1 is that of PTX10008. PEs are connected to CEs using 4 x 100G links. CE’s are simulated using a traffic generator. PEs are connected to the core using 8 x 100G links, which are bundled in 2 x LAGs each having four links. 

The ACX7100-32C is the Device Under Test (DUT). The underlay transport used for testing is SR-MPLS with both OSPF and ISIS protocols individually. We also covered LDP as the underlay transport. The CE-bound interface configurations are of SP-Style (see glossary).

Figure 1: Lab topology

Test configures 8,000 L2VPN instances. Bidirectional traffic in iMIX mode at 99.9% offer-load flows for all the L2VPN services. A total of ~798 Gbps traffic flows through the DUT.

Base Configurations

regress@PE1> show configuration protocols |display inheritance no-comments
bgp {
    group IBGP {
        type internal;
        local-address 12.1.1.1;
        family inet {
            unicast;
        }
        family l2vpn {
            signaling;
        }
        neighbor 12.1.1.3 {
            export nhself;
        }
    }
}
mpls {
    interface ae0.0;
    interface ae1.0;
}
ospf {
    source-packet-routing {
        node-segment ipv4-index 101;
        srgb start-label 45000 index-range 2000;
    }
    area 0.0.0.0 {
        interface lo0.0 {
            passive;
        }
        interface ae0.0 {
            interface-type p2p;
        }                               
        interface ae1.0 {
            interface-type p2p;
        }
    }
}

regress@PE1>

Routing-Instance configuration required to bring up the L2VPN instances is as shown below.

interfaces {
    et-0/0/0 {
        flexible-vlan-tagging;
        encapsulation flexible-ethernet-services;
        unit 1 {
            encapsulation vlan-ccc;
            vlan-id 1;
        }
    }
}

routing-instances {
    METRO_L2VPN_VRF_1 {
        instance-type l2vpn;
        protocols {
            l2vpn {
                site s1 {
                    interface et-0/0/0.1 {
                        remote-site-id 2001;
                    }
                    site-identifier 1001;
                }
                encapsulation-type ethernet-vlan;
                no-control-word;
            }
        }
        interface et-0/0/0.1;
        route-distinguisher 12.1.1.1:1;
        vrf-target target:65000:1;
    }

}

CE Bound Interface Schema

Each of the PE devices is connected to the traffic generator using 4 x 100G interfaces, each interface is logically split into 2000 VLANs to have 8000 interfaces, one interface bound to one L2VPN instance.

PE1 PE2 Port Service Type VRF VLAN Association Devices Traffic Load
et-0/0/0 et-0/0/48:0 P1 L2VPN 1-2000 "1-2000" 4,000 99.9%
et-0/0/1 et-0/0/48:1 P2 L2VPN 2001-4000 "1-2000" 4,000 99.9%
et-0/0/2 et-0/0/48:2 P3 L2VPN 4001-6000 "1-2000" 4,000 99.9%
et-0/0/3 et-0/0/48:3 P4 L2VPN 6001-8000 "1-2000" 4,000 99.9%

L2VPN Verification

The first step to verify the L2VPN service is to make sure that the PE devices are having BGP established with l2vpn-signalling. As shown below from PE1, BGP is in Established state for Peer PE2(12.1.1.3).

regress@PE1> show bgp summary
Threading mode: BGP I/O
Default eBGP mode: advertise - accept, receive - accept
Groups: 1 Peers: 1 Down peers: 0
Table          Tot Paths  Act Paths Suppressed    History Damp State    Pending
inet.0     
                      35          1          0          0          0          0
bgp.l2vpn.0          
                    8000       8000          0          0          0          0
Peer                     AS      InPkt     OutPkt    OutQ   Flaps Last Up/Dwn State|#Active/Received/Accepted/Damped...
12.1.1.3              65000       8024       8023       0       0        7:05 Establ
  inet.0: 1/35/35/0
  bgp.l2vpn.0: 8000/8000/8000/0
  METRO_L2VPN_VRF_1.l2vpn.0: 1/1/1/0
  METRO_L2VPN_VRF_2.l2vpn.0: 1/1/1/0
  METRO_L2VPN_VRF_3.l2vpn.0: 1/1/1/0
  ...
  METRO_L2VPN_VRF_7998.l2vpn.0: 1/1/1/0
  METRO_L2VPN_VRF_7999.l2vpn.0: 1/1/1/0
  METRO_L2VPN_VRF_8000.l2vpn.0: 1/1/1/0

regress@PE1>

The BGP l2vpn table of the METRO_L2VPN_VRF_1 instance is as follows:

regress@PE1> show route table METRO_L2VPN_VRF_1.l2vpn.0

METRO_L2VPN_VRF_1.l2vpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden)
+ = Active Route, - = Last Active, * = Both

12.1.1.1:1:1001:2001/96                
                   *[L2VPN/170/-101] 00:09:40, metric2 1
                       Indirect
12.1.1.3:1:2001:1001/96                
                   *[BGP/170] 00:08:27, localpref 100, from 12.1.1.3
                      AS path: I, validation-state: unverified
                       to 15.1.1.2 via ae0.0, Push 45103
                    >  to 15.1.2.2 via ae1.0, Push 45103

regress@PE1> 

The L2VPN connections is UP towards the 12.1.1.3 Peer for the routing-instance METRO_L2VPN_VRF_1

regress@PE1> show l2vpn connections instance METRO_L2VPN_VRF_1
Layer-2 VPN connections:

Legend for connection status (St)  
EI -- encapsulation invalid      NC -- interface encapsulation not CCC/TCC/VPLS
EM -- encapsulation mismatch     WE -- interface and instance encaps not same
VC-Dn -- Virtual circuit down    NP -- interface hardware not present
CM -- control-word mismatch      -> -- only outbound connection is up
CN -- circuit not provisioned    <- -- only inbound connection is up
OR -- out of range               Up -- operational
OL -- no outgoing label          Dn -- down
LD -- local site signaled down   CF -- call admission control failure
RD -- remote site signaled down  SC -- local and remote site ID collision
LN -- local site not designated  LM -- local site ID not minimum designated
RN -- remote site not designated RM -- remote site ID not minimum designated
XX -- unknown connection status  IL -- no incoming label
MM -- MTU mismatch               MI -- Mesh-Group ID not available
BK -- Backup connection          ST -- Standby connection
PF -- Profile parse failure      PB -- Profile busy
RS -- remote site standby  SN -- Static Neighbor
LB -- Local site not best-site   RB -- Remote site not best-site
VM -- VLAN ID mismatch           HS -- Hot-standby Connection

Legend for interface status
Up -- operational           
Dn -- down

Instance: METRO_L2VPN_VRF_1
Edge protection: Not-Primary
  Local site: s1 (1001)
    connection-site           Type  St     Time last up          # Up trans
    2001                      rmt   Up     Oct 23 23:43:12 2022           1
      Remote PE: 12.1.1.3, Negotiated control-word: No
      Incoming label: 16, Outgoing label: 16
      Local interface: et-0/0/0.1, Status: Up, Encapsulation: VLAN
      Flow Label Transmit: No, Flow Label Receive: No

regress@PE1>



The following output captures the number of instances in the DUT

regress@PE1> show l2vpn connections |match "rmt   Up" | count
Count: 8000 lines

regress@PE1> show route table bgp.l2vpn.0 |match BGP | count
Count: 8001 lines

regress@PE1> show route summary |match L2VPN: |count
Count: 16001 lines

regress@PE1> show route table mpls.0 protocol l2vpn |match L2VPN|count
Count: 16000 lines

Traffic Verification at Router at PE1

Traffic got load-balanced across the lag bundles ae0 and ae1 towards the core and across each of the member interfaces.

PE1                               Seconds: 8                   Time: 03:23:34

Interface    Link  Input packets        (pps)     Output packets        (pps)

 ae0           Up  3125232690044   (31603996)    3092758811605   (31275639)
 ae1           Up  3060284770339   (30951593)    3092758802956   (31279892)
 esi           Up              0          (0)                0          (0)
 et-0/0/0      Up  1546379263196   (15637224)    1546379158447   (15636310)
 et-0/0/1      Up  1546379214713   (15637071)    1546379163592   (15636440)
 et-0/0/2      Up  1546379149169   (15636320)    1546379164104   (15636866)
 et-0/0/3      Up  1546379157074   (15636913)    1546379157376   (15636834)
 et-0/0/4      Up              0          (0)             3568          (0)
 et-0/0/5    Down              0          (0)                0          (0)
 et-0/0/6      Up   796385380100    (8053366)     770096960274    (7787501)
 et-0/0/7      Up   777055626584    (7857916)     776282456360    (7850117)
 et-0/0/8      Up   770096919918    (7787568)     770096939235    (7787592)
 et-0/0/9      Up   781694763442    (7905146)     776282455736    (7850429)
 et-0/0/10     Up   770096934324    (7788487)     776282470328    (7850981)
 et-0/0/11     Up   763138220225    (7718340)     770096938714    (7788677)
 et-0/0/12     Up   761591833934    (7702754)     776282455310    (7851325)
 et-0/0/13     Up   765457781856    (7742012)     770096938604    (7788909)

Generating Scale Configuration

The Python Script is having three components, a Jinja File capturing the various configuration templates, a Params File capturing the various user variables and the script uses both these input files and generates the required configurations and commit that in the router.

Note: The configs are generated as per the CE-bound interface schema.

Jinja Configuration Template

The template contains configurations for both interfaces which are part of the routing instances and routing instance configurations.

# Routing Instance Configuration
# File Name: l2vpn_jinja.j2
# Version: 1.0
groups {{group_name}} {
    interfaces {
        {{ifd_name}} {
            flexible-vlan-tagging;
            encapsulation flexible-ethernet-services;
            {%- for unit_id, vlan_id in ifl_variables %}
            unit {{unit_id}} {
                vlan-id {{vlan_id}};
                encapsulation vlan-ccc;
            }
            {%- endfor %}
        }
    }
    routing-instances {
        {%- for vrf_id, ifl_name in l2vpn_variables %}
        METRO_L2VPN_VRF_{{vrf_id}} {
            instance-type l2vpn;
            interface {{ifl_name}};
            route-distinguisher {{router_id}}:{{vrf_id}};
            vrf-target target:{{local_as_no}}:{{vrf_id}};
            protocols {
                l2vpn {
                    site s1 {
                        interface {{ifl_name}} {
                            remote-site-id {{remote_site_id}};
                        }
                        site-identifier {{local_site_id}};
                    }
                    encapsulation-type ethernet-vlan;
                    no-control-word;
                }
            }
        }
        {%- endfor %}
    }
}

Configuration Parameters

---
# This file contains the parameters for l2vpn configuration building
# File Name: l2vpn_params.yml
# Version: 1.0

#Host Name
host: 'PE1.englab.juniper.net'

#UserName
username: 'regress'
password: 'xxxxxx'

#Config Group Name
group_name: 'METRO_L2VPN'

#vrfname creation
vrf_id: 1

#maximum l2vpn instances
vrf_max: 10

#interface name
ifd_name: 'et-0/0/3'

#ifl_start unit
ifl_start_unit: 1

#vlan_id start
vlan_id_start: 1

#My AS no
local_as_no: 65000

#My Router ID
router_id: '12.1.1.1'

#Site Identifiers
local_site_id: 1001 #At PE1
remote_site_id: 2001 #At PE1

# local_site_id: 2001 #At PE2
# remote_site_id: 1001 #At PE2

Configuration Script

#! /usr/bin/python
"""
    FileName: create_l2vpn_vrf.py
    Version: 1.0
    Description: This script will create l2vpn vrf instances and configure it on the router
    Author: Suneesh Babu
"""
import yaml
from glob import glob
from jinja2 import Template
import ipaddress
from jnpr.junos.utils.config import Config
from jnpr.junos import Device
from jnpr.junos.factory import loadyaml
from jnpr.junos.op import *

def iflrange(ifd_name, start_unit, max):
    """
    This subroutine yields the specified number of l3 ifls for the ifd
    """
    while(max):
        iflname = ifd_name + '.' + str(start_unit)
        yield iflname
        start_unit += 1
        max -= 1

def router_operation(config_filename, data, mode):
    """
        This sub-routine connectes to the router and do the necessary commit operations
    """
    router = Device(host=data['host'], user=data['username'], password=data['password'], port=22)
    router.open()

    cfg = Config(router)
    if mode == 'jinja':
        cfg.load(template_path=config_filename, template_vars=data, format='text', merge=True)
    elif mode == 'setfile':
        cfg.load(path=config_filename, format='set')
    cfg.pdiff()
    cfg.commit()
    router.close()

def jinja_template_input(data):
    """
        This sub-routine yields the variables required for the jinja template
    """
    ifl_unit_list = range(int(data['ifl_start_unit']), int(data['ifl_start_unit']) + int(data['vrf_max']))
    vlan_list = range(int(data['vlan_id_start']), int(data['vlan_id_start']) + int(data['vrf_max']))

    vrf_id_list = range(int(data['vrf_id']), int(data['vrf_id']) + int(data['vrf_max']))
    ifl_name_list = iflrange(data['ifd_name'], data['ifl_start_unit'], int(data['vrf_max']))

   
return zip(ifl_unit_list, vlan_list), zip(vrf_id_list, list(ifl_name_list))

def main():
    """
       To Build the desired number of l2vpn routing-instances
    """
    print("Step-1: Read the Variables from the Params File")
    with open(glob('l2vpn_params.yaml')[0]) as fh:
        data = yaml.safe_load(fh.read())

    print("Step-2: Build the Data Feed for the Jinja Template Input")
    ifl_attributes, l2vpn_attributes = jinja_template_input(data)
    data['ifl_variables'] = ifl_attributes
    data['l2vpn_variables'] = l2vpn_attributes

    print("Step-3: Build the configuration file from Jinja Template")
    with open(glob('l2vpn_jinja.j2')[0]) as t_fh:
        t_format = t_fh.read()
    l2vpn_snippet = Template(t_format)
    print (l2vpn_snippet.render(data))

  print("Step-4: Load the config into router and commit it")
    # router_operation('l2vpn_jinja.j2', data, 'jinja')

if __name__ == '__main__':
    main()

Traffic Generator Configuration

CE Devices Configuration are as follows:

CPU Usage during traffic flow

The 12 Core ACX7100-32C remains idle at 95% during the traffic flow

Data Plane Traffic Verification

As shown below traffic flows at 99.9% of 100G links over 4 CE bound interfaces and no packet drops are seen for a duration of 27 hours.

Conclusion

ACX7000 (ACX7100-32C, ACX7100-48L, ACX7509) can scale to 8000 L2VPN instances and interfaces making it a perfectly suitable platform for Metro aggregation. Next article will be dedicated to VPLS validation, stay tuned :)

References

Glossary

  • BGP – Border Gateway Protocol
  • CE – Customer Edge Node
  • DUT – Device Under Test
  • KPI – Key Performance Indicators
  • L2VPN – Layer 2 VPN
  • LAN – Local Area Network
  • MPLS – Multi Protocol Label Switching
  • OSPF – Open Shortest Path First
  • P/PE – Provider/Provider Edge Node
  • SP Style – Service Provider Style
  • SR-MPLS – Segment Routing - MPLS
  • VLAN – Virtual LAN
  • VRF – Virtual Routing and Forwarding

Acknowledgement

Many thanks to Ramdas Machat, Deepak Kumar Tripathi, Vasily Mukhin and Nicolas Fevrier for reviewing the article and providing the feedback

Feedback

Revision History

Version Author(s) Date Comments
1 Suneesh Babu November 2022 Initial publication


#Validation


#ACXSeries

Permalink