Data Center

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

That suggests comms between the lead and vSRX is ok.

Do you have the pro version of eve. Can you confirm pings are moving between the host vm and vSRX?

You can also add an IRB to the vSRX on vlan101 using an available IP and trying pinging the leaf from there. That will help narrow your troubleshooting.

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Hi cdoyle ,

Unfortunately i'm just testing using EVE community and also PNETLAB. The result is same. I think no issue on vSRX because if i enable irb on vSRX it can ping to host. The issue from host can ping Anycast gewatway on LEAF. I will try again do from scratch again.

Thanks

That suggests comms between the lead and vSRX is ok.

Do you have the pro version of eve. Can you confirm pings are moving between the host vm and vSRX?

You can also add an IRB to the vSRX on vlan101 using an available IP and trying pinging the leaf from there. That will help narrow your troubleshooting.

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

At least we've narrowed the issue something between the vSRX and the leafs.

Keep the irb on the vsrx and troubleshoot from there.

• do you have the correct ARP entry for the anycast gateway on the leafs?
• do the leafs have the correct ARP entry for the IP address on the vSRX irb?

One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?

Under Staged -> DCI -> Settings, the MAC-MSB value must be different in DC-B than the value in DC-A. When using OTT DCI to stretch L2 between DC-A and DC-B, the ESI values must be unique in each data center. Because these values are generated and deterministic, if you do not change the ESI MAC MSB value, you will end up with overlapping/identical ESI values in both DC's.

This predictably breaks ESI-LAG and causes all sorts of difficult-to-troubleshoot problems.

I did this to myself at one point and only discovered the issue when I realized that a ping towards a remote host had a local next-hop.

To be clear, I do not think this is the issue as a properly populated ARP and MAC table on the leaf should ensure forwarding if the destination is local, but overlapping ESI values can cause very strange problems.

In short, it's worth verifying.

Good luck!

Hi cdoyle ,

Unfortunately i'm just testing using EVE community and also PNETLAB. The result is same. I think no issue on vSRX because if i enable irb on vSRX it can ping to host. The issue from host can ping Anycast gewatway on LEAF. I will try again do from scratch again.

Thanks

That suggests comms between the lead and vSRX is ok.

Do you have the pro version of eve. Can you confirm pings are moving between the host vm and vSRX?

You can also add an IRB to the vSRX on vlan101 using an available IP and trying pinging the leaf from there. That will help narrow your troubleshooting.

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Hi cdoyle ,

When u said "One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?" u referring to OTT right? Currently i'm not reach on OTT slide yet? If i isolate DC-B just ( i means power-off all on DC-B) it still same. Host on DC-A cannot ping anycast gateway on Leaf DC-A.

Thanks

At least we've narrowed the issue something between the vSRX and the leafs.

Keep the irb on the vsrx and troubleshoot from there.

• do you have the correct ARP entry for the anycast gateway on the leafs?
• do the leafs have the correct ARP entry for the IP address on the vSRX irb?

One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?

Under Staged -> DCI -> Settings, the MAC-MSB value must be different in DC-B than the value in DC-A. When using OTT DCI to stretch L2 between DC-A and DC-B, the ESI values must be unique in each data center. Because these values are generated and deterministic, if you do not change the ESI MAC MSB value, you will end up with overlapping/identical ESI values in both DC's.

This predictably breaks ESI-LAG and causes all sorts of difficult-to-troubleshoot problems.

I did this to myself at one point and only discovered the issue when I realized that a ping towards a remote host had a local next-hop.

To be clear, I do not think this is the issue as a properly populated ARP and MAC table on the leaf should ensure forwarding if the destination is local, but overlapping ESI values can cause very strange problems.

In short, it's worth verifying.

Good luck!

Hi cdoyle ,

Unfortunately i'm just testing using EVE community and also PNETLAB. The result is same. I think no issue on vSRX because if i enable irb on vSRX it can ping to host. The issue from host can ping Anycast gewatway on LEAF. I will try again do from scratch again.

Thanks

That suggests comms between the lead and vSRX is ok.

Do you have the pro version of eve. Can you confirm pings are moving between the host vm and vSRX?

You can also add an IRB to the vSRX on vlan101 using an available IP and trying pinging the leaf from there. That will help narrow your troubleshooting.

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Packet captures are the next step. Every time I've encountered something like this, it's been an issue with eve forwarding pings over a link. This could be the link acting buggy. It could be a memory issue if you're using more than 85% of your total eve server RAM. If UKSM is enabled, it could absolutely be causing this sort of issue.

Sorry I can't offer more guidance. Everything looks good from the node and fabric configs.

Hi cdoyle ,

When u said "One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?" u referring to OTT right? Currently i'm not reach on OTT slide yet? If i isolate DC-B just ( i means power-off all on DC-B) it still same. Host on DC-A cannot ping anycast gateway on Leaf DC-A.

Thanks

At least we've narrowed the issue something between the vSRX and the leafs.

Keep the irb on the vsrx and troubleshoot from there.

• do you have the correct ARP entry for the anycast gateway on the leafs?
• do the leafs have the correct ARP entry for the IP address on the vSRX irb?

One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?

Under Staged -> DCI -> Settings, the MAC-MSB value must be different in DC-B than the value in DC-A. When using OTT DCI to stretch L2 between DC-A and DC-B, the ESI values must be unique in each data center. Because these values are generated and deterministic, if you do not change the ESI MAC MSB value, you will end up with overlapping/identical ESI values in both DC's.

This predictably breaks ESI-LAG and causes all sorts of difficult-to-troubleshoot problems.

I did this to myself at one point and only discovered the issue when I realized that a ping towards a remote host had a local next-hop.

To be clear, I do not think this is the issue as a properly populated ARP and MAC table on the leaf should ensure forwarding if the destination is local, but overlapping ESI values can cause very strange problems.

In short, it's worth verifying.

Good luck!

Hi cdoyle ,

Unfortunately i'm just testing using EVE community and also PNETLAB. The result is same. I think no issue on vSRX because if i enable irb on vSRX it can ping to host. The issue from host can ping Anycast gewatway on LEAF. I will try again do from scratch again.

Thanks

That suggests comms between the lead and vSRX is ok.

Do you have the pro version of eve. Can you confirm pings are moving between the host vm and vSRX?

You can also add an IRB to the vSRX on vlan101 using an available IP and trying pinging the leaf from there. That will help narrow your troubleshooting.

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!

Hi @cdoyle ,

I'm try do from scratch back the result still same. Btw i see the issue just on ESI-LAG. For single connection no issue.

Thanks

Packet captures are the next step. Every time I've encountered something like this, it's been an issue with eve forwarding pings over a link. This could be the link acting buggy. It could be a memory issue if you're using more than 85% of your total eve server RAM. If UKSM is enabled, it could absolutely be causing this sort of issue.

Sorry I can't offer more guidance. Everything looks good from the node and fabric configs.

Hi cdoyle ,

When u said "One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?" u referring to OTT right? Currently i'm not reach on OTT slide yet? If i isolate DC-B just ( i means power-off all on DC-B) it still same. Host on DC-A cannot ping anycast gateway on Leaf DC-A.

Thanks

At least we've narrowed the issue something between the vSRX and the leafs.

Keep the irb on the vsrx and troubleshoot from there.

• do you have the correct ARP entry for the anycast gateway on the leafs?
• do the leafs have the correct ARP entry for the IP address on the vSRX irb?

One other thought... If you have a lab with a DC-A and a DC-B, did you change the Most Significant Bit (MSB) in DC-B?

Under Staged -> DCI -> Settings, the MAC-MSB value must be different in DC-B than the value in DC-A. When using OTT DCI to stretch L2 between DC-A and DC-B, the ESI values must be unique in each data center. Because these values are generated and deterministic, if you do not change the ESI MAC MSB value, you will end up with overlapping/identical ESI values in both DC's.

This predictably breaks ESI-LAG and causes all sorts of difficult-to-troubleshoot problems.

I did this to myself at one point and only discovered the issue when I realized that a ping towards a remote host had a local next-hop.

To be clear, I do not think this is the issue as a properly populated ARP and MAC table on the leaf should ensure forwarding if the destination is local, but overlapping ESI values can cause very strange problems.

In short, it's worth verifying.

Good luck!

Hi cdoyle ,

Unfortunately i'm just testing using EVE community and also PNETLAB. The result is same. I think no issue on vSRX because if i enable irb on vSRX it can ping to host. The issue from host can ping Anycast gewatway on LEAF. I will try again do from scratch again.

Thanks

That suggests comms between the lead and vSRX is ok.

Do you have the pro version of eve. Can you confirm pings are moving between the host vm and vSRX?

You can also add an IRB to the vSRX on vlan101 using an available IP and trying pinging the leaf from there. That will help narrow your troubleshooting.

Hi @cdoyle ,

Below is the output:

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces stat
                                                          ^
invalid interface type in 'stat' at 'stat'
root@server-rack-dc-a-001-leaf1# run show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/7               38986       40405            0            0

Aggregated interface: ae2
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/8               38123       40403            0            0

root@A-BMS-1_Bridge> show lacp statistics interfaces 
Aggregated interface: ae1
    LACP Statistics:       LACP Rx     LACP Tx   Unknown Rx   Illegal Rx 
      ge-0/0/1               36001       36529            0            0
      ge-0/0/2               36011       36529            0            0

[edit]
root@server-rack-dc-a-001-leaf1# run show lacp interfaces 
Aggregated interface: ae1
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/7       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/7     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/7                  Current   Fast periodic Collecting distributing

Aggregated interface: ae2
    LACP state:       Role   Exp   Def  Dist  Col  Syn  Aggr  Timeout  Activity
      ge-0/0/8       Actor    No    No   Yes  Yes  Yes   Yes     Fast    Active
      ge-0/0/8     Partner    No    No   Yes  Yes  Yes   Yes     Fast    Active
    LACP protocol:        Receive State  Transmit State          Mux State 
      ge-0/0/8                  Current   Fast periodic Collecting distributing

You don't need a vxlan license - I see the same messages on the switches in my lab.

What does "show lacp interface" look like?

Hi @cdoyle ,

No issue on LACP. May i know whether in your lab vswitch-junos required vxlan-license? All EBGP and IBGP is establish between Spine and Leaf.  I think i follow exactly your step.

[edit]
root@server-rack-dc-a-001-leaf1# run show version 
Hostname: server-rack-dc-a-001-leaf1
Model: ex9214
Junos: 23.2R1.14

[edit]
root@server-rack-dc-a-001-leaf1# run show interfaces descriptions 
Interface       Admin Link Description
ge-0/0/0        up    up   facing_spine1:ge-0/0/2
ge-0/0/1        up    up   facing_spine2:ge-0/0/2
ge-0/0/7        up    up   to.server-rack-dc-a-001-sys001
ge-0/0/8        up    up   to.server-rack-dc-a-001-sys002
ge-0/0/9        up    up   to.server-rack-dc-a-001-sys003
ae1             up    up   to.server-rack-dc-a-001-sys001
ae2             up    up   to.server-rack-dc-a-001-sys002
fxp0            up    up   OOB Connection

[edit]
root@server-rack-dc-a-001-leaf1# run show configuration routing-instances 
Tenant-1 {
    instance-type vrf;
    routing-options {
        graceful-restart;
        multipath;
        auto-export;
    }
    protocols {
        evpn {
            irb-symmetric-routing {
                vni 10010;
            }

           ip-prefix-routes {
                advertise direct-nexthop;
                encapsulation vxlan;
                vni 10010;
                export BGP-AOS-Policy-Tenant-1;
            }
        }
    }
    interface irb.101;
    interface irb.102;
    interface lo0.2;
    route-distinguisher 192.168.1.4:10; 
    vrf-target target:10010:1;

}
evpn-1 {
    instance-type mac-vrf;
    protocols {
        evpn {
            encapsulation vxlan;
            default-gateway do-not-advertise;
            duplicate-mac-detection {
                auto-recovery-time 9;
            }
            extended-vni-list all;
            vni-options {
                vni 10101 {
                    vrf-target target:10101:1;
                }
                vni 10102 {
                    vrf-target target:10102:1;
                }
            }
        }

   }
    vtep-source-interface lo0.0;        
    service-type vlan-aware;
    interface ge-0/0/9.0;
    interface ae1.0;
    interface ae2.0;
    route-distinguisher 192.168.1.4:65534;
    vrf-target target:100:100;
    vlans {
        vn101 {
            description vn101;
            vlan-id 101;
            l3-interface irb.101;
            ##
            ## Warning: requires 'vxlan' license
            ##
            vxlan {
                vni 10101;
            }

        }
        vn102 {
            description vn102;
            vlan-id 102;
            l3-interface irb.102;
            ##                          
            ## Warning: requires 'vxlan' license
            ##                          
            vxlan {                     
                vni 10102;              
            }                           
        }                               
    }                                   
}          

Thanks          

It's a fair point about the interface-mode, but no, it's not a typo. If you view the configuration on a server leaf switch (e.g. DCA -> Active tab -> click one of the server rack switches in the topology diagram -> click Config in the Telemetry menu on the right), you'll see that your aggregate interface (ae1 in my case) is configured as a trunk, but also has a native-vlan-id defined. The vSRX aggregate can be configured as an access interface in our lab because our connectivity template is configured to only pass a single VLAN.

If we were connecting to something like a vm server that had multiple guest vm's with different VLANs, we would absolutely need to configure a trunk. For our simple lab, not required.

To confirm connectivity, first verify that your vSRX bridge is connected via LACP correctly using "show lacp interfaces". Assuming it's "distributing", yes, your test node should be able to ping the anycast gateway on the leaf.

See attachments for screenshots.

C

Hi @cdoyle ,

I think in 6-slides u have typo error. vSRX should in trunk mode facinng to leaf because on LEAF it configure trunk. One more thing may i know is it supposedly in Slide 6 the host should can ping irb.101 in LEAF? I'm follow your step but weird from host cannot ping irb.101? Can u confirm that?

Thanks

#5minutejunos

Video: 6. Deploying Testing Hosts - Apstra 4.2.1 in eve-ng

Video hashtag: #4.2.1_eve_video-6

In this video, we'll build out our lab bridge nodes and testing hosts. If you've been passing on downloading the lab guides from the Elevate Community link above, this is the video where I strongly suggest you start. It's that, or trying to keep up while I type (and mis-type/delete/retype) configuration into 5x different vSRX nodes.

As mentioned in video 2, I HATE configuring LACP in Linux. To spare myself the pain and torment of bonded interfaces, I use vSRX nodes running a packet-mode for ESI-LAG/LACP connectivity between the fabric border leafs and the uMate hosts I'm using for testing. This video will take you through the complete configuration of these vSRX bridge nodes, as well as the straightforward testing host configurations.

At the end of the video, we'll confirm reachability, but not before I trip over my own feet and have to troubleshoot a connectivity problem.

If you are comfortable building your own testing hosts, you can probably skip this video since there's no fabric/Apstra configuration elements.

Thank you for watching!