Routing

IMPORTANT MODERATION NOTICE

This community is currently under full moderation, meaning  all posts will be reviewed before appearing in the community. Please expect a brief delay—there is no need to post multiple times. If your post is rejected, you'll receive an email outlining the reason(s). We've implemented full moderation to control spam. Thank you for your patience and participation.



Expand all | Collapse all

Dual ISP with Point to Point Connection

  • 1.  Dual ISP with Point to Point Connection

    Posted 06-01-2021 09:24
    Dear,

    We have two sites that are connected with glass fiber cable(rental from ISP) and we have site-site VPN over it for internal traffic. We would share the Internet at each site as a backup connection for another side. I created probes so when the link is down, it will failover to another site's connection. The Internet traffic will be outside of the VPN.  Please see the diagram and my config below.  The failover with probe works for the Internet route but the return traffic still preferring VPN since it is first in the routing table. How can send the  Internet(external) return traffic outside of VPN? Maybe you better advice for the below scenario?

    Thank you very much in advance.
    Cheers,
    Isac


    SITE1 FIREWALL
    set routing-options static route 0/0 qualified-next-hop 10.13.1.2 preference 10
    edit routing-instances
    set to-SITE2-ISP instance-type virtual-router
    set to-SITE2-ISP interface reth2.0
    set to-SITE2-ISP routing-options static route 0/0 next-hop 10.13.1.2
    top
    set routing-options rib-groups to-SITE2-ISP-RouteGr import-rib [inet.0 to-SIE2-ISP.inet.0]
    set routing-options interface-routes rib-group inet to-SITE2-ISP-RouteGr
    set routing-instances to-SITE2-ISP routing-options interface-routes rib-group inet to-SITE2-ISP-RouteGr

    #HEALTH CHECK FOR SITE1 INTERNET CONNECTION
    set services rpm probe INTERNET-probe test TEST-Route-google probe-type icmp-ping
    set services rpm probe INTERNET-probe test TEST-Route-google target address 8.8.8.8
    set services rpm probe INTERNET-probe test TEST-Route-google probe-count 3
    set services rpm probe INTERNET-probe test TEST-Route-google probe-interval 15
    set services rpm probe INTERNET-probe test TEST-Route-google test-interval 10
    set services rpm probe INTERNET-probe test TEST-Route-google thresholds successive-loss 3
    set services rpm probe INTERNET-probe test TEST-Route-google thresholds total-loss 3
    set services rpm probe INTERNET-probe test TEST-Route-google destination-interface reth1.0
    set services rpm probe INTERNET-probe test TEST-Route-google next-hop 1.1.1.1
    set services ip-monitoring policy Server-Tracking match rpm-probe INTERNET-probe
    set services ip-monitoring policy Server-Tracking then preferred-route route 0.0.0.0/0 next-hop 10.13.1.2

    SITE1 FIREWALL - NAT for Incoming traffic from SITE2
    set security nat source rule-set ZONEP2P-nat-INTERNET from zone ZONEP2P
    set security nat source rule-set ZONEP2P-nat-INTERNET to zone INTERNET
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match destination-address 0.0.0.0/0
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match destination-address-name internet-ipv4
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match application any
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat then source-nat interface

    SITE1 FIREWALL - Security Policy for Incoming traffic from SITE2
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET description "From:ZONEP2P:any To:INTERNET:any Application: any Policy:permit"
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET match source-address any
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET match destination-address any
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET match application any
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET then permit
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET then log session-init
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET then log session-close

    SITE1 FIREWALL - Allow LAN to access ZONEP2P for backup Internet
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P description "From:LAN:any To:ZONEP2P:any Application: any Policy:permit"
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P match source-address any
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P match destination-address any
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P match application any
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P then permit
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P then log session-init
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P then log session-close

    #SITE2 FIREWALL - Virtual Router
    set routing-options static route 0/0 qualified-next-hop 10.13.1.1 preference 10
    edit routing-instances
    set to-SITE1-ISP instance-type virtual-router
    set to-SITE1-ISP interface reth2.0
    set to-SITE1-ISP routing-options static route 0/0 next-hop 10.13.1.1

    set routing-options rib-groups to-SITE1-ISP-RouteGr import-rib [inet.0 to-SITE1-ISP.inet.0]
    set routing-options interface-routes rib-group inet to-SITE1-ISP-RouteGr
    set routing-instances to-SITE1-ISP routing-options interface-routes rib-group inet to-SITE1-ISP-RouteGr

    #HEALTH CHECK FOR SITE2 INTERNET CONNECTION
    set services rpm probe INTERNET-probe test TEST-Route-google probe-type icmp-ping
    set services rpm probe INTERNET-probe test TEST-Route-google target address 8.8.8.8
    set services rpm probe INTERNET-probe test TEST-Route-google probe-count 3
    set services rpm probe INTERNET-probe test TEST-Route-google probe-interval 15
    set services rpm probe INTERNET-probe test TEST-Route-google test-interval 10
    set services rpm probe INTERNET-probe test TEST-Route-google thresholds successive-loss 3
    set services rpm probe INTERNET-probe test TEST-Route-google thresholds total-loss 3
    set services rpm probe INTERNET-probe test TEST-Route-google destination-interface reth1.0
    set services rpm probe INTERNET-probe test TEST-Route-google next-hop 2.2.2.2 #Public IP SITE1 GW
    set services ip-monitoring policy Server-Tracking match rpm-probe INTERNET-probe
    set services ip-monitoring policy Server-Tracking then preferred-route route 0.0.0.0/0 next-hop 10.13.1.1

    SITE2 FIREWALL - NAT for Incoming traffic from SITE1

    set security nat source rule-set ZONEP2P-nat-INTERNET from zone ZONEP2P
    set security nat source rule-set ZONEP2P-nat-INTERNET to zone INTERNET
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match destination-address 0.0.0.0/0
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match destination-address-name internet-ipv4
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match application any
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat then source-nat interface

    SITE2 FIREWALL - Security Policy for Incoming traffic from SITE2

    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET description "From:ZONEP2P:any To:INTERNET:any Application: any Policy:permit"
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET match source-address any
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET match destination-address any
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET match application any
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET then permit
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET then log session-init
    set security policies from-zone ZONEP2P to-zone INTERNET policy ZONEP2P-INTERNET then log session-close

    SITE2 FIREWALL - Allow LAN to access ZONEP2P for backup Internet

    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P description "From:LAN:any To:ZONEP2P:any Application: any Policy:permit"
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P match source-address any
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P match destination-address any
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P match application any
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P then permit
    set security policies from-zone LAN to-zone ZONEP2P policy LAN-ZONEP2P then log session-close


  • 2.  RE: Dual ISP with Point to Point Connection

     
    Posted 06-02-2021 05:38
    Since the return traffic has the same address range as the tunnel it matches for that path.

    You could add a source nat rule for traffic that passes on reth2 interface between the sites.  Then the return traffic will match this for removing the nat and returning via the reth interface directly instead of the tunnel.

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 3.  RE: Dual ISP with Point to Point Connection

    Posted 06-03-2021 03:01
    Thank you so much Steve. Just for confirmation, should I keep the NAT on other side to the internet? 

    I will add the below lines on both end; 
    set security nat source rule-set LAN-nat-ZONEP2P from zone LAN
    set security nat source rule-set LAN-nat-ZONEP2P to zone ZONEP2P
    set security nat source rule-set LAN-nat-ZONEP2P rule LAN-nat match destination-address 0.0.0.0/0
    set security nat source rule-set LAN-nat-ZONEP2P rule LAN-nat match destination-address-name internet-ipv4
    set security nat source rule-set LAN-nat-ZONEP2P rule LAN-nat match application any
    set security nat source rule-set LAN-nat-ZONEP2P rule LAN-nat then source-nat interface

    I will keep below. Should I add something else or remove?
    set security nat source rule-set ZONEP2P-nat-INTERNET from zone ZONEP2P
    set security nat source rule-set ZONEP2P-nat-INTERNET to zone INTERNET
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match destination-address 0.0.0.0/0
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match destination-address-name internet-ipv4
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat match application any
    set security nat source rule-set ZONEP2P-nat-INTERNET rule ZONEP2P-nat then source-nat interface


  • 4.  RE: Dual ISP with Point to Point Connection

     
    Posted 06-03-2021 05:44
    I'm not sure about your naming conventions.

    Site2 would source nat traffic to interface going to the internet across the link to site1
    And the reverse as well.

    This will insure that the return route from each site is specifically the interface that links the two sites instead of the vpn tunnel.

    ------------------------------
    Steve Puluka BSEET - Juniper Ambassador
    IP Architect - DQE Communications Pittsburgh, PA (Metro Ethernet & ISP)
    http://puluka.com/home
    ------------------------------



  • 5.  RE: Dual ISP with Point to Point Connection

    Posted 06-03-2021 06:03
    Thanks again, Steve. I understand now. I placed the NAT on the other site (in red), I should have placed it in the source site directly (green line).




  • 6.  RE: Dual ISP with Point to Point Connection

    Posted 06-07-2021 10:47
    Hello Steve,

    I see that some people also complained that it didn't work with  instance-type virtual-router,  instead, they choose forwarding but they required to reboot their firewalls. Where am I doing it wrong?

    Would it be possible to share an example config for the above diagram?  I have only static routes, no virtual router etc. yet.

    Thanks.
    Isac



  • 7.  RE: Dual ISP with Point to Point Connection

    Posted 08-06-2021 05:26
    Hi Isac

    I don't quite understand your setup here but I think I get the idea. I have done similar setups quite a number of times where I have 2 ISP's on site and I have 2 IPSEC VPNs between sites using each ISP. (So one VPN across each ISP)

    For the default route I do what you did, RPM probes and ip-monitoring. 

    For routes between sites I setup eBGP peering between sites with the routes being advertised across the second VPN having a higher as-path (as-path prepending)
    This is configured on both sides so that routes exchanged in both directions across the second VPN have a higher as-path so are less preferred. (No static routes across the VPN) Traffic always goes across the primary VPN and when that goes down the primary BGP routes disappear so the secondary BGP routes are used.

    I don't have any virtual-routers or anything fancy like that. It works well.

    Thanks