Reliable communications between Branch offices and Data Centre or Head Quarters

One of the most popular requests for network engineering people is about reliable communications between Branch offices and Data Centre(DC) or Head Quarters(HQ). I’ve seen lots from small and medium businesses and even more from big enterprises. In most cases Hub and Spoke topology is used, with Hub being located in HQ or DC and Spokes – numerous branch offices or shops. Usually each branch has a dedicated WAN connection through one of the major Service Providers(SP), it could be VPLS, or L2VPN or something else. Even though SPs have SLAs for a link recovery, it does not help much when fault has actually happened, – what could mean for a shop owner a four hours outage during X-mas stocksale? There is no built-in redundancy and no reliability at all, there could be a fault with a branch router rendering shop offline, or there could be a fault on ISP infrastructure breaking connectivity with HQ and many other unforeseen issues. Owners eager to protect their businesses from faults of any kind and in this article I try to show how it could be achieved with less effort and money.

One of the most popular solutions would be to utilize Internet connection with IPsec VPN tunnel over it as a fallback transport when WAN link is having issues or completely failed. Now with 4G communications all around it is becoming a very interesting case not only for a small shops but even for medium/big offices, because 4G can provide very high speeds, for example Telstra Australia offers 2-50Mbit/s speeds, former 3G technology speeds are 10 to 15 times slower with higher round trip times and jitter.

Sometimes branch routers are managed by ISP as well and in this case customer has even less options on issue solving.

To overcome such situation we have to think about several areas of improvement:

First point we can address by using additional router as a redundant option, and very often big enterprises have one in place already, however for our purpose we need router supporting 4G Internet connection.

Second one could be addressed by usage of first hop redundancy protocol(FHRP) – VRRP, HSRP or GLBP, they allow backup router to stand-in almost instantly in case if active router fails. I will explain their operation and configuration later. Additional benefit is the ability to select active router basing on some conditions (link state, route presence and etc).

And last one could be achieved by setting up IPsec VPN tunnel from branch office to HQ once we have traffic to send to HQ on 4G-enabled router. In short WAN router has to track WAN interface status and in case if it fails,  router lowers its priority in FHRP protocol of choice giving ability for 4G-enabled router to take over. Once 4G router became active it is starting to receive traffic from clients towards HQ and other branches, so it starts up an IPsec tunnel over Internet to IPsec gateway which is usually a firewall or router in HQ and then transmits packets through that tunnel.

However there is a small caveat, this only works if WAN networks are dynamically learnt by WAN routers, if there is static routing in place it prevents return traffic from reaching failed branch office over IPsec tunnel. With dynamic routing it is not happening, as WAN routers have no information about Branch networks anymore and send traffic towards default route which is usually ends up at HQ or DC firewall/router where IPsec tunnel is terminated.

So let’s get to real setup and configuration, in my example I used unnamed WAN router(managed by ISP which agreed to add VRRP configuration), Cisco 897 Multi-mode VDSL2 ADSL2+ POTS with Multi-Mode 4G LTE Router as 4G Router and Cisco ASA5525 as HQ firewall for IPsec tunnel termination, dynamic routing protocol is RIP. We need dynamic routing protocol only enabled on WAN routers and firewall, so we don’t enable it on 4G Router. Below is a diagram with IP addressing, I also removed other branches, because they should be basically set up the same way(the only difference is in the subnet):

Below are configurations for ASA and 4G router, please note I list here only specific configuration related to the solution, leaving most of the basic stuff behind the scenes. My comments starts with hash sign #.

Configuration for ASA firewall:

hostname HQ-Firewall


domain-name mybusinessname.local


interface GigabitEthernet0/0

# We have to allow traffic in same security area, because we have traffic coming in from outside to inside and from outside to outside through the IPsec tunnel
same-security-traffic permit inter-interface
same-security-traffic permit intra-interface

# Define out Branch 1 office subnet
object network 10.1.0.0-Branch1
 subnet 10.1.0.0 255.255.255.0
 description Branch1 network


# Create access list allowing any traffic to Branch subnet
access-list Outside_cryptomap_1 extended permit ip any object 10.1.0.0-Branch1


# Create static NAT rule literally saying that we should not change source nor destination IP addresses for traffic destined Branch 1 office (NAT exemption), we also specified no-proxy-arp and route-lookup options, first option tells ASA to skip ARP request on local LAN as it is remote network, and route-lookup allows ASA to use routing table in order to find egress interface, this is useful in case if WAN connectivity is restored and Branch 1 subnet gets back into routing table, so ASA will then send traffic through available route instead of IPsec tunnel.


nat (Inside,Outside) source static any any destination static 10.1.0.0-Branch1 10.1.0.0-Branch1 no-proxy-arp route-lookup


# This rule is required if Branch 1 office needs centralized Internet access through the HQ ASA
nat (Outside,Outside) source dynamic 10.1.0.0-Branch1 interface


# Perform NAT for inside clients

# Setting up RIP instance for dynamic routing (could be another protocol as well)
router rip
 network 10.255.0.0
 network 0.0.0.0
 passive-interface default
 no passive-interface Inside
 default-information originate
 version 2
 no auto-summary


# default route to ISP
route Outside 0.0.0.0 0.0.0.0 1.1.1.2 1


# Create crypto maps to listen for incoming IPsec connections and to reference Branch 1 network to IPsec tunnel
crypto dynamic-map Branch1 1 match address Outside_cryptomap_1
crypto map Outside_map 1 ipsec-isakmp dynamic Branch1
crypto map Outside_map interface Outside


# Define ikev2 policy
crypto ikev2 policy 1
 encryption aes-256 
 integrity sha256 
 group 19 
 prf sha256
 lifetime seconds 86400


# Enable IKEv2 
crypto ikev2 enable Outside


# Define tunnel group Branch1 as site to site vpn (lan to lan) and PSKs for local/remote peers
tunnel-group Branch1 type ipsec-l2l
tunnel-group Branch1 ipsec-attributes
 ikev2 remote-authentication pre-shared-key <my-secret-password-word>
 ikev2 local-authentication pre-shared-key <my-secret-password-word>

Configuration for 4G router:

!
# Using ACL 121 specify which traffic should go into the tunnel

 vrrp 1 ip 10.1.0.254
# We have to adjust advertised MSS value in tcp packets because IPsec headers consume additional space, header length differs depending on used crypto algorithm and integrity checking method.. I used MSS vlaue of 1350 meaning that any single TCP packet cannot have more than 1350 bytes of useful data, it worked for me but i am sure this value could be tuned further to allow more useful data in a single packet. Cisco has handy tool for that: https://cway.cisco.com/tools/ipsec-overhead-calc/
 ip tcp adjust-mss 1350

Instead of VRRP  we can use HSRP, in this case configuration for Vlan1 should look like: