Secure VPN Tunneling Options

Secure VPN tunnels can be created using L2TP/IPsec(IKEv1), IKEv2/IPsec, and GRE/IPsec protocols.

L2TP/IPsec VPN

With an L2TP/IPsec VPN, a tunnel is created using L2TP (RFC 3931) protocol where network-layer packets are transmitted inside PPP frames. Since L2TP itself does not provide strict authentication, confidentiality, and integrity of the data being transmitted, the IPsec (RFC 6071) group of protocols is used for those purposes.

The L2TP tunnel is created inside a secure IPsec link, and to establish it, you first need to create a secure IPsec connection between the hosts. In this case, IPsec works as a transport and uses ESP (Encapsulating Security Payload) to encrypt L2TP packets.

VPN has two encapsulation levels: internal L2TP encapsulation and external IPsec encapsulation. The internal level has L2TP and UDP headers in addition to the PPP frame. The external level adds an IPsec ESP header and trailer. An IPsec Auth trailer provides message integrity check and authentication.

The process of creating a VPN comprises these main steps:

1. Establish a secure data link

2. Establish a VPN tunnel.

Secure data link establishment

To establish a secure data link, the IPsec group of protocols is used.

IPsec has three underlying protocols:

• Authentication Header (AH)

• Encapsulating security payload (ESP)

• Internet Key Exchange (IKE).

Authentication Header (AH) ensures the integrity of the transmitted data, authentication of the data source, and protection from retransmission. AH does not provide confidentiality for the data in transit because it does not perform encryption. The IP protocol number for AH is 51.

Encapsulating security payload (ESP) uses encryption to ensure the confidentiality of the transmitted data and also supports data integrity and data source authentication. The IP protocol number for ESP is 50.

Internet Key Exchange (IKE) is a service data exchange protocol for negotiating an establishing a security association (SA). A security association includes a set of secure connection parameters that can be used by both sides of the connection for mutual authentication and the encryption of transmitted data. IKE uses the UDP port 500.

The IPsec has two modes of operation:

• Tunnel mode

• Transport mode.

In the tunnel mode, IPsec encrypts the entire original IP packet along with its header. Then it is encapsulated inside an additional packet that has its own header. The tunnel mode is used when two private networks transmit data via an insecure public network.

In the transport mode, only the payload of the IP packet is encrypted, and the original IP header is kept with some extra information added to it. The transport mode is used when the two hosts already have an IP connection, but that connection does not provide security for the transmitted data.

When a VPN is created using L2TP/IPsec, the tunnel between the two hosts is created using the L2TP protocol, and IPsecis to provide security for the data link. In this case, IPsec operates in the transport mode, and the negotiation of the security association and establishment of the secure link is done using the IKE protocol (IKEv1) in two phases.

In phase 1, the neighboring hosts are mutually authenticated, an IKE SA is negotiated, and a secure service link is established between the hosts for IKE data exchange.

The negotiated parameters of an IKE SA are:

• Hash algorithms (MD5, SHA)

• Encryption algorithms (DES, 3DES, AES)

• Tunnel lifetime parameters

• Diffie-Hellman groups.

Pre-shared keys are used for authenticating the link peers.

IKEv1 phase 1 negotiation can be carried out in two modes:

• Main

• Aggressive

In the main mode, the devices exchange six messages. During the first exchange (messages 1 and 2), the encryption and authentication algorithms are negotiated for IKE SA. The second exchange (messages 3 and 4) implements the Diffie-Hellman (DH) key exchange. After the second exchange, the IKE service on each device creates a master key to use for authentication. The third exchange (messages 5 and 6) authenticates the reporter and responder of the connection (identity checking) and the information is secured using the encryption algorithm established earlier.

In the aggressive mode, there are two message exchanges with three messages in total. In the first message, the reporter transmits information corresponding to messages 1 and 3 of the main mode --- that is, the information on encryption and authentication algorithms as well as the DH key. The second message, transmitted by the responder, contains information corresponding to messages 2 and 4 of the main mode and also authenticates the responder. The third message authenticates the reporter and confirms the exchange.

The fewer number of messages allows for a quicker connection establishment in the aggressive mode, but the peer IDs are exchanged in plain text. The main mode is considered more secure because it encrypts the ID data.

The result of phase 1 is a successfully negotiated bidirectional IKE SA and the establishment of a secure service link. This channel will be used in phase 2 for negotiating the IPsec SA parameters for the main data channel.

In phase2, the secure service link established earlier in phase1 is used to negotiate an IPsec SA for the secure transmission of data over the IPsec link.

IKEv1 has one mode in phase 2 called quick mode.

The negotiated parameters of an IPsec SA are:

• Encryption algorithms (DES, 3DES, AES)

• Hash algorithms (MD5, SHA-1, SHA-2)

• SA lifetime parameters.

Based on the IPsec SA negotiation results, a secure data link is created operating in the IPsec transport mode with ESP encapsulation that has two unidirectional SAs to both sides of the link.

After this mode is set, the service link established in phase 1 does not disappear and is used to update the SA for the main channel.

The IPsec SA is terminated when the VPN is disconnected on one side of the connection or upon a timeout. A timeout occurs when the key lifetime has elapsed or the link lifesize has been reached. When the SA is terminated, the keys are deleted. If additional IPsec SAs are required for data transmission, a new IKE negotiation phase follows.

VPN tunnel establishment

At this stage, an L2TP tunnel is negotiated and established between the SA endpoints. The actual parameter negotiation occurs over a secure IPsec SA link. The L2TP protocol uses UDP port 1701.

When the VPN has been established, L2TP packets flowing between the endpoints are encapsulated using IPsec. Since the L2TP packet itself is wrapped into an IPsec packet, the original source and destination IP addresses are encrypted inside the outer packet. In addition, there is no need to open UDP port 1701 on the firewall between the endpoints because the inner packets are not processed until the IPsec data have been decrypted, which only happens at the tunnel endpoints.

IKEv2/IPsec VPN

When a VPN is created using IKEv2/IPsec, a secure VPN tunnel is established only using the IPsec group of protocols together with IKEv2(RFC 7296, RFC 7427).

In this scenario, IPsec operates in the tunnel mode where the original IP packets are fully encapsulated and encrypted inside a new packet that has its own header and trailers.

Similar to IKEv1, IKEv2 also has a two-stage secure connection establishment process, but with fewer message exchanges.

The first stage is known as IKE_SA_INIT. In the two messages comprising an IKE_SA_INIT exchange between two neighboring hosts, IKE SA parameters are negotiated, and a secure service link is established. The negotiated parameters of an IKE SA are:

• Hash algorithms

• Encryption algorithms

• Diffie-Hellman key.

Every host generates a seed key (SKEYSEED) that is used afterwards to generate keys used in the IKE SA. All the subsequent IKE keys are generated using the SKEYSEED.

The second stage is called IKE_AUTH. At this stage, the neighboring hosts are authenticated. The two messages comprising an IKE_AUTH exchange are authenticated and encrypted in the context of the IKE SA created during the IKE_SA_INIT message exchange.

At the end of the second negotiation stage, a child security association (CHILD SA) is created under the IKE SA for secure data transmission. CHILD SA is an IKEv2 term that is similar to IPsec SA in IKEv1. IKEv2 uses UDP ports 500 and 4500 (IPsec NAT Traversal).

Additional CHILD SA can be created to establish a new tunnel. This message exchange is called CREATE_CHILD_SA and can be used to negotiate new Diffie-Hellman group values and encryption/hash algorithm combinations.

IPsec tunnel peer authentication can be done using certificates based on a Public Key Infrastructure (PKI) or using the Extensible Authentication Protocol (EAP).

GRE/IPsec VPN

GRE (RFC 2784) is a tunneling protocol that can encapsulate the packets of various types of protocols inside IP tunnels, creating a virtual point-to-point link over an IP network. GRE is used for managing the process of transmitting multiprotocol and multicast IP traffic between two or more sites that can only communicate using the IP protocol. Note that GRE does not provide confidentiality and integrity for the transmitted data. To that end, IPsec protocols are used together with GRE.

When GRE and IPsec are used together, two types of connection can be created: IPsec over GRE and GRE over IPsec.

In case of an IPsec over GRE connection, encrypted traffic is transmitted over an unencrypted GRE tunnel, meaning that GRE encapsulation follows IPsec encapsulation. A downside of IPsec over GRE is that multicast and broadcast packets are not supported.

GRE over IPsec connections allow you to combine the advantages of GRE (multicast and broadcast support) and IPsec (encrypted traffic transmission). A GRE over IPsec connection encapsulates the traffic packets in GRE and then transmits them over an encrypted link (IPsec encapsulation).

To configure GRE over IPSec, follow these steps: