Tag Archives: IPsec VPN concepts fortigate

IPsec VPN concepts

IPsec VPN concepts

Virtual Private Network (VPN) technology enables remote users to connect to private computer networks to gain access to their resources in a secure way. For example, an employee traveling or working from home can use a VPN to securely access the office network through the Internet.

Instead of remotely logging on to a private network using an unencrypted and unsecure Internet connection, the use of a VPN ensures that unauthorized parties cannot access the office network and cannot intercept any of the information that is exchanged between the employee and the office. It is also common to use a VPN to connect the private networks of two or more offices.

Fortinet offers VPN capabilities in the FortiGate Unified Threat Management (UTM) appliance and in the FortiClient Endpoint Security suite of applications. A FortiGate unit can be installed on a private network, and FortiClient software can be installed on the user’s computer. It is also possible to use a FortiGate unit to connect to the private network instead of using FortiClient software.

This chapter discusses VPN terms and concepts including: VPN tunnels

VPN gateways

Clients, servers, and peers



Phase 1 and Phase 2 settings

Security Association

IKE and IPsec packet processing


VPN tunnels

The data path between a user’s computer and a private network through a VPN is referred to as a tunnel. Like a physical tunnel, the data path is accessible only at both ends. In the telecommuting scenario, the tunnel runs between the FortiClient application on the user’s PC, or a FortiGate unit or other network device and the FortiGate unit on the office private network.

Encapsulation makes this possible. IPsec packets pass from one end of the tunnel to the other and contain data packets that are exchanged between the local user and the remote private network. Encryption of the data packets ensures that any third-party who intercepts the IPsec packets can not access the data.


Encoded data going through a VPN tunnel

You can create a VPN tunnel between:

  • A PC equipped with the FortiClient application and a FortiGate unit
  • Two FortiGate units
  • Third-party VPN software and a FortiGate unit

For more information on third-party VPN software, refer to the Fortinet Knowledge Base for more information.


Tunnel templates

Several tunnel templates are available in the IPsec VPN Wizard that cover a variety of different types of IPsec VPN. A list of these templates appear on the first page of the Wizard, located at VPN > IPsec Wizard. The tunnel template list follows.


IPsec VPN Wizard options

VPN Type                Remote Device Type               NAT Options                   Description

Site to Site                FortiGate                                    l No NAT between sites

  • This site is behind NAT
  • The remote site is behind NAT

Static tunnel between this FortiGate and a remote FortiGate.



  • No NAT between sites
  • This site is behind NAT
  • The remote site is behind NAT

Static tunnel between this FortiGate and a remote Cisco firewall.


VPN Type                Remote Device Type               NAT Options                   Description

Remote Access



























FortiClient VPN for OS X,         N/A                                     On-demand tunnel for

Windows, and Android                                                        users using the FortiCli- ent software.


iOS Native                                  N/A                                     On-demand tunnel for iPhone/iPad users using the native iOS IPsec cli- ent.


Android Native                          N/A                                     On-demand tunnel for Android users using the native L2TP/IPsec client.


Windows Native                        N/A                                     On-demand tunnel for Android users using the native L2TP/IPsec client.


Cisco Client                               N/A                                     On-demand tunnel for users using the Cisco IPsec client.


N/A                                           N/A                                     No Template.


VPN tunnel list

Once you create an IPsec VPN tunnel, it appears in the VPN tunnel list at VPN > IPsec Tunnels. By default, the tunnel list indicates the name of the tunnel, its interface binding, the tunnel template used, and the tunnel status. If you right-click on the table header row, you can include columns for comments, IKE version, mode (aggressive vs main), phase 2 proposals, and reference number. The tunnel list page also includes the option to create a new tunnel, as well as the options to edit or delete a highlighted tunnel.


VPN gateways

A gateway is a router that connects the local network to other networks. The default gateway setting in your computer’s TCP/IP properties specifies the gateway for your local network.

A VPN gateway functions as one end of a VPN tunnel. It receives incoming IPsec packets, decrypts the encapsulated data packets and passes the data packets to the local network. Also, it encrypts data packets destined for the other end of the VPN tunnel, encapsulates them, and sends the IPsec packets to the other VPN gateway. The VPN gateway is a FortiGate unit because the private network behind it is protected, ensuring the security of the unencrypted VPN data. The gateway can also be FortiClient software running on a PC since the unencrypted data is secure on the PC.

The IP address of a VPN gateway is usually the IP address of the network interface that connects to the Internet. Optionally, you can define a secondary IP address for the interface and use that address as the local VPN gateway address. The benefit of doing this is that your existing setup is not affected by the VPN settings.

The following diagram shows a VPN connection between two private networks with FortiGate units acting as the VPN gateways. This configuration is commonly referred to as Gateway-to-Gateway IPsec VPN.


VPN tunnel between two private networks

Although the IPsec traffic may actually pass through many Internet routers, you can visualize the VPN tunnel as a simple secure connection between the two FortiGate units.

Users on the two private networks do not need to be aware of the VPN tunnel. The applications on their computers generate packets with the appropriate source and destination addresses, as they normally do. The FortiGate units manage all the details of encrypting, encapsulating, and sending the packets to the remote VPN gateway.

The data is encapsulated in IPsec packets only in the VPN tunnel between the two VPN gateways. Between the user’s computer and the gateway, the data is on the secure private network and it is in regular IP packets.

For example User1 on the Site A network, at IP address, sends packets with destination IP address, the address of User2 on the Site B network. The Site A FortiGate unit is configured to send packets with destinations on the network through the VPN, encrypted and encapsulated. Similarly, the Site B FortiGate unit is configured to send packets with destinations on the network through the VPN tunnel to the Site A VPN gateway.

In the site-to-site, or gateway-to-gateway VPN shown below, the FortiGate units have static (fixed) IP addresses and either unit can initiate communication.

You can also create a VPN tunnel between an individual PC running FortiClient and a FortiGate unit, as shown below. This is commonly referred to as Client-to-Gateway IPsec VPN.


VPN tunnel between a FortiClient PC and a FortiGate unit

On the PC, the FortiClient application acts as the local VPN gateway. Packets destined for the office network are encrypted, encapsulated into IPsec packets, and sent through the VPN tunnel to the FortiGate unit. Packets for other destinations are routed to the Internet as usual. IPsec packets arriving through the tunnel are decrypted to recover the original IP packets.


Clients, servers, and peers

A FortiGate unit in a VPN can have one of the following roles:

  • Server — responds to a request to establish a VPN tunnel.
  • Client — contacts a remote VPN gateway and requests a VPN tunnel.
  • Peer — brings up a VPN tunnel or responds to a request to do so.

The site-to-site VPN shown above is a peer-to-peer relationship. Either FortiGate unit VPN gateway can establish the tunnel and initiate communications. The FortiClient-to-FortiGate VPN shown below is a client-server relationship. The FortiGate unit establishes a tunnel when the FortiClient PC requests one.

A FortiGate unit cannot be a VPN server if it has a dynamically-assigned IP address. VPN clients need to be configured with a static IP address for the server. A FortiGate unit acts as a server only when the remote VPN gateway has a dynamic IP address or is a client-only device or application, such as FortiClient.

As a VPN server, a FortiGate unit can also offer automatic configuration for FortiClient PCs. The user needs to know only the IP address of the FortiGate VPN server and a valid user name/password. FortiClient downloads the VPN configuration settings from the FortiGate VPN server. For information about configuring a FortiGate unit as a VPN server, see the FortiClient Administration Guide.



Encryption mathematically transforms data to appear as meaningless random numbers. The original data is called plaintext and the encrypted data is called ciphertext. The opposite process, called decryption, performs the inverse operation to recover the original plaintext from the ciphertext.

The process by which the plaintext is transformed to ciphertext and back again is called an algorithm. All algorithms use a small piece of information, a key, in the arithmetic process of converted plaintext to ciphertext, or vice-versa. IPsec uses symmetrical algorithms, in which the same key is used to both encrypt and decrypt the data. The security of an encryption algorithm is determined by the length of the key that it uses. FortiGate IPsec VPNs offer the following encryption algorithms, in descending order of security:

AESGCM                                Galois/Counter Mode (GCM), a block cipher mode of operation providing both confidentiality and data origin authentication.

AES256                                       A 128-bit block algorithm that uses a 256-bit key.

AES192                                       A 128-bit block algorithm that uses a 192-bit key.

AES128                                       A 128-bit block algorithm that uses a 128-bit key.

3DES                                           Triple-DES, in which plain text is DES-encrypted three times by three keys.

DES                                             Digital Encryption Standard, a 64-bit block algorithm that uses a 56-bit key

The default encryption algorithms provided on FortiGate units make recovery of encrypted data almost impossible without the proper encryption keys.

There is a human factor in the security of encryption. The key must be kept secret, known only to the sender and receiver of the messages. Also, the key must not be something that unauthorized parties might easily guess, such as the sender’s name, birthday or simple sequence such as 123456.


IPsec overheads

The FortiGate sets an IPsec tunnel Maximum Transmission Unit (MTU) of 1436 for 3DES/SHA1 and an MTU of 1412 for AES128/SHA1, as seen with diag vpn tunnel list. This indicates that the FortiGate allocates 64 bytes of overhead for 3DES/SHA1 and 88 bytes for AES128/SHA1, which is the difference if you subtract this MTU from a typical ethernet MTU of 1500 bytes.

During the encryption process, AES/DES operates using a specific size of data which is block size. If data is smaller than that, it will be padded for the operation. MD5/SHA-1 HMAC also operates using a specific block size.

The following table describes the potential maximum overhead for each IPsec encryption:


IPsec Transform Set                                                                 IPsec Overhead (Max. bytes)

ESPAES (256, 192, or 128),ESP-SHA-HMAC, or MD5               73

ESPAES (256, 192, or 128)                                                           61


IPsec Transform Set IPsec Overhead (Max. bytes)



















To protect data via encryption, a VPN must ensure that only authorized users can access the private network. You must use either a preshared key on both VPN gateways or RSA X.509 security certificates. The examples in this guide use only preshared key authentication. Refer to the Fortinet Knowledge Base for articles on RSA X.509 security certificates.


Preshared keys

A preshared key contains at least six random alphanumeric characters. Users of the VPN must obtain the preshared key from the person who manages the VPN server and add the preshared key to their VPN client configuration.

Although it looks like a password, the preshared key, also known as a shared secret, is never sent by either gateway. The preshared key is used in the calculations at each end that generate the encryption keys. As soon as the VPN peers attempt to exchange encrypted data, preshared keys that do not match will cause the process to fail.


Additional authentication

To increase security, you can require additional means of authentication from users, such as:

  • An identifier, called a peer ID or a local ID.
  • Extended authentication (XAUTH) which imposes an additional user name/password requirement.


A Local ID is an alphanumeric value assigned in the Phase 1 configuration. The Local ID of a peer is called a Peer ID.

In FortiOS 5.2, new authentication methods have been implemented for IKE: ECDSA-256, ECDSA-384, and ECDSA-521. However, AES-XCBC is not supported.


Phase 1 and Phase 2 settings

A VPN tunnel is established in two phases: Phase 1 and Phase 2. Several parameters determine how this is done. Except for IP addresses, the settings simply need to match at both VPN gateways. There are defaults that are appropriate for most cases.

FortiClient distinguishes between Phase 1 and Phase 2 only in the VPN Advanced settings and uses different terms. Phase 1 is called the IKE Policy. Phase 2 is called the IPsec Policy.

Phase 1

In Phase 1, the two VPN gateways exchange information about the encryption algorithms that they support and then establish a temporary secure connection to exchange authentication information.

When you configure your FortiGate unit or FortiClient application, you must specify the following settings for Phase 1:

Remote gateway                        The remote VPN gateway’s address.

FortiGate units also have the option of operating only as a server by select- ing the “Dialup User” option.

Preshared key                           This must be the same at both ends. It is used to encrypt Phase 1 authen- tication information.

Local interface                          The network interface that connects to the other VPN gateway. This applies on a FortiGate unit only.

All other Phase 1 settings have default values. These settings mainly configure the types of encryption to be used. The default settings on FortiGate units and in the FortiClient application are compatible. The examples in this guide use these defaults.

For more detailed information about Phase 1 settings, see Phase 1 parameters on page 1624.


Phase 2

Similar to the Phase 1 process, the two VPN gateways exchange information about the encryption algorithms that they support for Phase 2. You may choose different encryption for Phase 1 and Phase 2. If both gateways have at least one encryption algorithm in common, a VPN tunnel can be established. Keep in mind that more algorithms each phase does not share with the other gateway, the longer negotiations will take. In extreme cases this may cause timeouts during negotiations.

To configure default Phase 2 settings on a FortiGate unit, you need only select the name of the corresponding Phase 1 configuration. In FortiClient, no action is required to enable default Phase 2 settings. For more detailed information about Phase 2 settings, see Phase 2 parameters on page 1642.


Security Association

The establishment of a Security Association (SA) is the successful outcome of Phase 1 negotiations. Each peer maintains a database of information about VPN connections. The information in each SA can include cryptographic algorithms and keys, keylife, and the current packet sequence number. This information is kept synchronized as the VPN operates. Each SA has a Security Parameter Index (SPI) that is provided to the remote peer at the time the SA is established. Subsequent IPsec packets from the peer always reference the relevant SPI. It is possible for peers to have multiple VPNs active simultaneously, and correspondingly multiple SPIs.


IKE and IPsec packet processing

Internet Key Exchange (IKE) is the protocol used to set up SAs in IPsec negotiation. As described in Phase 1 parameters on page 1624, you can optionally choose IKEv2 over IKEv1 if you configure a route-based IPsec VPN. IKEv2 simplifies the negotiation process, in that it:

  • Provides no choice of Aggressive or Main mode in Phase 1.
  • Does not support Peer Options or Local ID.
  • Does not allow Extended Authentication (XAUTH).
  • Allows you to select only one Diffie-Hellman Group.
  • Uses less bandwidth.

The following sections identify how IKE versions 1 and 2 operate and differentiate.



Phase 1

A peer, identifed in the IPsec policy configuration, begins the IKE negotiation process. This IKE Security Association (SA) agreement is known as Phase 1. The Phase 1 parameters identify the remote peer or clients and supports authentication through preshared keys or digital certificates. You can increase access security further using peer identifiers, certificate distinguished names, group names, or the FortiGate extended authentication (XAuth) option for authentication purposes. Basically, Phase 1 authenticates a remote peer and sets up a secure communication channel for establishing Phase 2, which negotiates the IPsec SA.

IKE Phase 1 can occur in either Main mode or Aggressive mode. For more information, see Phase 1 parameters on page 1624.

IKE Phase 1 is successful only when the following are true:

  • Each peer negotiates a matching IKE SA policy.
  • Each peer is authenticated and their identities protected.
  • The Diffie-Hellman exchange is authenticated (the pre-shared secret keys match). For more information on Phase 1, see Phase 1 parameters on page 1624.

Phase 2

Phase 2 parameters define the algorithms that the FortiGate unit can use to encrypt and transfer data for the remainder of the session in an IPsec SA. The basic Phase 2 settings associate IPsec Phase 2 parameters with a Phase 1 configuration.

In Phase 2, the VPN peer or client and the FortiGate unit exchange keys again to establish a more secure communication channel. The Phase 2 Proposal parameters select the encryption and authentication algorithms needed to generate keys for protecting the implementation details of the SA. The keys are generated automatically using a Diffie-Hellman algorithm.

In Phase 2, Quick mode selectors determine which IP addresses can perform IKE negotiations to establish a tunnel. By only allowing authorized IP addresses access to the VPN tunnel, the network is more secure. For more information, see Phase 2 parameters on page 1642.

IKE Phase 2 is successful only when the following are true:

  • The IPsec SA is established and protected by the IKE SA.
  • The IPsec SA is configured to renegotiate after set durations (see Phase 2 parameters on page 1642 and Phase 2 parameters on page 1642).
  • Optional: Replay Detection is enabled. Replay attacks occur when an unauthorized party intercepts a series of IPsec packets and replays them back into the tunnel. See Phase 2 parameters on page 1642.
  • Optional: Perfect Forward Secrecy (PFS) is enabled. PFS improves security by forcing a new Diffie-Hellman exchange whenever keylife expires. See Phase 2 parameters on page 1642.

For more information on Phase 2, see Phase 2 parameters on page 1642.

With Phase 2 established, the IPsec tunnel is fully negotiated and traffic between the peers is allowed until the SA terminates (for any number of reasons; time-out, interruption, disconnection, etc).


Phase 1

Unlike Phase 1 of IKEv1, IKEv2 does not provide options for Aggressive or Main mode. Furthermore, Phase 1 of IKEv2 begins immediately with an IKE SA initiation, consisting of only two packets (containing all the information typically contained in four packets for IKEv1), securing the channel such that all following transactions are encrypted (see Phase 1 parameters on page 1624).

The encrypted transactions contain the IKE authentication, since remote peers have yet to be authenticated. This stage of IKE authentication in IKEv2 can loosely be called Phase 1.5.


Phase 1.5

As part of this phase, IKE authentication must occur. IKE authentication consists of the following:

  • The authentication payloads and Internet Security Association and Key Management Protocol (ISAKMP) identifier.
  • The authentication method (RSA, PSK, ECDSA, or EAP).
  • The IPsec SA parameters.

Due to the number of authentication methods potentially used, and SAs established, the overall IKEv2 negotiation can range from 4 packets (no EAP exchange at all) to many more.

At this point, both peers have a security association complete and ready to encrypt traffic.


Phase 2

In IKEv1, Phase 2 uses Quick mode to negotiate an IPsec SA between peers. In IKEv2, since the IPsec SA is already established, Phase 2 is essentially only used to negotiate “child” SAs, or to re-key an IPsec SA. That said, there are only two packets for each exchange of this type, similar to the exchange at the outset of Phase 1.5.


Support for IKEv2 session resumption described in RFC 5723

If a gateway loses connectivity to the network, clients can attempt to re-establish the lost session by presenting the ticket to the gateway. As a result, sessions can be resumed much faster, as DH exchange that is necessary to establish a brand new connection is skipped. This feature implements “ticket-by-value”, whereby all information necessary to restore the state of a particular IKE SA is stored in the ticket and sent to the client.