What’s new in FortiOS 5.6 IPSec

What’s new in FortiOS 5.6

This chapter describes new IPsec VPN features added to FortiOS 5.6.0 and FortiOS 5.6.1.

FortiOS 5.6.1

These features first appeared in FortiOS 5.6.1.

Support for Brainpool curves specified in RFC 6954 for IKE (412795)

Added support for Brainpool curves specified in RFC 6954 (originally RFC 5639) for IKE. Four new values are added for VPN phase1 and phase2 DH groups.

The allocated transform IDs are 27, 28, 29, 30:

  • 27 – Brainpool 224-Bit Curve
  • 28 – Brainpool 256-Bit Curve
  • 29 – Brainpool 384-Bit Curve
  • 30 – Brainpool 512-Bit Curve


config vpn ipsec phase1/phase1-interface edit <name> set dhgrp {1 | 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 27 | 28 | 29 | 30}



config vpn ipsec phase2/phase2-interface edit <name> set dhgrp {1 | 2 | 5 | 14 | 15 | 16 | 17 | 18 | 19 | 20 | 21 | 27 | 28 | 29 | 30}



Removed “exchange-interface-ip” option from “vpn ipsec phase1” (411981)

The command exchange-interface-ip only works for interface-based IPsec VPN (vpn ipsec phase1interface), and so it has been removed from policy-based IPsec VPN (vpn ipsec phase1).

IKEv2 ancillary RADIUS group authentication (406497)

This feature provides for the IDi information to be extracted from the IKEv2 AUTH exchange and sent to a RADIUS server, along with a fixed password configurable via CLI, to perform an additional group authentication step prior to tunnel establishment. The RADIUS server may return framed-IP-address, framed-ip-netmask, and dns-server attributes, which are then applied to the tunnel.



It should be noted, unlike Xauth or EAP, this feature does not perform individual user authentication, but rather treats all users on the gateway as a single group, and authenticates that group with RADIUS using a fixed password. This feature also works with RADIUS accounting, including the phase1 acct-verify option.


config vpn ipsec phase1-interface edit <name> set mode-cfg enable set type dynamic set ike-version 2

set group-authentication {enable | disable} set group-authentication-secret <password>



IPsec mode-cfg can assign IPs from firewall address and sharing IP pools (393331)

This feature adds the ability for users to configure assign-IPs from firewall addresses/groups.

Previously, different policies accessing the same network needed to ensure that non-overlapping IP-ranges were assigned to policies to avoid the same IP address being assigned to multiple clients. With this feature, the address name is used to identify an IP pool and different policies can refer to the same IP pool to check for available IPs, thus simplifying the task of avoiding IP conflicts.


config vpn ipsec phase1-interface edit <name> set mode-cfg enable set type dynamic

set assign-ip-from {range | dhcp | name} set ipv4-name <name> set ipv6-name <name>



Improve interface-based dynamic IPsec up/down time (379937)

This feature makes it possible to use a single interface for all instances that spawn via a given phase1. Instead of creating an interface per instance, all traffic will run over the single interface and any routes that need creating will be created on that single interface.

A new CLI option net-device is added in the phase1-interface command sets. The default is disable so that the new feature kicks in for all the new configurations. An upgrade feature will add a set net-device enable for all the existing configurations so that they will keep the old behavior.

Under the new single-interface scheme, instead of relying on routing to guide traffic to the specific instance, all traffic will flow to the specific device and IPsec will need to take care of locating the correct instance for outbound traffic. For this purpose, another new CLI option tunnel-search is created. The option is only available when the above net-device option is set to disable.

There are two options for tunnel-search, corresponding to the two ways to select the tunnel for outbound traffic. One is selectors, meaning selecting a peer using the IPsec selectors (proxy-ids). The other is


nexthop where all the peers use the same default selectors (0/0) while using some routing protocols such as BGP, OSPF, RIPng, etc. to resolve the routing. The default for tunnel-search is selectors.


config vpn ipsec phase1-interface edit <name> set net-device {enable | disable} set tunnel-search {selectors | nexthop}



Hide psksecret option when peertype is dialup (415480)

In aggressive mode and IKEv2, when peertype is dialup, pre-shared key is per-user based. There is no need to configure the psksecret in the phase1 setup. Previously, if left unconfigured, CLI would output psksecret error and fail to create the phase1 profile.

To prevent psksecret length check running on the configuration end, the psksecret option will be hidden. Prior to Mantis 397712, the length check passed because it was incorrectly checking the legnth of encrypted password which is always 204 length long.

Peertype dialup option removed for main mode.

New enforce-ipsec option added to L2TP config (423988)

A new enforce-ipsec option is added in L2TP configuration to force the FortiGate L2TP server to accept only IPsec encrypted connections.


config vpn l2tp set eip set sip set status enable

set enforce-ipsec-interface {disable | enable}      (default = disable) set usrgrp <group_name>


IPsec VPN Wizard improvements (368069)

Previously, when using wan-load-balance (WLB) feature, and when configuring an IPsec tunnel with the wizard, the setting ‘incoming interface’ list does not contain the wan-load-balance nor the wan2 interface. Disabling the WLB permits the configuration.

The solution in 5.6.1 is as follows:

  • (368069) The IPsec VPN wizard now allows users to select members of virtual-wan-link (VWL) as IPsec phase1interface. Before saving, if the phase1 interface is a VWL member, then the Wizard automatically sets the virtualwan-link as the destination interface in the L2TP policy.
  • (246552) List VPN tunnels for VWL members if VWL is set as the destination interface in policy-based IPsec VPN.

IPsec manual key support removed from GUI (436041)

The majority of customers are not using policy-based IPsec today, and beyond that, very few are using manual key VPN. As a result, the IPsec manual key feature is removed from the GUI; the feature store option is removed as well.

Added GUI support for local-gw when configuring custom IPsec tunnels (423786)

Previously, the local-gw option was not available on the GUI when configuring a custom IPsec tunnel. This feature adds the local-gw setting to the IPsec VPN Edit dialog. The user is able to choose the primary or secondary IP address from the currently selected interface, or specify an ip address manually. Both local-gw and local-gw6 are supported.

Moved the dn-format CLI option from phase1 config to vdom settings (435542)

Previous fix for dn-format didn’t take into account that, at the time isakmp_set_peer_identifier is used, we don’t have a connection and haven’t matched our gateway yet, so we can’t use that to determine the dn-format configuration setting.

The solution was to move the dn-format CLI option from phase1 config to vdom settings. It is renamed to ike-dn-format.

FGT IKE incorrect NAT detection causes ADVPN hub behind VIP to not generate shortcuts (416786)

When ADVPN NAT support was added, only spokes behind NAT was considered. No thought was given to a hub behind a VIP or the problems that occurred due to the way that FortiOS clients behind NAT enable NAT-T even when it is not required.

The solution in 5.6.1 is as follows:

  • Moved shortcut determination out of the kernel and up to IKE. The shortcut message now contains the ID of both tunnels so that IKE can check the NAT condition of both.
  • Added IKE debug to cover sending the initial shortcut query. The lack of this previously meant it could be awkard to determine if the offer had been converted into a query correctly.
  • Added “nat:” output in diag vpn ike gateway list output to indicate whether this device or the peer is behind NAT.
  • Tweaked the diag vpn tunnel list output so that the auto-discovery information now includes symbolic as well as numeric values, which makes it easier to see what type of auto-discovery was enabled.

FortiOS 5.6.0

These features first appeared in FortiOS 5.6.0.


Improvement to stats crypto command output (403995)

The CLI command get vpn ipsec stats crypto now has a better format for the information it shows in differentiating between NP6 lite and SOC3 (CP). To further avoid confusion, all engine’s encryption (encrypted/decrypted) and integrity (generated/validated) information is shown under the same heading, not separate headings.

Improved certificate key size control commands (397883)

Proxy will choose the same SSL key size as the HTTPS server. If the key size from the server is 512, the proxy will choose 1024. If the key size is bigger than 1024, the proxy will choose 2048.

As a result, the firewall ssl-ssh-profile commands certname-rsa, certname-dsa, and certname-ecdsa have been replaced with more specific key size control commands under vpn certificate setting.

CLI syntax

config vpn certificate setting set certname-rsa1024 <name> set certname-rsa2048 <name> set certname-dsa1024 <name> set certname-dsa2048 <name> set certname-ecdsa256 <name> set certname-ecdsa384 <name>


Support bit-based keys in IKE (397712)

As per FIPS-CC required standards, as well as RFC 4306, IKE supports pre-shared secrets to be entered as both ASCII string values and as hexadecimal encoded values. This feature parses hex encoded input (indicated by the leading characters 0x) and converts the input into binary data for storage.

With this change, the psksecret and psksecret-remote entries under the IPsec VPN CLI command config vpn ipsec-phase1-interface have been amended to differentiate user input as either ASCII string or hex encoded values.

IKEv2 asymmetric authentication (393073)

Support added for IKEv2 asymmetric authentication, allowing both sides of an authentication exchange to use different authentication methods, for example the initiator may be using a shared key, while the responder may have a public signature key and certificate.

A new command, authmethod-remote, has been added to config vpn ipsec phase1-interface.

For more detailed information on authentication of the IKE SA, see RFC 5996 Internet Key Exchange Protocol Version 2 (IKEv2).

Allow mode-cfg with childless IKEv2 (391567)

An issue that prevented childless-ike from being enabled at the same time as mode-cfg has been resolved. Both options can now be enabled at once under config vpn ipsec phase1-interface.

IKEv2 Digital Signature Authentication support (389001)

FortiOS supports the use of Digital Signature authentication, which changes the format of the Authentication Data payload in order to support different signature methods.

Instead of just  containing a raw signature value calculated as defined in the original IKE RFCs,  the Auth Data now includes an ASN.1 formatted object that provides  details on how the signature was calculated, such as the signature type, hash algorithm, and signature padding method.

For more detailed information on IKEv2 Digital Signature authentication, see RFC 7427 Signature Authentication in the Internet Key Exchange Version 2 (IKEv2).

Passive static IPsec VPN (387913)

New commands have been added to config vpn ipsec phase1-interface to prevent initiating

VPN connection. Static IPsec VPNs can be configured in tunnel mode, without initiating tunnel negotiation or rekey.

To allow a finer configuration of the tunnel, the rekey option is removed from config system global and added to config vpn ipsec phase1-interface.

CLI syntax

config vpn ipsec phase1-interface edit <example> set rekey {enable | disable} set passive-mode {enable | disable} set passive-tunnel-interface {enable | disable}


Phase 2 wizard simplified (387725)

Previously, for a site-to-site VPN, phase 2 selectors had their static routes created in the IPsec VPN wizard by adding IP addresses  in string format. Now, since addresses and address groups are already created for these addresses, the address group can be used in the route directly. This means that the route can be modified simply by modifying the address/groups that were created when the VPN was initially created.

With this change, the VPN wizard will create less objects internally, and reduce complexity.

In addition, a blackhole route route will be created by default with a higher distance-weight set than the default route. This is to prevent traffic from flowing out of another route if the VPN interface goes down. In these instances, the traffic will instead be silently discarded.

Unique IKE ID enforcement (383296)

All IPsec VPN peers now connect with unique IKE identifiers. To implement this, a new phase1 CLI command has been added (enforce-unique-id) which, when enabled, requires all IPsec VPN clients to use a unique identifier when connecting.

CLI syntax

config vpn ipsec phase1 edit <name>


set enforce-unique-id {keep-new | keep-old | disable} Default is disable. next



Use keep-new to replace the old connection if an ID collision is detected on the gateway. Use keep-old to reject the new connection if an ID collision is detected.

FortiView VPN tunnel map feature (382767)

A geospatial map has been added to FortiView to help visualize IPsec and SSL VPN connections to a FortiGate using Google Maps. Adds geographical-IP API service for resolving spatial locations from IP addresses.

This feature can be found under FortiView > VPN.

Childless IKEv2 initiation (381650)

As documented in RFC 6023, when both sides support the feature, no child IPsec SA is brought up during the initial AUTH of the IKEv2 negotiation. Support for this mode is not actually negotiated, but the responder indicates support for it by including a CHILDLESS_IKEV2_SUPPORTED Notify in the initial SA_INIT reply. The initiator is then free to send its AUTH without any SA or TS payloads if it also supports this extension.

CLI syntax

config vpn ipsec phase1-interface edit ike set ike-version 2 set childless-ike enable



Due to the way configuration payloads (IKEV2_PAYLOAD_CONFIG) are handled in the current code base, mode-cfg and childless-ike aren’t allowed to be enabled at the same time. Processing config payloads for mode-cfg requires a child ph2handle to be created, but with childless-ike we completely avoid creating the child ph2 in the first place which makes the two features incompatible. It may be possible to support both in the future, but a deeper rework of the config payload handling is required.

Allow peertype dialup for IKEv2 pre-shared key dynamic phase1 (378714)

Restored peertype dialup that was removed in a previous build (when IKEv2 PSK gateway re-validation was not yet supported).

If peertype is dialup, IKEv2 AUTH verify uses user password in the user group “usrgrp” of phase1. The “psksecret” in phase1 is ignored.

CLI syntax

config vpn ipsec phase1-interface edit “name” set type dynamic set interface “wan1” set ike-version 2 set peertype dialup

set usrgrp “local-group”



IPsec default phase1/phase1-interface peertype changed from ‘any’ to ‘peer’ (376340)

Previously, when authmethod was changed to signature, peertype automatically changed to peer and required a peer to be set. This change was done to try to provide a more secure initial configuration, while allowing the admin to set peertype back to any if that’s what they really wanted. The default value was kept at any in the CLI. However, this caused problems with copy/pasting configurations and with FMG because if peertype any wasn’t explicitly provided, the CLI was switched to peertype peer.

This patch changes the default peertype to peer now; peertype any is considered non-default and will be printed out on any config listing. Upgrade code has been written to ensure that any older build that was implicitly using set peertype any has this setting preserved.

IPsec GUI bug fixes (374326)

Accept type “Any peer ID” is available when creating IPsec tunnel with authmethod, pre-shared key, ikev1 main mode/aggressive mode, and ikev2.

Support for IKEv2 Message Fragmentation (371241)

Added support for IKEv2 Message Fragmentation, as described in RFC 7383.

Previously, when sending and IKE packets with IKEv1, the whole packet is sent once, and it is only fragmented if there is a retransmission. With IKEv2, because RFC 7383 requires each fragment to be individually encrypted and authenticated, we would have to keep a copy of the unencrypted payloads around for each outgoing packet, in case the original single packet was never answered and we wanted to retry with fragments. So with this implementation, if the IKE payloads are greater than a configured threshold, the IKE packets are preemptively fragmented and encrypted.

CLI syntax

config vpn ipsec phase1-interface edit ike set ike-version 2

set fragmentation [enable|disable] set fragmentation-mtu [500-16000]



IPsec monitoring pages now based on phase 1 proposals not phase 2 (304246)

The IPsec monitor, found under Monitor > IPsec Monitor, was in some instances showing random uptimes even if the tunnel was in fact down.

Tunnels are considered as “up” if at least one phase 2 selector is active. To avoid confusion, when a tunnel is down, IPsec Monitor will keep the Phase 2 Selectors column, but hide it by default and be replaced with Phase 1 status column.


Having trouble configuring your Fortinet hardware or have some questions you need answered? Ask your questions in the comments below!!! Want someone else to deal with it for you? Get some consulting from Fortinet GURU!

Don't Forget To Buy Your Fortinet Hardware From The Fortinet GURU