Setting Up The System

Deployment guidelines

Generally speaking, gateway mode is suitable for most deployment environments. It is usually easier to implement and better understood. Exceptions are situations where neither DNS MX records nor IP addresses cannot be modified.

Transparent mode was developed for the purpose of implementing FortiMail in carrier environments to combat outgoing spam. It is suitable for certain environments but needs more careful routing handling and good understanding of network and application layer transparency.

Transparent mode is the best choice for combatting outgoing spam in carrier environments.

You use server mode to set up a standalone email server or to replace an existing email server.

After you set the operation mode, run the Quick Start Wizard to set up a basic system. Then deploy your FortiMail unit. The details vary depending on the operation mode you chose. For instructions, consult the applicable sections:

  • Gateway mode deployment
  • Transparent mode deployment
  • Server mode deployment

Characteristics of gateway mode

When operating in gateway mode, the FortiMail unit acts as a mail transfer agent (MTA), sometimes known as an email gateway or relay. The FortiMail unit receives email messages, scans for viruses and spam, then relays email to its destination email server for delivery. External MTAs connect to the FortiMail unit, rather than directly to the protected email server.

FortiMail units operating in gateway mode provide a web-based user interface from which email users can access personal preferences and their per-recipient quarantined email. However, FortiMail units operating in gateway mode do not locally host mailboxes such as each email user’s inbox. Mailboxes are stored on the protected email servers.

Gateway mode requires some changes to an existing network. Requirements include MX records on public DNS servers for each protected domain, which must refer to the FortiMail unit instead of the protected email servers. You may also need to configure firewalls or routers to direct SMTP traffic to the FortiMail unit rather than your email servers.

Figure 2: Example gateway mode topology

mail IN A 172.16.1.10                                                                                             fortimail IN A 10.10.10.1

For example, an Internet service provider (ISP) could deploy a FortiMail unit to protect their customers’ email servers. For security reasons, customers do not want their email servers to be directly visible to external MTAs. Therefore, the ISP installs the FortiMail unit in gateway mode, and configures its network such that all email traffic must pass through the FortiMail unit before reaching customers’ email servers.

For sample deployment scenarios, see “Gateway mode deployment” on page 50.

Characteristics of transparent mode

When operating in transparent mode, the FortiMail acts as either an implicit relay or a proxy. The FortiMail unit intercepts email messages, scans for viruses and spam, then transmits email to its destination email server for delivery. External MTAs connect through the FortiMail unit to the protected email server.

Transparency at both the network and application layers is configurable, but not required. When hiding, the FortiMail unit preserves the IP address and domain name of the SMTP client in IP headers and the SMTP envelope and message headers, rather than replacing them with its own.

FortiMail units operating in transparent mode provide a web-based user interface from which email users can access personal preferences and email quarantined to their per-recipient quarantine. However, FortiMail units operating in transparent mode do not locally host mailboxes such as each email user’s inbox. These mailboxes are stored on the protected email servers.

By default, FortiMail units operating in transparent mode are configured as a bridge, with all network interfaces on the same subnet. You can configure out-of-bridge network interfaces if you require them, such as if you have some protected email servers that are not located on the same subnet. If you set an interface to route mode, you must assign the interface a local IP address that belongs to a different subnet from that of the management IP.

Transparent mode usually requires no changes to an existing network. Requirements include that the FortiMail unit must be physically inline between the protected email server and all SMTP clients—unlike gateway mode. Because FortiMail units operating in transparent mode are invisible, clients cannot be configured to route email directly to the FortiMail unit; so, it must be physically placed where it can intercept the connection.

Figure 3: Example transparent mode topology

172.16.1.10                                        Private DNS Server Public DNS Server

Email Domain: example.com IN MX 10 mail.example.com example.com IN MX 10 mail.example.com

@example.com mail IN A 172.16.1.10 mail IN A 10.10.10.1

Do not connect two ports to the same VLAN on a switch or the same hub. Some Layer 2 switches become unstable when they detect the same media access control (MAC) address originating on more than one network interface on the switch, or from more than one VLAN.

For example, a school might want to install a FortiMail unit to protect its mail server, but does not want to make any changes to its existing DNS and SMTP client configurations or other network topology. Therefore, the school installs the FortiMail unit in transparent mode.

For sample deployment scenarios, see “Transparent mode deployment” on page 78.

Characteristics of server mode

When operating in server mode, the FortiMail is a standalone email server. The FortiMail unit receives email messages, scans for viruses and spam, and then delivers email to its email users’ mailboxes. External MTAs connect to the FortiMail unit, which itself is also the protected email server.

FortiMail units operating in server mode provide a web-based user interface from which email users can access:

  • personal preferences
  • email quarantined to their per-recipient quarantine
  • their locally hosted mailboxes such as each email user’s inbox.

In addition, email users can retrieve email using POP3 or IMAP.

Server mode requires some changes to an existing network. Requirements include MX records on public DNS servers for each protected domain. The records must refer to the FortiMail unit. You may also need to configure firewalls or routers to direct SMTP traffic to the FortiMail unit.

Figure 4: Example server mode topology

For example, a company might be creating a network, and does not have an existing email server. The company wants the convenience of managing both their email server and email security on one network device. Therefore, the company deploys the FortiMail unit in server mode.

For sample deployment scenarios, see “Server mode deployment” on page 101.

Changing the operation mode

By default, FortiMail units operate in gateway mode. If you do not want your FortiMail unit to operate in gateway mode, before configuring the FortiMail unit or using the Quick Start Wizard, select the operation mode.

To select the operation mode

  1. Open the web UI. (See “Connecting to the FortiMail web UI for the first time”.)
  2. The dashboard appears on the System Status > Status
  3. In the System Information widget, select either Gateway, Server, or Transparent from the Operation mode drop-down list.

A confirmation dialog appears, warning you that many settings will revert to their default value for the version of your FortiMail unit’s firmware.

  1. Select OK.

The FortiMail unit changes the operation mode and restarts. The Login dialog of the web UI appears.


Having trouble configuring your Fortinet hardware or have some questions you need answered? Check Out The Fortinet Guru Youtube Channel! Want someone else to deal with it for you? Get some consulting from Fortinet GURU!

This entry was posted in Administration Guides, FortiMail and tagged , on by .

About Mike

Michael Pruett, CISSP has a wide range of cyber-security and network engineering expertise. The plethora of vendors that resell hardware but have zero engineering knowledge resulting in the wrong hardware or configuration being deployed is a major pet peeve of Michael's. This site was started in an effort to spread information while providing the option of quality consulting services at a much lower price than Fortinet Professional Services. Owns PacketLlama.Com (Fortinet Hardware Sales) and Office Of The CISO, LLC (Cybersecurity consulting firm).

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.