Category Archives: FortiOS

Explicit proxy concepts

The following is information that is specific to Explicit Proxy concepts. Any information that is common to Web

The FortiGate explicit web proxy

You can use the FortiGate explicit web proxy to enable explicit proxying of IPv4 and IPv6 HTTP, and HTTPS traffic on one or more FortiGate interfaces. The explicit web proxy also supports proxying FTP sessions from a web browser and proxy auto-config (PAC) to provide automatic proxy configurations for explicit web proxy users. From the CLI you can also configure the explicit web proxy to support SOCKS sessions from a web browser.

The explicit web and FTP proxies can be operating at the same time on the same or on different FortiGate interfaces.

In most cases you would configure the explicit web proxy for users on a network by enabling the explicit web proxy on the FortiGate interface connected to that network. Users on the network would configure their web browsers to use a proxy server for HTTP and HTTPS, FTP, or SOCKS and set the proxy server IP address to the IP address of the FortiGate interface connected to their network. Users could also enter the PAC URL into their web browser PAC configuration to automate their web proxy configuration using a PAC file stored on the FortiGate unit.

Enabling the explicit web proxy on an interface connected to the Internet is a security risk because anyone on the Internet who finds the proxy could use it to hide their source address.

If the FortiGate unit is operating in transparent mode, users would configure their browsers to use a proxy server with the FortiGate management IP address.

If the FortiGate unit is operating with multiple VDOMs the explicit web proxy is configured for each VDOM.

The web proxy receives web browser sessions to be proxied at FortiGate interfaces with the explicit web proxy enabled. The web proxy uses FortiGate routing to route sessions through the FortiGate unit to a destination interface. Before a session leaves the exiting interface, the explicit web proxy changes the source addresses of the session packets to the IP address of the exiting interface. When the FortiGate unit is operating in transparent mode the explicit web proxy changes the source addresses to the management IP address. You can configure the explicit web proxy to keep the original client IP address. See The FortiGate explicit web proxy on page 374.

For more information about explicit web proxy sessions, see The FortiGate explicit web proxy on page 374.

Example explicit web proxy topology

To allow all explicit web proxy traffic to pass through the FortiGate unit you can set the explicit web proxy default firewall policy action to accept. However, in most cases you would want to use security policies to control explicit web proxy traffic and apply security features such as access control/authentication, virus scanning, web filtering, application control, and traffic logging. You can do this by keeping the default explicit web proxy security policy action to deny and then adding web-proxy security policies.

You can also change the explicit web proxy default security policy action to accept and add explicit web proxy security policies. If you do this, sessions that match web-proxy security policies are processed according to the security policy settings. Connections to the explicit web proxy that do not match a web-proxy security policy are allowed with no restrictions or additional security processing. This configuration is not recommended and is not a best practice.

The explicit web-proxy can accept VIP addresses for destination address. If an external IP matches a VIP policy, the IP is changed to the mapped-IP of the VIP.

Web-proxy policies can selectively accept or deny traffic, apply authentication, enable traffic logging, and use security profiles to apply virus scanning, web filtering, IPS, application control, DLP, and SSL/SSH inspection to explicit web proxy traffic.

You cannot configure IPsec, SSL VPN, or Traffic shaping for explicit web proxy traffic. Web Proxy policies can only include firewall addresses not assigned to a FortiGate unit interface or with interface set to Any. (On the web-based manager you must set the interface to Any. In the CLI you must unset the associatedinterface.)

Authentication of explicit web proxy sessions uses HTTP authentication and can be based on the user’s source IP address or on cookies from the user’s web browser. For more information, see The FortiGate explicit web proxy on page 374.

To use the explicit web proxy, users must add the IP address of a FortiGate interface on which the explicit web proxy is enabled and the explicit web proxy port number (default 8080) to the proxy configuration settings of their web browsers.

On FortiGate units that support it, you can also enable web caching for explicit web proxy sessions.

Other explicit web proxy options

You can change the following explicit web proxy options as required by your configuration.

HTTP port, HTTPS port, FTP port, PAC port

The TCP port that web browsers use to connect to the explicit proxy for HTTP, HTTPS, FTP and PAC services. The default port is 8080 for all services. By default HTTPS, FTP. and PAC use the same port as HTTP. You can change any of these ports as required. Users configuring their web browsers to use the explicit web proxy should add the same port numbers to their browser configurations.

Multi-port support for Explicit Proxy

Support exists for the use of multiple ports and port range in the explicit FTP or Web proxies. These changes have been added in both CLI and GUI.

CLI: set http-incoming-port <port_low>[-<port_high>]

Where:

l port_low – the low value of the port l port_high – the high value of the port

The port_high value can be omitted if port_low and port_high are the same.

Proxy FQDN

Enter the fully qualified domain name (FQDN) for the proxy server. This is the domain name to enter into browsers to access the proxy server.

Max HTTP request length

Enter the maximum length of an HTTP request in Kbytes. Larger requests will be rejected.

Max HTTP message length

Enter the maximum length of an HTTP message in Kbytes. Larger messages will be rejected.

Multiple incoming ports and port ranges

Web proxy can be configured to listen on multiple ports on the same IP as well as listen for HTTP and HTTPS on those same (or different) ports. This is done in the CLI.

Define the IP ranges using a hyphen (–). As shown below, port_high is not necessary to specify if port_low is equal to port_high.

CLI syntax

config web-proxy explicit set http-incoming-port <port_low> [-<port_high>] end

Internet services

FortiOS can use the Internet Service Database (introduced in 5.4.1) as a web-proxy policy matching factor. This can only be done in the CLI.

CLI syntax:

config firewall proxy-policy edit 0 set internet-service <application-id> set internet-service-custom <application-name>

IP pools

IP Pools can be used with web proxy. When using this option of setting the IP pool name, the outgoing IP will be selected.

CLI syntax

config firewall proxy-policy edit <example> set poolname <name> end

Proxy chaining (web proxy forwarding servers)

For the explicit web proxy you can configure web proxy forwarding servers to use proxy chaining to redirect web proxy sessions to other proxy servers. Proxy chaining can be used to forward web proxy sessions from the FortiGate unit to one or more other proxy servers on your network or on a remote network. You can use proxy chaining to integrate the FortiGate explicit web proxy with an web proxy solution that you already have in place.

A FortiGate unit can forward sessions to most web proxy servers including a remote FortiGate unit with the explicit web proxy enabled. No special configuration of the explicit web proxy on the remote FortiGate unit is required.

You can deploy the explicit web proxy with proxy chaining in an enterprise environment consisting of small satellite offices and a main office. If each office has a FortiGate unit, users at each of the satellite offices can use their local FortiGate unit as an explicit web proxy server. The satellite office FortiGate units can forward explicit web proxy sessions to an explicit web proxy server at the central office. From here the sessions can connect to web servers on the Internet.

FortiGate proxy chaining does not support authenticating with the remote forwarding server.

Adding a web proxy forwarding server

To add a forwarding server, select Create New in the Web Proxy Forwarding Servers section of the Explicit Proxy page by going to Network > Explicit Proxy.

Server Name

Enter the name of the forwarding server.

Proxy chaining (web proxy forwarding servers)

Proxy Address	Enter the IP address of the forwarding server.
Proxy Address Type	Select the type of IP address of the forwarding server. A forwarding server can have an FQDN or IP address.
Port	Enter the port number on which the proxy receives connections. Traffic leaving the FortiGate explicit web proxy for this server has its destination port number changed to this number.
Server Down action	Select what action the explicit web proxy to take if the forwarding server is down. Block means if the remote server is down block traffic. Use Original Server means do not forward traffic to the forwarding sever but instead forward it from the FortiGate to its destination. In other words operate as if there is no forwarding server configured.
Enable Health Monitor	Select to enable health check monitoring and enter the address of a remote site. See “Web proxy forwarding server monitoring and health checking”.
Health Check Monitor Site

Use the following CLI command to add a web proxy forwarding server named fwd-srv at address proxy.example.com and port 8080.

config web-proxy forward-server edit fwd-srv set addr-type fqdn set fqdn proxy.example.com

set port 8080

end

Web proxy forwarding server monitoring and health checking

By default, a FortiGate unit monitors web proxy forwarding server by forwarding a connection to the remote server every 10 seconds. If the remote server does not respond it is assumed to be down. Checking continues and when the server does send a response the server is assumed to be back up. If you configure health checking, every 10 seconds the FortiGate unit attempts to get a response from a web server by connecting through the remote forwarding server.

You can configure health checking for each remote server and specify a different website to check for each one.

If the remote server is found to be down you can configure the FortiGate unit to block sessions until the server comes back up or to allow sessions to connect to their destination, bypassing the remote forwarding server. You cannot configure the FortiGate unit to fail over to another remote forwarding server.

Configure the server down action and enable health monitoring from the web-based manager by going to Network > Explicit Proxy, selecting a forwarding server, and changing the server down action and changing the health monitor settings.

Use the following CLI command to enable health checking for a web proxy forwarding server and set the server down option to bypass the forwarding server if it is down.

config web-proxy forward-server edit fwd-srv set healthcheck enable set monitor http://example.com set server-down-option pass

end

Grouping forwarding servers and load balancing traffic to them

You can add multiple web proxy forwarding servers to a forwarding server group and then add the server group to an explicit web proxy policy instead of adding a single server. Forwarding server groups are created from the FortiGate CLI but can be added to policies from the web-based manager (or from the CLI).

When you create a forwarding server group you can select a load balancing method to control how sessions are load balanced to the forwarding servers in the server group. Two load balancing methods are available:

l Weighted load balancing sends more sessions to the servers with higher weights. You can configure the weight for each server when you add it to the group. l Least-session load balancing sends new sessions to the forwarding server that is processing the fewest sessions.

When you create a forwarding server group you can also enable affinity. Enable affinity to have requests from the same client processed by the same server. This can reduce delays caused by using multiple servers for a single multi-step client operation. Affinity takes precedence over load balancing.

You can also configure the behavior of the group if all of the servers in the group are down. You can select to block traffic or you can select to have the traffic pass through the FortiGate explicit proxy directly to its destination instead of being sent to one of the forwarding servers.

Use the following command to add a forwarding server group that users weighted load balancing to load balance traffic to three forwarding servers. Server weights are configured to send most traffic to server2. The group has affinity enabled and blocks traffic if all of the forward servers are down:

config web-proxy forward-server edit server_1 set ip 172.20.120.12 set port 8080

next edit server_2 set ip 172.20.120.13 set port 8000

next edit server_3 set ip 172.20.120.14 set port 8090

next end

config web-proxy forward-server-group edit New-fwd-group set affinity enable set ldb-method weight set group-down-option block config server-list edit server_1 set weight 10

next edit server_2 set weight 40

next edit server_3 set weight 10

next end

Adding proxy chaining to an explicit web proxy policy

You enable proxy chaining for web proxy sessions by adding a web proxy forwarding server or server group to an explicit web proxy policy. In a policy you can select one web proxy forwarding server or server group. All explicit web proxy traffic accepted by this security policy is forwarded to the specified web proxy forwarding server or server group.

To add an explicit web proxy forwarding server – web-based manager:

Go to Policy & Objects > Proxy Policy and select Create New.
Configure the policy:

Explicit Proxy Type	Web
Source Address	Internal_subnet
Outgoing Interface	wan1
Destination Address	all
Schedule	always
Action	ACCEPT
Web Proxy Forwarding Server	Select, fwd-srv

Select OK to save the security policy.

To add an explicit web proxy forwarding server – CLI:

Use the following command to add a security policy that allows all users on the 10.31.101.0 subnet to use the explicit web proxy for connections through the wan1 interface to the Internet. The policy forwards web proxy sessions to a remote forwarding server named fwd-srv config firewall proxy-policy edit 0 set proxy explicit-web set dstintf wan1 set scraddr Internal_subnet

set dstaddr all set action accept set schedule always

set webproxy-forward-server fwd-srv end

Security profiles, threat weight, device identification, and the explicit web proxy

You can apply all security profiles to explicit web proxy sessions. This includes antivirus, web filtering, intrusion protection (IPS), application control, data leak prevention (DLP), and SSL/SSH inspection. Security profiles are applied by selecting them in an explicit web proxy policy or in authentication rules added to web proxy policies.

Traffic accepted by explicit web proxy policies contributes to threat weight data.

The explicit web proxy is not compatible with device identification.

Since the traffic accepted by the explicit web proxy is known to be either HTTP, HTTPS, or FTP over HTTP and since the ports are already known by the proxy, the explicit web proxy does not use all of the SSL/SSH inspection options. The explicit web proxy does support the following proxy options:

Enable chunked bypass
HTTP oversized file action and threshold

The explicit web proxy does not support the following proxy options:

Client comforting l Server comforting l Monitor content information from dashboard. URLs visited by explicit web proxy users are not added to dashboard usage and log and archive statistics widgets.

For explicit web proxy sessions, the FortiGate unit applies antivirus scanning to HTTP POST requests and HTTP responses. The FortiGate unit starts virus scanning a file in an HTTP session when it receives a file in the body of an HTML request. The explicit web proxy can receive HTTP responses from either the originating web server or the FortiGate web cache module.

Explicit web proxy sessions and user limits

Web browsers and web servers open and close multiple sessions with the explicit web proxy. Some sessions open and close very quickly. HTTP 1.1 keepalive sessions are persistent and can remain open for long periods of time. Sessions can remain on the explicit web proxy session list after a user has stopped using the proxy (and has, for example, closed their browser). If an explicit web proxy session is idle for more than 3600 seconds it is torn down by the explicit web proxy. See RFC 2616 for information about HTTP keepalive/persistent HTTP sessions.

This section describes proxy sessions and user limits for both the explicit web proxy and the explicit FTP proxy. Session and user limits for the two proxies are counted and calculated together. However, in most cases if both proxies are active there will be many more web proxy sessions than FTP proxy sessions.

The FortiGate unit adds two sessions to its session table for every explicit proxy session started by a web browser and every FTP session started by an FTP client. An entry is added to the session table for the session from the web browser or client to the explicit proxy. All of these sessions have the same destination port as the explicit web proxy port (usually 8080 for HTTP and 21 for FTP). An entry is also added to the session table for the session between the exiting FortiGate interface and the web or FTP server destination of the session. All of these sessions have a FortiGate interface IP address and the source address of the session and usually have a destination port of 80 for HTTP and 21 for FTP.

Proxy sessions that appear in FortiView do not include the Policy ID of the web-proxy or ftp-proxy security policy that accepted them. However, the explicit proxy sessions include a destination port that matches the explicit Explicit web proxy sessions and user limits

proxy port number (usually 8080 for the web proxy and 21 for the FTP proxy). The proxied sessions from the FortiGate unit have their source address set to the IP address of the FortiGate unit interface that the sessions use to connect to their destinations (for example, for connections to the Internet the source address would be the IP address of the FortiGate interface connected to the Internet).

FortiOS limits the number of explicit proxy users. This includes both explicit FTP proxy and explicit web proxy users. The number of users varies by FortiGate model from as low as 10 to up to 18000 for high end models. You cannot raise this limit.

If your FortiGate unit is configured for multiple VDOMs you can go to System > Global Resourcesto view the maximum number of Concurrent explicit proxy users and optionally reduce the limit. You can also use the following command:

config global config system resource-limits set proxy 50

end

To limit the number of explicit proxy users for a VDOM, from the web-based manager enable multiple VDOMs and go to System > VDOM and edit a VDOM or use the following command to change the number of explicit web proxy users for VDOM_1:

config global config system vdom-property edit VDOM_1 set proxy 25

end

You can use the diagnose wad user list command to view the number of explicit web proxy users. Users may be displayed with this command even if they are no longer actively using the proxy. All idle sessions time out after 3600 seconds.

You can use the command diagnose wad user clear to clear current explicit proxy users. You can also use the command diagnose wad user clear <user-name> to clear individual users. This means delete information about all users and force them re-authenticate.

Users that authenticate with explicit web-proxy or ftp-proxy security policies do not appear in the Monitor > Firewall User Monitor list and selecting De-authenticate All Users has no effect on explicit proxy users.

How the number of concurrent explicit proxy users is determined depends on their authentication method:

For session-based authenticated users, each authenticated user is counted as a single user. Since multiple users can have the same user name, the proxy attempts to identify users according to their authentication membership (based upon whether they were authenticated using RADIUS, LADAP, FSAE, local database etc.). If a user of one session has the same name and membership as a user of another session, the explicit proxy assumes this is one user.
For IP Based authentication, or no authentication, or if no web-proxy security policy has been added, the source IP address is used to determine a user. All sessions from a single source address are assumed to be from the same user.

The explicit proxy does not limit the number of active sessions for each user. As a result the actual explicit proxy session count is usually much higher than the number of explicit web proxy users. If an excessive number of Explicit web proxy sessions and user limits

explicit web proxy sessions is compromising system performance you can limit the amount of users if the FortiGate unit is operating with multiple VDOMs.

Web proxy configuration

Leave a reply

Web proxy configuration

General web proxy configuration steps

You can use the following general steps to configure the explicit web proxy.

To enable the explicit web proxy – web-based manager:

Go to Network > Explicit Proxy and enable Explicit Web Proxy. From here you can optionally change the HTTP port that the proxy listens on (the default is 8080) and optionally specify different ports for HTTPS, FTP, PAC, and other options.
Optionally enable IPv6 Explicit Proxy to turn on the explicit web proxy for IPv6 traffic.

If you enable both the IPv4 and the IPv6 explicit web proxy you can combine IPv4 and IPv6 addresses in a single explicit web proxy policy to allow both IPv4 and IPv6 traffic through the proxy.

Select Apply.
Go to Network > Interfaces and select one or more interfaces for which to enable the explicit web proxy. Edit the interface. Under the Miscellaneous heading select Enable Explicit Web Proxy.

Enabling the explicit web proxy on an interface connected to the Internet is a security risk because anyone on the Internet who finds the proxy could use it to hide their source address. If you enable the proxy on such an interface make sure authentication is required to use the proxy.

Go to Policy & Objects > Addresses and select Create New to add a firewall address that matches the source address of packets to be accepted by the explicit proxy.

Category	Address
Name	Internal_subnet
Type	IP Range
Subnet / IP Range	10.31.101.1 – 10.31.101.255
Interface	any*

*The Interface must be set to Any.

You can also set the Type to URL Pattern (Explicit Proxy) to add a destination URL that is only used by the explicit proxy. For example, to create an explicit policy that only allows access to Fortinet.com:

Category	Address
Name	Fortinet-web-sites
Type	URL Pattern (Explicit Proxy)
URL Pattern	fortinet.com
Interface	any

Go to Policy & Objects > Proxy Policyand select Create New. Configure the policy as required to accept the traffic that you want to be allowed to use the explicit web proxy.
Set the Outgoing Interface parameter by selecting the field with the “+” next to the field label. Selecting the field will slide out a window from the right where you can select from the available interfaces. You can select one or more specific interfaces For more information on interfaces, check the Concepts section called Interfaces and Zones.
The Source of the policy must match the client’s source IP addresses. The interface of this firewall address must be set to any.
The Destination field should match the addresses of web sites that clients are connecting to. Usually the destination address would be all if proxying Internet web browsing. You could also specify a URL firewall address to limit the policy to allowing access to this URL.
Set the Schedule parameter by using the drop down menu to select a preconfigured schedule. The “+” icon next to the Search field is a shortcut for creating a new schedule object. For more information on addresses, check the Firewall Objects section called Firewall schedules
If Default Firewall Policy Action is set to Deny (under Network > Explicit Proxy), traffic sent to the explicit web proxy that is not accepted by a web-proxy policy is dropped. If Default Firewall Policy Action is set to Allow then all web-proxy sessions that don’t match with a security policy are allowed.

For example, the following security policy allows users on an internal network to access fortinet.com websites through the wan1 interface of a FortiGate unit.

Explicit Proxy Type	Web
Source Address	Internal_subnet
Outgoing Interface	wan1
Destination Address	Fortinet-web-sites
Schedule	always
Action	ACCEPT

Set the Disclaimer Options

You can configure a disclaimer for each Authentication Rule by enabling one of the options here. The

choices are:

Disable	No disclaimer (default setting)
By Domain	The disclaimer will be displayed on different domains. The explicit web proxy will check the referring header to mitigate the javascript/css/images/video/etc page.
By Policy	The disclaimer will be displayed if the HTTP request matches a different explicit firewall policy.
By User	The disclaimer will be displayed when a new user logs on.

If you chose a disclaimer option other than Disable, you will have the option to enable Customize Messages. If enabled, select the Edit Disclaimer Message button to customize the message to your needs. This can be done as text or as HTML. The default HTML version is there if you just want to make minor changes.

Enable Security Profiles as required. Once the profile type is toggled to enabled, you can use the drop down menu to select a specific profile. The available profile types are:
- AntiVirus l WebFilter l Application Control l IPS l DLP Sensor
- ICAP
- Web Application Firewall

Just like with a regular policy, as soon as any of the Security Profiles is enabled, the following fields, with their own drop down menus for specific profiles will appear:

Proxy Options l SSL/SSH Inspection

Select OK.

To enable the explicit web proxy – CLI:

Enter the following command to turn on the IPv4 and IPv6 explicit web proxy for HTTP and HTTPS traffic.

config web-proxy explicit set status enable set ipv6-status enable

end

You can also enter the following command to enable the web proxy for FTP sessions in a web browser.

config web-proxy explicit set ftp-over-http enable

end

The default explicit web proxy configuration has sec-default-action set to deny and requires

you to add a security policy to allow access to the explicit web proxy.

Enter the following command to enable the explicit web proxy for the internal interface.

config system interface edit internal set explicit-web-proxy enable

end

Use the following command to add a firewall address that matches the source address of users who connect to the explicit web proxy.

config firewall address edit Internal_subnet set type iprange set start-ip 10.31.101.1 set end-ip 10.31.101.255

end

The source address for a web-proxy security policy cannot be assigned to a FortiGate interface.

Optionally use the following command to add a destination URL that is only used by the explicit proxy. For example, to create an explicit policy that only allows access to Fortinet.com:

config firewall address edit Fortinet-web-sites set type url set url fortinet.com

end

Use the following command to add an explicit web proxy policy that allows all users on the internal subnet to use the explicit web proxy for connections through the wan1 interface to the Internet.

config firewall proxy-policy edit 0 set proxy explicit-web set dstintf wan1 set scraddr Internal_subnet

set dstaddr all set action accept set service webproxy set schedule always

end

Use the following command to add an explicit web proxy policy that allows authenticated users on the internal subnet to use the explicit web proxy for connections through the wan1 interface to the Internet.

config firewall proxy-policy edit 0 set proxy explicit-web set dstintf wan1 set scraddr Internal_subnet set dstaddr Fortinet-web-sites set action accept set service webproxy set schedule always set groups <User group>

end

Use the following command to change global web proxy settings, for example to set the maximum request length for the explicit web proxy to 10:

config web-proxy global set max-request-length 10

end

Determine whether or not to use Botnet feature.

The option scan-botnet-connections uses the following syntax:

config firewall proxy-policy edit <policy id> set scan-botnet-connections [disable|block|monitor] end

Where:

l disable means do not scan connections to botnet servers l block means block connection to botnet servers l monitor means log connections to botnet servers

Logging options in web proxy profiles

There is an option on what action to take regarding the authenticated user’s name in the header information for reading by upstream proxies and systems. This option can be used when a FortiGate is operating as an explicit proxy and authenticating users. The header is the x-authenticated-user and is used by the upstream proxy to ensure correct policy enforcement and to log the user’s activity.

The log-header-change option enables the logging of any header changes in the web-proxy profile, including changes to authenticated users or groups.

Syntax

config web-proxy profile edit <profile ID#> set header-x-authenticated-user {pass|add|remove} set header-x-authenticated-groups {pass|add|remove} set log-header-change {enable|disable} end

Option	Description
header-x-authenticateduser	Action to take on the HTTP x-authenticated-user header in forwarded requests: l pass – Forward the same HTTP header l add – Add the HTTP header l remove – Remove the HTTP header
Option	Description
header-x-authenticatedgroups	Action to take on the HTTP x-authenticated-groups header in forwarded requests: l pass – Forward the same HTTP header l add – Add the HTTP header l remove – Remove the HTTP header
log-header-change	enable or disable the logging of HTTP header changes

Policy matching based on referrer headers and query strings

Web proxy policies support creating web proxy addresses to match referrer headers and query strings.

Matching referrer headers

For example, to create a web proxy address to match the referrer header to block access to the following YouTube URL http://youtube.com/user/test321. The http request will have the following format:

GET /user/test321 HTTP/1.1

Host: www.youtube.com

User-Agent: curl/7.52.1

Accept: */*

Create the following web proxy addresses to match this page:

config firewall proxy-address edit youtube set type host-regex set host-regex “.*youtube.com”

next edit test321 set host “youtube” set path “/user/test321” set referrer enable

end

Then create two proxy policies, one that allows access to all traffic and a second one that blocks access to the page that matches the referrer header:

config firewall proxy-policy edit 1 set uuid 92273e4e-8c53-51e7-a7bd-f26e6e15fc98 set proxy explicit-web set dstintf “wan2” set srcaddr “all” set dstaddr “all” set service “webproxy-connect” set action accept set schedule “always” set utm-status enable set profile-protocol-options “test” set ssl-ssh-profile “test”

next edit 2 set uuid d35ad06a-8c53-51e7-8511-17200f682a4a set proxy explicit-web set dstintf “wan2” set srcaddr “all” set dstaddr “test321” set service “webproxy” set action accept set schedule “always” set utm-status enable set av-profile “default” set profile-protocol-options “test” set ssl-ssh-profile “test”

end

Matching query strings

To match the video with URL youtube.com/watch?v=XXXXXXXXX, (where XXXXXXXXX is an example YouTube query string) you need to match an HTTP request with the following format:

GET /user/watch?v=GLCHldlwQsg HTTP/1.1

Host: www.youtube.com

User-Agent: curl/7.52.1

Accept: */*

Create the following web proxy addresses to match this video or query string:

config firewall proxy-address edit “youtube” set uuid 4ad63880-971e-51e7-7b2e-c69423ac6314

set type host-regex set host-regex “.*youtube.com”

edit “query-string” set uuid 7687a8c0-9727-51e7-5063-05edda03abbf

set host “youtube” set path “/watch” set query “v=XXXXXXXXX”

end

Then create two proxy policies, one that allows access to all traffic and a second one that blocks access to the page that matches the query string

config firewall proxy-policy

edit 1

set uuid 92273e4e-8c53-51e7-a7bd-f26e6e15fc98 set proxy explicit-web set dstintf “wan2” set srcaddr “all” set dstaddr “all” set service “webproxy-connect” set action accept set schedule “always” set utm-status enable set profile-protocol-options “test” set ssl-ssh-profile “test”

next edit 2 set uuid d35ad06a-8c53-51e7-8511-17200f682a4a set proxy explicit-web set dstintf “wan2”

set srcaddr “all” set dstaddr “query-string” set service “webproxy” set action accept set schedule “always” set utm-status enable set av-profile “default” set profile-protocol-options “test” set ssl-ssh-profile “test”

next end

Multiple web proxy PAC files in one VDOM

Proxy auto-config (PAC) files automatically choose the appropriate proxy server for browsers and other user agents. Not every user in an organization has the same proxy server requirements. Supporting multiple PAC files provides granular control. To manage multiple PAC files, you use PAC policies.

This capability is available only when the FortiGate is in Proxy-based inspection mode.

If there is no matching PAC policy (by name), in the PAC policies, the global PAC file is used by default.

To enable Proxy mode:

GUI

Go to System > Settings.
In System Operation Settings, set the Inspection Mode to Proxy.

CLI

config system settings set inspection-mode proxy end

To configure a PAC policy

config web-proxy explicit set status enable

set pack-file-server-status enable config pac-policy edit <policy ID#> set srcaddr <name of IPv4 address object> set srcaddr6 <name of IPv6 address object> set dstaddr <name of address object> set pac-file-name <string> set pac-file-data “<PAC-file>”

end

Option	Description
srcaddr or srcaddr6	This address must conform to the following criteria: l a range, mask or wildcard mask type of address or address group l source type proxy-address or group It can be either IPv4 or IPv6.
dstaddr	This address must conform to the following criteria: l a range, mask or wildcard type of address or address group l it must be resolved as the FortiGate address
pacfilename	Name of the PAC file.
pacfiledata	Enter the contents of the PAC file enclosed in quotes. It is permissible to use the Return key when entering the contents. Place the closing quote at the end of the last line. If quotes are used within the content of the file, use the escape character \ before the quote. Example: \”

The pack-file-server-status setting must be set to enable in order for the config pac-policy command to work.

Web proxy concepts

Leave a reply

Web proxy concepts

These are concepts that apply to both Transparent and Explicit Proxy.

Proxy policy

Information on Proxy policy options can be found at Proxy option components on page 50

Configuration information can be found at Web proxy configuration on page 365

Proxy authentication

Beginning in FortiOS 5.6, authentication is separated from authorization for user based policy. You can add authentication to proxy policies to control access to the policy and to identify users and apply different UTM features to different users. The described authentication methodology works with Explicit Web Proxy and Transparent Proxy.

Authentication of web proxy sessions uses HTTP basic and digest authentication as described in RFC 2617 (HTTP Authentication: Basic and Digest Access Authentication) and prompts the user for credentials from the browser allowing individual users to be identified by their web browser instead of IP address. HTTP authentication allows the FortiGate unit to distinguish between multiple users accessing services from a shared IP address.

The methodology of adding authentication has changed from FortiOS version 5.4 and previous version. Splitpolicy has been obsoleted and instead of identity-based-policy, authentication is managed by authenticationscheme, setting and rule settings. These authentication settings are no longer configured with the individual policies. Authentication is set up in the contexts of:

config authentication scheme config authentication setting config authentication rule

The Authentication rule table defines how to identify user-ID. It uses the match factors:

l Protocol l Source Address

For one address and protocol, there is only one authentication rule. It is possible to configure multiple authentication methods for on one address. The client browser will chose one authentication method from the authentication methods list, but you can not control which authentication method will be chosen by the browser.

Matching

If a rule is matched, the authentication methods defined in the rule will be used to authenticate a user. The procedure works as the following:

Proxy authentication

If it is IP-based, look up active user list to see a user existed from the source IP. If found, return the user ID.
If no method is set, an anonymous user is created to associate to the source-IP. Return the anonymous user. It is another way to bypass user authentication for some source IPs.
Use authentication methods to authenticate the user.
- If no active method is defined, a failure will result to return an anonymous user. l Otherwise, a valid or guest user has to be identified to move on.
- Return the identified user ID.

Once a user is returned, the policy match resumes until a policy is matched or default policy will be used.

Processing policies for authentication

Authentication rules are checked once a User-ID is needed in order to resolve a match to a policy Use the following scenario as an example of the process.

There are 3 policies:

l policy1 does not have an associated user group l policy2 has an associated user group l policy3 does not have an associated user group

Step 1

If the traffic, based on protocol and source address matchespolicy 1, no user authentication is needed. The traffic is processed by policy1.

Step 2

If the traffic does not match policy 1, and any factor of policy 2 is not matched, continue to next policy.

If all the factors except the user-group of policy 2 are matched the authentication rule table is checked to get user-ID in the process in based on the procedure described earlier in Matching.

Step 3

When a user-ID is returned, whether it is a valid user or anonymous user, it is checked to see if the user is authorized by the user group associated with policy2. If yes, it is a match of policy2, and the traffic is processed by policy2. If not move on the next policy.

Step 4

For the purposes of the scenario, it will be assumed that the traffic either matches policy3 or that policy3 is the final policy that denies everything.

CLI syntax

Removals:

l “split-policy” from firewall explicit-proxy-policy.

The previous method to set up a split policy was: config firewall explicit-proxy-policy Proxy authentication

edit 1

set proxy web set identity-based enable set groups <User group> config identity-based-policy edit 1

set schedule “always” set utm-status enable set users “guest”

set profile-protocol-options “default” next

end

“auth relative” from firewall explicit-proxy-policy

The following attributes have been removed from firewall explicit-proxy-policy:

identity-based l ip-based l active-auth-method l sso-auth-method l require-tfa

Moves:

users and groups from firewall explicit-proxy-policy identity-based-policy to

config firewall proxy-policy edit 1 set groups <Group name> set users <User name> end Additions:

authentication scheme

config authentication scheme

ntlm – NTLM authentication. l basic – Basic HTTP authentication. l digest – Digest HTTP authentication. l form – Form-based HTTP authentication. l negotiate – Negotiate authentication. l fsso – FSSO authentication.
rsso – RADIUS Single Sign-On authentication. l none – No authentication.

authentication setting

config authentication setting set active-auth-scheme <string> set sso-auth-scheme <string> set captive-portal <string>

set captive-portal-port <integer value from 1 to 65535>

l active-auth-scheme – Active authentication method. l sso-auth-scheme – SSO authentication method. l captive-portal – Captive portal host name. l captive-portal-port – Captive portal port number.

authentication rule

config authentication rule edit <name of rule> set status [enable|disable] set protocol [http|ftp|socks] set srcaddr <name of address object> set srcaddr6 <name of address object> set ip-based [enable|disable] set active-auth-method <string> set sso-auth-method <string> set web-auth-cookie [enable|disable] set transaction-based [enable|disable] set comments

status – Enable/disable auth rule status. l protocol – set protocols to be matched l srcaddr /srcaddr6 – Source address name. [srcaddr or srcaddr6(web proxy only) must be set]. l ip-based – Enable/disable IP-based authentication. l active-auth-method – Active authentication method.
sso-auth-method – SSO authentication method (require ip-based enabled) l web-auth-cookie – Enable/disable Web authentication cookie.
transaction-based – Enable/disable transaction based authentication. l comments – Comment.

Configuring authentication in transparent proxy

You can enable transparent web-proxy feature to support authentication. Follow these steps

Configure a firewall policy
Enable a UTM profile in the firewall policy. Whenever there is a UTM item enabled, the feature enables the profile-protocol-options.
Go to the Proxy Options

l In the GUI this is Security Profiles > Proxy Options. l In the CLI it is config firewall profile-protocol-options.

Edit the profile used by the policy.

Enable HTTP in the profile.

Proxy addresses

In the GUI toggle on HTTP under Protocol Port Mapping In the CLI, the command sequence is:

config firewall profile-protocol-options edit <profile id> config http set status enable end

Fill out any other appropriate values.

Configure the proxy-policy, and set the value transparent-web for proxy option, others configuration are same as the explicit-web proxy

In the GUI, go to Policy & Objects > Proxy Policy. In the Proxy Type field choose Transparent Web .

In the CLI, the command sequence is:

config firewall proxy-policy edit <profile id> set proxy transparent-web end

Fill out any other appropriate values.

Setup the authentication rule and scheme

With this configuration, if a HTTP request passes through FortiGate without explicit web proxy being applied, the traffic will be redirected to WAD daemon after it matches the proxy with HTTP-policy enabled, then WAD will do the proxy-policy matching, and all of the proxy authentication method can be used for the request.

Proxy addresses

Information on Proxy addresses can be found at Proxy addresses on page 229

Proxy address group

In the same way that IPv4 and IPv6 addresses can only be grouped together, Proxy addresses can only be grouped with other Proxy addresses. Unlike the other address groups, the Proxy address groups are further divided into source address groups and destination address groups. To see the configuration steps go to Proxy address groups on page 231

Web proxy firewall services and service groups

Configure web proxy services by selecting Explicit Proxy when configuring a service. Web proxy services can be selected in a explicit web proxy policy when adding one from the CLI. If you add a policy from the web-based manager the service is set to the webproxy service. The webproxy service should be used in most cases, it matches with any traffic with any port number. However, if you have special requirements, such as using a custom protocol type or a reduced port range or need to add an IP/FQDN to an proxy service you can create custom explicit web proxy services.

Web proxy services are similar to standard firewall services. You can configure web proxy services to define one or more protocols and port numbers that are associated with each web proxy service. Web proxy services can also be grouped into web proxy service groups.

One way in which web proxy services differ from firewall services is the protocol type you can select. The following protocol types are available:

l ALL l CONNECT l FTP l HTTP l SOCKS-TCP l SOCKS-UDP

To add a web proxy service go to Policy & Objects > Servicesand select Create New. Set Service Type to Explicit Proxy and configure the service as required.

To add a web proxy service from the CLI enter:

config firewall service custom edit my-socks-service set explicit-proxy enable set category Web Proxy set protocol SOCKS-TCP set tcp-portrange 3450-3490

end

To add a web proxy service group go to Policy & Objects > Servicesand select Create New > Service Group. Set Type to Explicit Proxy and add web proxy services to the group as required.

To add a web proxy service group from the CLI enter:

config firewall service group edit web-group set explicit-proxy enable set member webproxy my-socks-service

end

Learn client IP

If there is another NATing device between the FortiGate and the Client (browser), this feature can be used to identify the real client in spite of the address translation. Knowing the actual client is imperative in cases where authorization is taking place.

The settings for the feature are in the CLI in the context of config web-proxy global

Once here, enable the feature with the command:

set learn-client-ip enable

Once the feature is enabled, the other settings become available.

learn-client-ip-from-header

This command has the following options:

true-client-ip		Support HTTP header True-Client-IP.
x-real-ip		Support HTTP header X-Real-IP.
x-forwarded-for		Support HTTP header X-Forwarded-For.

learn-client-ip-srcaddr/learn-client-ip-srcaddr6

The options for this setting are selected from the list of IPv4 address or IPv6 address objects.

Example

Below is a config example where the real client ip address will be used to match policy or fsso authentication after the learn-client-ip feature enabled.

The value of learn-client-ip-from-header option can be set to true-client-ip, x-real-ip or x-forwarded-for, but in this case it has been set to x-forward-for.

config web-proxy global set proxy-fqdn “default.fqdn” set webproxy-profile “default” set learn-client-ip enable

set learn-client-ip-from-header x-forwarded-for set learn-client-ip-srcaddr “all” end

config firewall proxy-policy edit 1 set proxy explicit-web set dstintf “mgmt1” set srcaddr “all” set dstaddr “all” set service “w” set action accept set schedule “always” set groups “fsso1” set utm-status enable set av-profile “default” set dlp-sensor “default” set profile-protocol-options “default” set ssl-ssh-profile “deep-inspection” end

config authentication rule edit “rule1” set srcaddr “all” set sso-auth-method “scheme1” end

config authentication scheme edit “scheme1” set method fsso

end

WCCP concepts

Leave a reply

WCCP concepts

The Web Cache Communication Protocol (WCCP) can be used to provide web caching with load balancing and fault tolerance. In a WCCP configuration, a WCCP server receives HTTP requests from user’s web browsers and redirects the requests to one or more WCCP clients. The clients either return cached content or request new content from the destination web servers before caching it and returning it to the server which in turn returns the content to the original requestor. If a WCCP configuration includes multiple WCCP clients, the WCCP server load balances traffic among the clients and can detect when a client fails and failover sessions to still operating clients. WCCP is described by the Web Cache Communication Protocol Internet draft.

The sessions that are cached by WCCP depend on the configuration of the WCCP clients. If the client is a FortiGate unit, you can configure the port numbers and protocol number of the sessions to be cached. For example, to cache HTTPS traffic on port 443 the WCCP client port must be set to 443 and protocol must be set to

If the WCCP client should also cache HTTPS traffic on port 993 the client ports option should include both port 443 and 993.

On a FortiGate unit, WCCP sessions are accepted by a security policy before being cached. If the security policy that accepts sessions that do not match the port and protocol settings in the WCCP clients the traffic is dropped.

WCCP is configured per-VDOM. A single VDOM can operate as a WCCP server or client (not both at the same time). FortiGate units are compatible with third-party WCCP clients and servers. If a FortiGate unit is operating as an Internet firewall for a private network, you can configure it to cache and serve some or all of the web traffic on the private network using WCCP by adding one or more WCCP clients, configuring WCCP server settings on the FortiGate unit and adding WCCP security policies that accept HTTP session from the private network.

FortiGate units support WCCPv1 and WCCPv2. A FortiGate unit in NAT/Route or transparent mode can operate as a WCCP server. To operate as a WCCP client a FortiGate unit must be in NAT/Route mode. FortiGate units communicate between WCCP servers and clients over UDP port 2048. This communication can be encapsulated in a GRE tunnel or just use layer 2 forwarding.

WCCP Cisco to FortiGate client using L2-forwarding tunneling

FortiGate supports the option of using Mask mode, in addition to Hash mode, when operating as a WCCP client using L2 forwarding. As a result, you can configure a WCCP FortiGate client to connect to a Cisco Nexxus, which doesn’t accept the Hash mode assignment method, using the Mask mode assignment method.

WCCP configuration

Leave a reply

WCCP configuration

WCCP configuration overview

To configure WCCP you must create a service group that includes WCCP servers and clients. WCCP servers intercept sessions to be cached (for example, sessions from users browsing the web from a private network). To intercept sessions to be cached the WCCP server must include a security policy that accepts sessions to be cached and WCCP must be enabled in this security policy.

The server must have an interface configured for WCCP communication with WCCP clients. That interface sends and receives encapsulated GRE traffic to and from WCCP clients. The server must also include a WCCP service group that includes a service ID and the addresses of the WCCP clients as well as other WCCP configuration options.

To use a FortiGate unit as a WCCP client, the FortiGate unit must be set to be a WCCP client (or cache engine). You must also configure an interface on the client for WCCP communication. The client sends and receives encapsulated GRE traffic to and from the WCCP server using this interface.

The client must also include a WCCP service group with a service ID that matches a service ID on the server. The client service group also includes the IP address of the servers in the service group and specifies the port numbers and protocol number of the sessions that will be cached on the client.

When the client receives sessions from the server on its WCCP interface, it either returns cached content over the WCCP interface or connects to the destination web servers using the appropriate interface depending on the client routing configuration. Content received from web servers is then cached by the client and returned to the WCCP server over the WCCP link. The server then returns the received content to the initial requesting user web browser.

Finally you may also need to configure routing on the server and client FortiGate units and additional security policies may have to be added to the server to accept sessions not cached by WCCP.

WCCP service groups, service numbers, service IDs and well known services

A FortiGate unit configured as a WCCP server or client can include multiple server or client configurations. Each of these configurations is called a WCCP service group. A service group consists of one or more WCCP servers (or routers) and one or more WCCP clients working together to cache a specific type of traffic. The service group configuration includes information about the type of traffic to be cached, the addresses of the WCCP clients and servers and other information about the service.

A service group is identified with a numeric WCCP service ID (or service number) in the range 0 to 255. All of the servers and clients in the same WCCP service group must have service group configurations with the same WCCP service ID.

The value of the service ID provides some information about the type of traffic to be cached by the service group. Service IDs in the range 0 to 50 are reserved for well known services. A well known service is any service that is defined by the WCCP standard as being well known. Since the service is well known, just the service ID is required to identify the traffic to be cached.

WCCP service groups, service numbers, service IDs and well known services

Even though the well known service ID range is 0 to 50, at this time only one well known service has been defined. Its service ID 0, which is used for caching HTTP (web) traffic.

So to configure WCCP to cache HTTP sessions you can add a service group to the WCCP router and WCCP clients with a service ID of 0. No other information about the type of traffic to cache needs to be added to the service group.

Since service IDs 1 to 50 are reserved for well know services and since these services are not defined yet, you should not add service groups with IDs in the range 1 to 50.

FortiOS does allow you to add service groups with IDs between 1 and 50. Since these service groups have not been assigned well known services, however, they will not cache any sessions. Service groups with IDs 51 to 255 allow you to set the port numbers and protocol number of the traffic to be cached. So you can use service groups with IDs 51 to 255 to cache different kinds of traffic based on port numbers and protocol number of the traffic. Service groups 1 to 50; however, do not allow you to set port numbers or protocol numbers so cannot be used to cache any traffic.

To cache traffic other than HTTP traffic you must add service groups with IDs in the range 51 to 255. These service group configurations must include the port numbers and protocol number of the traffic to be cached. It is the port and protocol number configuration in the service group that determines what traffic will be cached by WCCP.

Example WCCP server and client configuration for caching HTTP sessions (service ID = 0)

Enter the following command to add a WCCP service group to a WCCP server that caches HTTP sessions. The IP address of the server is 10.31.101.100 and the WCCP clients are on the 10.31.101.0 subnet. The service ID of this service group is 0.

config system wccp edit 0 set router-id 10.31.101.100

set server-list 10.31.101.0 255.255.255.0

end

Enter the following commands to configure a FortiGate unit to operate as a WCCP client and add a service group that configures the client to cache HTTP sessions. The IP address of the server is 10.31.101.100 and IP address of this WCCP clients is 10.31.101.1 subnet. The service ID of this service group is 0.

config system settings set wccp-cache-engine enable

end

config system wccp edit 0 set cache-id 10.31.101.1 set router-list 10.31.101.100 end

WCCP service groups, service numbers, service IDs and well known services

You cannot enter the wccp-cache-engine enable command if you have already added a WCCP service group. When you enter this command an interface named w.<vdom_name> is added to the FortiGate configuration (for example w.root). All traffic redirected from a WCCP router is considered to be received at this interface of the FortiGate unit operating as a WCCP client. A default route to this interface with lowest priority is added.

Example WCCP server and client configuration for caching HTTPS sessions

Enter the following command to add a service group to a WCCP server that caches HTTPS content on port 443 and protocol 6. The IP address of the server is 10.31.101.100 and the WCCP clients are on the 10.31.101.0 subnet. The service ID of this service group is 80.

config system settings set wccp-cache-engine enable

end

config system wccp edit 80 set router-id 10.31.101.100 set server-list 10.31.101.0 255.255.255.0

set ports 443 set protocol 6

end

Enter the following commands to configure a FortiGate unit to operate as a WCCP client and add a service group that configures client to cache HTTPS sessions on port 443 and protocol 6. The IP address of the server is 10.31.101.100 and IP address of this WCCP clients is 10.31.101.1 subnet. The service ID of this service group must be 80 to match the service ID added to the server.

config system settings set wccp-cache-engine enable

end

config system wccp edit 80 set cache-id 10.31.101.1 set router-list 10.31.101.100

set ports 443 set protocol 6

end

Example WCCP server and client configuration for caching HTTP and HTTPS sessions

You could do this by configuring two WCCP service groups as described in the previous examples. Or you could use the following commands to configure one service group for both types of traffic. The example also caches HTTP sessions on port 8080.

Enter the following command to add a service group to a WCCP server that caches HTTP sessions on ports 80 and 8080 and HTTPS sessions on port 443. Both of these protocols use protocol number 6. The IP address of the server is 10.31.101.100 and the WCCP clients are on the 10.31.101.0 subnet. The service ID of this service group is 90.

config system wccp edit 90

WCCP service groups, service numbers, service IDs and well known services

set router-id 10.31.101.100

set server-list 10.31.101.0 255.255.255.0

set ports 443 80 8080 set protocol 6

end

Enter the following commands to configure a FortiGate unit to operate as a WCCP client and add a service group that configures client to cache HTTP sessions on port 80 and 8080 and HTTPS sessions on port 443. The IP address of the server is 10.31.101.100 and IP address of this WCCP clients is 10.31.101.1 subnet. The service ID of this service group must be 90 to match the service ID added to the server.

config system settings set wccp-cache-engine enable

end

config system wccp edit 90 set cache-id 10.31.101.1 set router-list 10.31.101.100 set ports 443 80 8080 set protocol 6

end

Other WCCP service group options

In addition to using WCCP service groups to define the types of traffic to be cached by WCCP the following options are available for servers and clients.

Server configuration options

The server configuration must include the router-id, which is the WCCP server IP address. This is the IP address of the interface that the server uses to communicate with WCCP clients.

The group-address is used for multicast WCCP configurations to specify the multicast addresses of the clients.

The server-list defines the IP addresses of the WCCP clients that the server can connect to. Often the server list can be the address of the subnet that contains the WCCP clients.

The authentication option enables or disables authentication for the WCCP service group. Authentication must be enabled on all servers and clients in a service group and members of the group must have the same password.

The forward-method option specifies the protocol used for communication between the server and clients. The default forwarding method is GRE encapsulation. If required by your network you can also select to use unencapsulated layer-2 packets instead of GRE or select any to allow both. The return-method allows you to specify the communication method from the client to the server. Both GRE and layer-2 are supported.

The assignment-method determines how the server load balances sessions to the clients if there are multiple clients. Load balancing can be done using hashing or masking.

Client configuration options

The client configuration includes the cache-id which is the IP address of the FortiGate interface of the client that communicates with WCCP server. The router-list option is the list of IP addresses of the WCCP servers in the WCCP service group.

Example caching HTTP sessions on port 80 using WCCP

The ports option lists the port numbers of the sessions to be cached by the client and the protocol sets the protocol number of the sessions to be cached. For TCP sessions the protocol is 6.

The service-type option can be auto, dynamic or standard. Usually you would not change this setting.

The client configuration also includes options to influence load balancing including the primary-hash, priority, assignment-weight and assignment-bucket-format.

Example caching HTTP sessions on port 80 using WCCP

In this example configuration (shown below), a FortiGate unit with host name WCCP_srv is operating as an Internet firewall for a private network is also configured as a WCCP server. The port1 interface of WCCP_srv is connected to the Internet and the port2 interface is connected to the internal network.

All HTTP traffic on port 80 that is received at the port2 interface of WCCP_srv is accepted by a port2 to port1 security policy with WCCP enabled. All other traffic received at the port2 interface is allowed to connect to the Internet by adding a general port2 to port1 security policy below the HTTP on port 80 security policy.

A WCCP service group is added to WCCP_srv with a service ID of 0 for caching HTTP traffic on port 80. The port5 interface of WCCP_srv is configured for WCCP communication.

A second FortiGate unit with host name WCCP_client is operating as a WCCP client. The port1 interface of WCCP_client is connected to port5 of WCCP_srv and is configured for WCCP communication.

WCCP_client is configured to cache HTTP traffic because it also has a WCCP service group with a service ID of

WCCP_client connects to the Internet through WCCP_srv. To allow this, a port5 to port1 security policy is added to WCCP_srv.

FortiGate WCCP server and client configuration

Configuring the WCCP server (WCCP_srv)

Use the following steps to configure WCCP_srv as the WCCP server for the example network. The example steps only describe the WCCP-related configuration.

Example caching HTTP sessions on port 80 using WCCP

To configure WCCP_srv as a WCCP server

Add a port2 to port1 security policy that accepts HTTP traffic on port 80 and is configured for WCCP:

config firewall policy edit 0 set srtintf port2 set dstintf port1 set srcaddr all set dstaddr all set action accept set schedule always set service HTTP set wccp enable set nat enable

end

Add another port2 to port1 security policy to allow all other traffic to connect to the Internet.

config firewall policy edit 0 set srtintf port2 set dstintf port1 set srcaddr all set dstaddr all set action accept set schedule always set service ANY set nat enable

end

Move this policy below the WCCP policy in the port2 to port1 policy list.
Enable WCCP on the port5 interface.

config system interface edit port5 set wccp enable

end

Add a WCCP service group with service ID 0.

config system wccp edit 0 set router-id 10.51.101.100 set server-list 10.51.101.0 255.255.255.0

end

Add a firewall address and security policy to allow the WCCP_client to connect to the internet.

config firewall address edit WCCP_client_addr set subnet 10.51.101.10

end

config firewall policy edit 0 set srtintf port5 set dstintf port1 set srcaddr WCCP_client_addr

set dstaddr all set action accept

Example caching HTTP sessions on port 80 and HTTPS sessions on port 443 using WCCP

set schedule always set service ANY set nat enable end

Configuring the WCCP client (WCCP_client)

Use the following steps to configure WCCP_client as the WCCP client for the example network. The example steps only describe the WCCP-related configuration.

To configure WCCP_client as a WCCP client

Configure WCCP_client to operate as a WCCP client.

config system settings set wccp-cache-engine enable

end

Enable WCCP on the port1 interface.

config system interface edit port1 set wccp enable

end

Add a WCCP service group with service ID 0.

config system wccp edit 0 set cache-id 10.51.101.10 set router-list 10.51.101.100

end

Example caching HTTP sessions on port 80 and HTTPS sessions on port 443 using WCCP

This example configuration is the same as that described in Example caching HTTP sessions on port 80 and

HTTPS sessions on port 443 using WCCP on page 351 except that WCCP now also cached HTTPS traffic on port 443. To cache HTTP and HTTPS traffic the WCCP service group must have a service ID in the range 51 to 255 and you must specify port 80 and 443 and protocol 6 in the service group configuration of the WCCP client.

Also the security policy on the WCCP_srv that accepts sessions from the internal network to be cached must accept HTTP and HTTPS sessions.

Example caching HTTP sessions on port 80 and HTTPS sessions on port 443 using WCCP

FortiGate WCCP server and client configuration

Configuring the WCCP server (WCCP_srv)

Use the following steps to configure WCCP_srv as the WCCP server for the example network. The example steps only describe the WCCP-related configuration.

To configure WCCP_srv as a WCCP server

Add a port2 to port1 security policy that accepts HTTP traffic on port 80 and HTTPS traffic on port 443 and is configured for WCCP:

config firewall policy edit 0 set srtintf port2 set dstintf port1 set srcaddr all set dstaddr all set action accept set schedule always set service HTTP HTTPS set wccp enable set nat enable

end

Add another port2 to port1 security policy to allow all other traffic to connect to the Internet. config firewall policy edit 0 set srtintf port2 set dstintf port1 set srcaddr all set dstaddr all set action accept set schedule always set service ANY

set nat enable end

Example caching HTTP sessions on port 80 and HTTPS sessions on port 443 using WCCP

Move this policy below the WCCP policy in the port2 to port1 policy list.
Enable WCCP on the port5 interface.

config system interface edit port5 set wccp enable

end

Add a WCCP service group with service ID 90 (can be any number between 51 and 255).

config system wccp edit 90 set router-id 10.51.101.100 set server-list 10.51.101.0 255.255.255.0

end

Add a firewall address and security policy to allow the WCCP_client to connect to the internet.

config firewall address edit WCCP_client_addr set subnet 10.51.101.10

end

config firewall policy edit 0 set srtintf port5 set dstintf port1 set srcaddr WCCP_client_addr

set dstaddr all set action accept set schedule always set service ANY set nat enable end

Configuring the WCCP client (WCCP_client)

Use the following steps to configure WCCP_client as the WCCP client for the example network. The example steps only describe the WCCP-related configuration.

To configure WCCP_client as a WCCP client

Configure WCCP_client to operate as a WCCP client.

config system settings set wccp-cache-engine enable

end

Enable WCCP on the port1 interface.

config system interface edit port1

WCCP packet flow

set wccp enable

end

Add a WCCP service group with service ID 90. This service group also specifies to cache sessions on ports 80 and 443 (for HTTP and HTTPS) and protocol number 6.

config system wccp edit 90 set cache-id 10.51.101.10 set router-list 10.51.101.100

ports 80 443 set protocol 6 end

WCCP packet flow

The following packet flow sequence assumes you have configured a FortiGate unit to be a WCCP server and one or more FortiGate units to be WCCP clients.

A user’s web browser sends a request for web content.
The FortiGate unit configured as a WCCP server includes a security policy that intercepts the request and forwards it to a WCCP client.

The security policy can apply UTM features to traffic accepted by the policy.

The WCCP client receives the WCCP session.
The client either returns requested content to the WCCP server if it is already cached, or connects to the destination web server, receives and caches the content and then returns it to the WCCP server.
The WCCP server returns the requested content to the user’s web browser.
The WCCP router returns the request to the client web browser.

The client we browser is not aware that all this is taking place and does not have to be configured to use a web proxy.

Configuring the forward and return methods and adding authentication

The WCCP forwarding method determines how intercepted traffic is transmitted from the WCCP router to the WCCP cache engine. There are two different forwarding methods:

GRE forwarding (the default) encapsulates the intercepted packet in an IP GRE header with a source IP address of the WCCP router and a destination IP address of the target WCCP cache engine. The results is a tunnel that allows the WCCP router to be multiple hops away from the WCCP cache server.
L2 forwarding rewrites the destination MAC address of the intercepted packet to match the MAC address of the target WCCP cache engine. L2 forwarding requires that the WCCP router is Layer 2 adjacent to the WCCP client.

You can use the following command on a FortiGate unit configured as a WCCP router to change the forward and return methods to L2:

config system wccp edit 1 set forward-method L2

WCCP messages

set return-method L2

end

You can also set the forward and return methods to any in order to match the cache server configuration.

By default the WCCP communication between the router and cache servers is unencrypted. If you are concerned about attackers sniffing the information in the WCCP stream you can use the following command to enable hashbased authentication of the WCCP traffic. You must enable authentication on the router and the cache engines and all must have the same password.

config system wccp edit 1 set authentication enable set password <password>

end

WCCP messages

When the WCCP service is active on a web cache server it periodically sends a WCCP HERE I AM broadcast or unicast message to the FortiGate unit operating as a WCCP router. This message contains the following information:

Web cache identity (the IP address of the web cache server). l Service info (the service group to join).

If the information received in the previous message matches what is expected, the FortiGate unit replies with a WCCP I SEE YOU message that contains the following details:

Router identity (the FortiGate unit’s IP address. l Sent to IP (the web cache IP addresses to which the packets are addressed)

When both ends receive these two messages the connection is established, the service group is formed and the designated web cache is elected.

Troubleshooting WCCP

Two types of debug commands are available for debugging or troubleshooting a WCCP connection between a FortiGate unit operating as a WCCP router and its WCCP cache engines.

Real time debugging

The following commands can capture live WCCP messages:

diag debug en diag debug application wccpd <debug level>

Application debugging

The following commands display information about WCCP operations:

get test wccpd <integer> diag test application wccpd <integer> Where <integer> is a value between 1 and 6:

Troubleshooting WCCP

Display WCCP stats
Display WCCP config
Display WCCP cache servers
Display WCCP services
Display WCCP assignment
Display WCCP cache status

Enter the following command to view debugging output:

diag test application wccpd 3

Sample output from a successful WCCP connection:

service-0 in vdom-root: num=1, usable=1 cache server ID: len=44, addr=172.16.78.8, weight=4135, status=0 rcv_id=6547, usable=1, fm=1, nq=0, dev=3(k3),

to=192.168.11.55 ch_no=0, num_router=1:

192.168.11.55

Sample output from the same command from an unsuccessful WCCP connection (because of a service group password mismatch):

service-0 in vdom-root: num=0, usable=0 diag debug application wccpd -1 Sample output: wccp_on_recv()-98: vdom-root recv: num=160, dev=3(3),

172.16.78.8->192.168.11.55

wccp2_receive_pkt()-1124: len=160, type=10, ver=0200, length=152 wccp2_receive_pkt()-1150: found component:t=0, len=20 wccp2_receive_pkt()-1150: found component:t=1, len=24 wccp2_receive_pkt()-1150: found component:t=3, len=44 wccp2_receive_pkt()-1150: found component:t=5, len=20 wccp2_receive_pkt()-1150: found component:t=8, len=24 wccp2_check_security_info()-326: MD5 check failed

Web cache configuration

Leave a reply

Web cache configuration

Forwarding URLs to forwarding servers and exempting web sites from web caching

You can go to Network > Explicit Proxy and use the URL match list to forward URL patterns to forwarding servers and create a list of URLs that are exempt from web caching.

Forwarding URLs and URL patterns to forwarding servers

As part of configuring the explicit web proxy you can configure proxy chaining by adding web proxy forwarding servers. See Proxy chaining (web proxy forwarding servers) .

You can then use the URL match list to always forward explicit web proxy traffic destined for configured URLs or URL patterns to one of these forwarding servers. For example, you might want to forward all traffic for a specific country to a proxy server located in that country.

To forward traffic destined for a URL to a forwarding server that you have already added, go to Network > Explicit Proxy and select Create New. Add a name for the URL match entry and enter the URL or URL pattern. You can use wildcards such as * and ? and you can use a numeric IP address. Select Forward to Server and select a web proxy forwarding server from the list.

You can also exempt the URL or URL pattern from web caching.

Use the following command to forward all .ca traffic to a proxy server and all .com traffic to another proxy server.

config web-proxy url-match edit “com” set forward-server “server-commercial” set url-pattern “com”

next edit “ca” set forward-server “server-canada” set url-pattern “ca”

edit “www.google.ca” set cache-exemption enable set url-pattern “www.google.ca”

end

Exempting web sites from web caching

You may want to exempt some URLs from web caching for a number of reasons. For example, if your users access websites that are not compatible with FortiGate web caching you can add the URLs of these web sites to the web caching exempt list. You can add URLs and numeric IP addresses to the web cache exempt list.

You can also add URLs to the web cache exempt list by going to Network > Explicit Proxy, going to the URL Match List

Web cache configuration Forwarding URLs to forwarding servers and exempting web sites from web caching

and selecting Create New. Add a URL pattern to be exempt and select Exempt from Cache.

You can also add URLs and addresses to be exempt from caching using the CLI. Enter the following command to add www.example.com to the web cache exempt list:

config web-proxy url-match set cache-exemption enable set url-pattern www.example.com

end

Exempting specific files from caching

You can exempt files from being cached, so long as you specify its full URL. Enter the following command to add the URL, with the file extension (in this example, .exe), to the web cache exempt list:

config web-proxy url-match edit “exe” set url-pattern “iavs9x.u.avast.com/custom/iavs9x/20160613t1237z/avast_free_ antivirus_setup_online.exe”

set cache-exemption enable

next end

Monitoring web caching performance

The web cache monitor shows the percentage of web cache requests that retrieved content from the cache (hits) and the percentage that did not receive content from the cache (misses). A higher the number of hits usually indicates that the web cache is being more effective at reducing WAN traffic.

The web cache monitor also shows a graph of web traffic on the WAN and LAN. A lower WAN line on the graph indicates the web cache is reducing traffic on the WAN. The web cache monitor also displays the total number of web requests processed by the web cache.

To view the web cache monitor, go to Monitor > Cache Monitor.

Web cache monitor

Example web caching of HTTP and HTTPS Internet content for users on an internal network

This example describes how to configure web caching of HTTP and HTTPS for users on a private network connecting to the Internet.

Network topology and assumptions

This example includes a client network with subnet address 10.31.101.0 connecting to web servers on the

Internet. All of the users on the private network access the Internet though a single general security policy on the FortiGate unit that accepts all sessions connecting to the Internet. Web caching for HTTP and HTTPS traffic is added to this security policy.

Since users on the private network have unrestricted access to the Internet and can be accessing many web servers the webcache-https is set to any and users may see error messages on their web browsers when accessing HTTPS content.

The GUI is less versatile than the CLI so the example instructions for the GUI give settings for one port for each protocol, while the CLI example shows how to use multiple ports.

Web cache configuration Example web caching of HTTP and HTTPS Internet content for users on an internal network

The example also describes how to configure the security policy to cache HTTP traffic on port 80 and 8080 in the CLI, by adding a proxy options profile that looks for HTTP traffic on TCP ports 80 and 8080. The example also describes how to configure the security policy to cache HTTPS traffic on port 443 and 8443 using the same proxy options profile.

Example web caching topology

General configuration steps

This section breaks down the configuration for this example into smaller procedures. For best results, follow the procedures in the order given:

Add HTTP web caching to the security policy that all users on the private network use to connect to the Internet.
Add HTTPS web caching.
Add a protocol options profile to look for HTTP traffic on ports 80 and 8080 and HTTPS traffic on ports 443 and 8443 and add this protocol options profile to the security policy.

If you perform any additional actions between procedures, your configuration may have different results.

Configuration steps – web-based manager

Use the following steps to configure the example configuration from the FortiGate web-based manager.

To add HTTP web caching to a security policy

Go to Policy & Objects > IPv4 Policyand add a security policy that allows all users on the internal network to access the Internet.

Incoming Interface	Internal
Outgoing Interface	wan1
Source	all
Destination	all
Schedule	always
Service	ALL
Action	ACCEPT

Toggle NAT to enabled, and select Use Outgoing Interface Address.
Turn on Web cache.
Select OK.

Example web caching of HTTP and HTTPS Internet content for users on an internal network Web cache configuration

To add HTTPS web caching

From the CLI enter the following command to add HTTPS web caching to the policy.

Assume the index number of the policy is 5.

config firewall policy edit 5 set webcache-https any

end

To cache HTTP traffic on port 80 and HTTPS on 8443

Go to Network > Explicit Proxy and edit the Explicit Proxy options profile. 2. Under Explicit Web Proxy , l For the HTTP port, enter 80.

l For HTTPS port, select Specify and enter 8443 in the field.

Click on Apply.

Configuration steps – CLI

Use the following steps to configure the example configuration from the FortiGate CLI.

To add HTTP and HTTPS web caching to a security policy

Enter the following command to add a security policy that allows all users on the internal network to access the Internet and that includes web caching of HTTP and HTTPS traffic.

config firewall policy edit 0 set srcintf internal set srcaddr all set dstintf wan1 set distinf all set schedule always set service ANY set action accept set nat enable set webcache enable set webcache-https any

end

To cache HTTP traffic on port 80 and 8080 and HTTPS traffic on ports 443 and 8443

Enter the following command to edit the default proxy options profile to configure it to look for HTTP traffic on ports 80 and 8080:

config firewall profile-protocol-options edit default config http set status enable set ports 80 8080

Web cache Example reverse proxy web caching and SSL offloading for an Internet web server using a static configuration one-to-one virtual IP

end

Enter the following command to edit the certification-inspection SSL SSH options profile to configure it to look for HTTPS traffic on ports 443 and 8443:

config firewall ssl-ssh-profile edit certificate-inspection config https set status certificate-inspection

set ports 443 8443 end

Enter the following command to add the default proxy options profile and the certificate-inspection SSL SSH profile to the firewall policy.

config firewall policy edit 5 set utm-status enable set profile-protocol-options default set ssl-ssh-profile certificate-inspection end

Example reverse proxy web caching and SSL offloading for an Internet web server using a static one-to-one virtual IP

This section describes configuring SSL offloading for a reverse proxy web caching configuration using a static one-to-one firewall virtual IP (VIP). While the static one-to-one configuration described in this example is valid, its also common to change the destination port of the unencrypted HTTPS traffic to a commonly used HTTP port such as 8080 using a port forwarding virtual IP.

Network topology and assumptions

In this configuration, clients on the Internet use HTTP and HTTPS to browse to a web server that is behind a FortiGate unit. A policy added to the FortiGate unit forwards the HTTP traffic to the web server. The policy also offloads HTTPS decryption and encryption from the web server so the web server only sees HTTP traffic.

The FortiGate unit also caches HTTP and HTTPS pages from the web server so when users access cached pages the web server does not see the traffic. Replies to HTTPS sessions are encrypted by the FortiGate unit before returning to the clients.

In this configuration, the FortiGate unit is operating as a web cache in reverse proxy mode. Reverse proxy caches can be placed directly in front of a web server. Web caching on the FortiGate unit reduces the number of requests that the web server must handle, therefore leaving it free to process new requests that it has not serviced before.

Using a reverse proxy configuration:

l avoids the capital expense of additional web servers by increasing the capacity of existing servers l serves more requests for static content from web servers l serves more requests for dynamic content from web servers l reduces operating expenses including the cost of bandwidth required to serve content l accelerates the response time of web servers and of page download times to end users.

Example reverse proxy web caching and SSL offloading for an Internet web server using a static one-to-one virtual IP

Web cache configuration

When planning a reverse proxy implementation, the web server’s content should be written so that it is “cache aware” to take full advantage of the reverse proxy cache.

In reverse proxy mode, the FortiGate unit functions more like a web server for clients on the Internet. Replicated content is delivered from the proxy cache to the external client without exposing the web server or the private network residing safely behind the firewall.

In this example, the site URL translates to IP address 192.168.10.1, which is the port2 IP address of the FortiGate unit. The port2 interface is connected to the Internet.

This example assumes that all HTTP traffic uses port 80 and all HTTPS traffic uses port 443.

The FortiGate unit includes the web server CA and an SSL server configuration for IP address 172.10.20.30 and port to 443. The name of the file containing the CA is Rev_Proxy_Cert_1.crt.

The destination address of incoming HTTP and HTTPS sessions is translated to the IP address of the web server using a static one-to-one virtual IP that performs destination address translation (DNAT) for the HTTP packets. The DNAT translates the destination address of the packets from 192.168.10.1 to 172.10.20.30 but does not change the destination port number.

When the SSL server on the FortiGate unit decrypts the HTTPS packets their destination port is changed to port 80.

Reverse proxy web caching and SSL offloading for an Internet web server using static one-to-one virtual IPs

General configuration steps

This section breaks down the configuration for this example into smaller procedures. For best results, follow the procedures in the order given:

Configure the FortiGate unit as a reverse proxy web cache server.
Configure the FortiGate unit for SSL offloading of HTTPS traffic.
Add an SSL server to offload SSL encryption and decryption for the web server.

Also note that if you perform any additional actions between procedures, your configuration may have different results.

Web cache

configuration

Example reverse proxy web caching and SSL offloading for an Internet web server using a static one-to-one virtual IP

Configuration steps – web-based manager

To configure the FortiGate unit as a reverse proxy web cache server

Go to Policy & Objects > Virtual IPsand select Create New to add a static NAT virtual IP that translates destination IP addresses from 192.168.10.1 to 172.10.20.30 (and does not translate destination ports):

VIP Type	IPv4
Name	Reverse_proxy_VIP
Interface	port2
Type	Static NAT
Optional Filters	Do not select.
External IP Address/Range	192.168.10.1
Mapped IP Address/Range	172.10.20.30
Port Forwarding	Do not select.

Select OK.
Go to Policy & Objects > IPv4 Policy and select Create New to add a port2 to port1 security policy that accepts HTTP and HTTPS traffic from the Internet.

Do not select security profiles. Set the destination address to the virtual IP. You do not have to enable NAT.

Incoming Interface	port2
Outgoing Interface	port1
Source	all
Destination	Reverse_proxy_VIP
Schedule	always
Service	HTTP HTTPS
Action	ACCEPT

Turn on Web Cache.
Select OK.
From the CLI enter the following command to add HTTPS web caching to the security policy

Assume the index number of the policy is 5.

config firewall policy edit 5 set webcache-https ssl-server

Example reverse proxy web caching and SSL offloading for an Internet web server using a static Web cache one-to-one virtual IP configuration

end

To configure the FortiGate unit to offload SSL encryption and cache HTTPS content

Go to System > Certificates and select Import to import the web server’s CA.

For Type, select Local Certificate. Select the Browse button to locate the file (example file name: Rev_Proxy_

Cert_1.crt).

The certificate key size must be 1024 or 2048 bits. 4096-bit keys are not supported.

Select OK to import the certificate.
From the CLI, enter the following command to add the SSL server and to add the server’s certificate to the SSL server.

The SSL server ip must match the destination address of the SSL traffic after being translated by the virtual IP (172.10.20.30) and the SSL server port must match the destination port of the SSL traffic (443). The SSL server operates in half mode since it performs a single-step conversion (HTTPS to HTTP or HTTP to HTTPS).

config firewall ssl-server edit rev_proxy_server set ip 172.10.20.30 set port 443 set ssl-mode half set ssl-cert Rev_Proxy_Cert_1 end

Configuration steps – CLI

To configure the FortiGate unit as a reverse proxy web cache server

Enter the following command to add a static NAT virtual IP that translates destination IP addresses from 192.168.10.1 to 172.10.20.30 (and does not translate destination ports):

config firewall vip edit Reverse_proxy_VIP set extintf port2 set type static-nat set extip 192.168.10.1 set mappedip 172.10.20.30

end

Enter the following command to add a port2 to port1 security policy that accepts HTTP and HTTPS traffic from the Internet. Enable web caching and HTTPS web caching.

Do not select security profiles. Set the destination address to the virtual IP. You do not have to enable NAT.

config firewall policy edit 0 set srcintf port2 set srcaddr all set dstintf port1 set dstaddr Reverse_proxy_VIP set schedule always set service HTTP HTTPS set action accept

set webcache enable set webcache-https ssl-server

end

To add an SSL server to offload SSL encryption and decryption for the web server

Place a copy of the web server’s CA (file name Rev_Proxy_Cert_1.crt) in the root folder of a TFTP server.
Enter the following command to import the web server’s CA from a TFTP server. The IP address of the TFTP server is 10.31.101.30:

execute vpn certificate local import tftp Rev_Proxy_Cert_1.crt 10.31.101.30 The certificate key size must be 1024 or 2048 bits. 4096-bit keys are not supported.

From the CLI, enter the following command to add the SSL server.

config firewall ssl-server edit rev_proxy_server set ip 172.10.20.30 set port 443 set ssl-mode half set ssl-cert Rev_Proxy_Cert_1

end

Configure other ssl-server settings that you may require for your configuration.

Using a FortiCache as a cache service

Some FortiGate devices don’t have sufficient memory or disk space to run a cache service. This feature allows a FortiGate to connect to a FortiCache that has a higher cache capability than most FortiGates.

Syntax:

config wanopt remote-storage set status {enable|disable} set local-cache-id <name ID for connection> set remote-cache-id <ID of the remote device> set remote-cache-ip <IP address of the remote device> end

Option	Description
status	Enable or disable whether the FortiGate uses a remote caching device as web-cache storage. If disabled, uses local disk(s) as web storage.
localcache-id	ID that this device uses to connect to the remote caching device

Option	Description
remotecache-id	ID of the remote caching device that this FortiGate connects to
remotecache-ip	IP address of the remote caching device that this FortiGate connects to.

Web cache concepts

Leave a reply

Web cache concepts

FortiGate web caching is a form of object caching that accelerates web applications and web servers by reducing bandwidth usage, server load, and perceived latency. Web caching supports caching of HTTP 1.0 and HTTP 1.1 web sites. See RFC 2616 for information about web caching for HTTP 1.1.

Web caching supports caching of Flash content over HTTP but does not cache audio and video streams including Flash videos and streaming content that use native streaming protocols such as RTMP.

The first time a file is received by web caching it is cached in the format it is received in, whether it be compressed or uncompressed. When the same file is requested by a client but in a different compression format, the cached file is converted to the new compressed format before being sent to the client.

There are three significant advantages to using web caching to improve HTTP and WAN performance:

reduced bandwidth consumption because fewer requests and responses go over the WAN or Internet. l reduced web server load because there are fewer requests for web servers to handle.
reduced latency because responses for cached requests are available from a local FortiGate unit instead of from across the WAN or Internet.

You can use web caching to cache any web traffic that passes through the FortiGate unit, including web pages from web servers on a LAN, WAN or on the Internet. You apply web caching by enabling the web caching option in any security policy. When enabled in a security policy, web caching is applied to all HTTP sessions accepted by the security policy. If the security policy is an explicit web proxy security policy, the FortiGate unit caches explicit web proxy sessions.

Turning on web caching for HTTP and HTTPS traffic

Web caching can be applied to any HTTP or HTTPS traffic by enabling web caching in a security policy that accepts the traffic. This includes IPv4, IPv6, WAN optimization and explicit web proxy traffic. Web caching caches all HTTP traffic accepted by a policy on TCP port 80.

You can add web caching to a policy to:

Cache Internet HTTP traffic for users on an internal network to reduce Internet bandwidth use. Do this by selecting the web cache option for security policies that allow users on the internal network to browse web sites on the

Internet.

Reduce the load on a public facing web server by caching objects on the FortiGate unit. This is a reverse proxy with web caching configuration. Do this by selecting the web cache option for a security policy that allows users on the Internet to connect to the web server.
Cache outgoing explicit web proxy traffic when the explicit proxy is used to proxy users in an internal network who are connecting to the web servers on the Internet. Do this by selecting the web cache option for explicit web proxy security policies that allow users on the internal network to browse web sites on the Internet.
Combine web caching with WAN optimization. You can enable web caching in any WAN optimization security policy. This includes manual, active, and passive WAN optimization policies and WAN optimization tunnel policies.

Turning on web caching for HTTPS traffic

You can enable web caching on both the client-side and the server-side FortiGate units or on just one or the other. For optimum performance you can enable web caching on both the client-side and server-side FortiGate units. In this way only uncached content is transmitted through the WAN optimization tunnel. All cached content is access locally by clients from the client side FortiGate unit.

One important use for web caching is to cache software updates (for example, Windows Updates or iOS updates. When updates occur a large number of users may all be trying to download these updates at the same time. Caching these updates will be a major performance improvement and also have a potentially large impact on reducing Internet bandwidth use. You may want to adjust the maximum cache object size to make sure these updates are cached. See Turning on web caching for HTTP and HTTPS traffic on page 325.

Turning on web caching for HTTPS traffic

Web caching can also cache the content of HTTPS traffic on TCP port 443. With HTTPS web caching, the FortiGate unit receives the HTTPS traffic on behalf of the client, opens up the encrypted traffic and extracts content to be cached. Then FortiGate unit re-encrypts the traffic and sends it on to its intended recipient. It is very similar to a man-in-the-middle attack.

You enable HTTPS web caching from the CLI in a security policy or an explicit proxy policy that accepts the traffic to be cached using webcache-https. For a firewall policy:

config firewall policy edit 0 .

. . set webcache enable set webcache-https enable .

end

For an explicit web proxy policy:

config firewall proxy-policy edit 0 set proxy explicit-web .

. . set webcache enable set webcache-https enable .

. end

The webcache-https field is available only if webcache is enabled.

Web caching for HTTPS traffic is not supported if WAN optimization or FTP proxy is enabled: i.e., if srcintf is ftp-proxy or wanopt.

Turning on web caching for HTTPS traffic

The any setting causes the FortiGate unit to re-encrypt the traffic with the FortiGate unit’s certificate rather than the original certificate. This configuration can cause errors for HTTPS clients because the name on the certificate does not match the name on the web site.

You can stop these errors from happening by configuring HTTPS web caching to use the web server’s certificate by setting webcache-https to ssl-server. This option is available for both firewall policies and explicit web proxy policies.

config firewall policy edit 0 .

. . set webcache enable set webcache-https enable .

. end

The ssl-server option causes the FortiGate unit to re-encrypt the traffic with a certificate that you imported into the FortiGate unit. You can add certificates using the following command:

config firewall ssl-server edit corporate-server set ip <Web-Server-IP> set port 443 set ssl-mode { full | half} set ssl-cert <Web-Server-Cert>

end Where:

Web-Server-IP is the web server’s IP address.

Web-Server-Cert is a web server certificate imported into the FortiGate unit.

The SSL server configuration also determines whether the SSL server is operating in half or full mode and the port used for the HTTPS traffic.

You can add multiple SSL server certificates in this way. When web caching processing an SSL stream if it can find a certificate that matches the web server IP address and port of one of the added SSL servers; that certificate is used to encrypt the SSL traffic before sending it to the client. As a result the client does not generate SSL certificate errors.

Web caching uses the FortiGate unit’s FortiASIC to accelerate SSL decryption/encryption performance.

Full mode SSL server configuration

The ssl-mode option determines whether the SSL server operates in half or full mode. In full mode the FortiGate unit performs both decryption and encryption of the HTTPS traffic. The full mode sequence is shown below.

Turning on web caching for HTTPS traffic

Full mode SSL server configuration

In full mode the FortiGate unit is acting as a man in the middle, decrypting and encrypting the traffic. So both the client and the web server see encrypted packets.

Usually the port of the encrypted HTTPS traffic is always 443. However, in the SSL server configuration you can set the port used for HTTPS traffic. This port is not altered by the SSL Server. So for example, if the SSL Server receives HTTPS traffic on port 443, the re-encrypted traffic forwarded to the FortiGate unit to the server or client will still use port 443.

Half mode SSL server configuration

In half mode, the FortiGate unit only performs one encryption or decryption action. If HTTP packets are received, the half mode SSL server encrypts them and converts them to HTTPS packets. If HTTPS packets are received, the SSL server decrypts them and converts them to HTTP packets.

Half mode SSL server configuration

In half mode, the FortiGate unit is acting like an SSL accelerator, offloading HTTPS decryption from the web server to the FortiGate unit. Since FortiGate units can accelerate SSL processing, the end result could be improved web site performance.

Usually the port of the encrypted traffic is always 443. However, in the SSL server configuration you can set the port used for HTTPS traffic. No matter what port is used for the HTTPS traffic, the decrypted HTTP traffic uses port 80.

Changing the ports on which to look for HTTP and HTTPS traffic to cache

By default FortiOS assumes HTTP traffic uses TCP port 80 and HTTPS traffic uses port 443. So web caching caches all HTTP traffic accepted by a policy on TCP port 80 and all HTTPS traffic on TCP port 443. If you want to cache HTTP or HTTPS traffic on other ports, you can enable security profiles for the security policy and configure a proxy options profile to that looks for HTTP and HTTPS traffic on other TCP ports. To configure a proxy options profile go to Network > Explicit Proxy.

Setting the HTTP port to Any in a proxy options profile is not compatible with web caching. If you set the HTTP port to any, web caching only caches HTTP traffic on port 80.

Web caching and HA

You can configure web caching on a FortiGate HA cluster. The recommended best practice HA configuration for web caching is active-passive mode. When the cluster is operating, all web caching sessions are processed by the primary unit only. Even if the cluster is operating in active-active mode, HA does not load-balance web caching sessions.

In a cluster, only the primary unit stores the web cache database. The databases is not synchronized to the subordinate units. So, after a failover, the new primary unit must build its web cache.

Web caching and memory usage

To accelerate and optimize disk access and to provide better throughput and less latency, web caching uses provisioned memory to reduce disk I/O and increase disk I/O efficiency. In addition, web caching requires a small amount of additional memory per session for comprehensive flow control logic and efficient traffic forwarding.

When web caching is enabled you will see a reduction in available memory. The reduction increases when more web caching sessions are being processed. If you are thinking of enabling web caching on an operating FortiGate unit, make sure its memory usage is not maxed out during high traffic periods.

In addition to using the system dashboard to see the current memory usage you can use the get test wad 2 command to see how much memory is currently being used by web caching. See get test {wad | wccpd} <test_ level> on page 1 for more information.

Changing web cache settings

In most cases, the default settings for the WAN optimization web cache are acceptable. However, you may want to change them to improve performance or optimize the cache for your configuration. To change these settings, go to WAN Opt. & Cache > Settings.

From the FortiGate CLI, you can use the config wanopt webcache command to change these WAN optimization web cache settings.

Changing web cache settings

Always revalidate

Select to always revalidate requested cached objects with content on the server before serving them to the client.

Max cache object size

Set the maximum size of objects (files) that are cached. The default size is 512000 KB and the range is 1 to 4294967 KB. This setting determines the maximum object size to store in the web cache. Objects that are larger than this size are still delivered to the client but are not stored in the FortiGate web cache.

For most web traffic the default maximum cache object size is recommended. However, since web caching can also cache larger objects such as Windows updates, Mac OS updates, iOS updates or other updates delivered using HTTP you might want to increase the object size to make sure these updates are cached. Caching these updates can save a lot of Internet bandwidth and improve performance when major updates are released by these vendors.

Negative response duration

Set how long in minutes that the FortiGate unit caches error responses from web servers. If error responses are cached, then subsequent requests to the web cache from users will receive the error responses regardless of the actual object status.

The default is 0, meaning error responses are not cached. The content server might send a client error code (4xx HTTP response) or a server error code (5xx HTTP response) as a response to some requests. If the web cache is configured to cache these negative responses, it returns that response in subsequent requests for that page or image for the specified number of minutes.

Fresh factor

Set the fresh factor as a percentage. The default is 100, and the range is 1 to 100%. For cached objects that do not have an expiry time, the web cache periodically checks the server to see if the objects have expired. The higher the Fresh Factor the less often the checks occur.

For example, if you set the Max TTL value and Default TTL to 7200 minutes (5 days) and set the Fresh Factor to 20, the web cache check the cached objects 5 times before they expire, but if you set the Fresh Factor to 100, the web cache will check once.

Max TTL

The maximum amount of time (Time to Live) an object can stay in the web cache without the cache checking to see if it has expired on the server. The default is 7200 minutes (120 hours or 5 days) and the range is 1 to 5256000 minutes (5256000 minutes in a year).

Changing web cache settings

Min TTL

The minimum amount of time an object can stay in the web cache before the web cache checks to see if it has expired on the server. The default is 5 minutes and the range is 1 to 5256000 minutes (5256000 minutes in a year).

Default TTL

The default expiry time for objects that do not have an expiry time set by the web server. The default expiry time is 1440 minutes (24 hours) and the range is 1 to 5256000 minutes (5256000 minutes in a year).

Proxy FQDN

The fully qualified domain name (FQDN) for the proxy server. This is the domain name to enter into browsers to access the proxy server. This field is for information only can be changed from the explicit web proxy configuration.

Max HTTP request length

The maximum length of an HTTP request that can be cached. Larger requests will be rejected. This field is for information only can be changed from the explicit web proxy configuration.

Max HTTP message length

The maximum length of an HTTP message that can be cached. Larger messages will be rejected. This field is for information only can be changed from the explicit web proxy configuration.

Ignore

Select the following options to ignore some web caching features.

If-modified-since

By default, if the time specified by the if-modified-since (IMS) header in the client’s conditional request is greater than the last modified time of the object in the cache, it is a strong indication that the copy in the cache is stale. If so, HTTP does a conditional GET to the Overlay Caching Scheme (OCS), based on the last modified time of the cached object. Enable ignoring if-modified-since to override this behavior.

HTTP 1.1

conditionals

HTTP 1.1 provides additional controls to the client over the behavior of caches toward stale objects. Depending on various cache-control headers, the FortiGate unit can be forced to consult the OCS before serving the object from the cache. For more information about the behavior of cache-control header values, see RFC 2616 .Enable ignoring HTTP 1.1 Conditionals to override this behavior.

Changing web cache settings

Pragma-no-cache

Typically, if a client sends an HTTP GET request with a pragma no-cache (PNC) or cache-control no-cache header, a cache must consult the OCS before serving the content. This means that the FortiGate unit always re-fetches the entire object from the OCS, even if the cached copy of the object is fresh. Because of this behavior, PNC requests can degrade performance and increase server-side bandwidth utilization. However, if you enable ignoring Pragma-no-cache, then the PNC header from the client request is ignored. The FortiGate unit treats the request as if the PNC header is not present.

IE Reload

Some versions of Internet Explorer issue Accept / header instead of Pragma no-cache header when you select Refresh. When an Accept header has only the / value, the FortiGate unit treats it as a PNC header if it is a type-N object. Enable ignoring IE reload to cause the FortiGate unit to ignore the PNC interpretation of the Accept / header.

Cache expired objects

Applies only to type-1 objects. When this option is selected, expired type-1 objects are cached (if all other conditions make the object cacheable).

Revalidated pragma-no-cache

The pragma-no-cache (PNC) header in a client’s request can affect how efficiently the FortiGate unit uses bandwidth. If you do not want to completely ignore PNC in client requests (which you can do by selecting to ignore Pragma-no-cache, above), you can nonetheless lower the impact on bandwidth usage by selecting Revalidate Pragma-no-cache.

When you select Revalidate Pragma-no-cache, a client’s non-conditional PNC-GET request results in a conditional GET request sent to the OCS if the object is already in the cache. This gives the OCS a chance to return the 304 Not Modified response, which consumes less server-side bandwidth, because the OCS has not been forced to otherwise return full content.

By default, Revalidate Pragma-no-cache is disabled and is not affected by changes in the top-level profile.

Most download managers make byte-range requests with a PNC header. To serve such requests from the cache, you should also configure byte-range support when you configure the Revalidate pragma-no-cache option.

Peers and authentication groups

Leave a reply

Peers and authentication groups

All communication between WAN optimization peers begins with one WAN optimization peer (or client-side FortiGate unit) sending a WAN optimization tunnel request to another peer (or server-side FortiGate unit). During this process, the WAN optimization peers identify and optionally authenticate each other.

Basic WAN optimization peer requirements

WAN optimization requires the following configuration on each peer. For information about configuring local and peer host IDs, see Basic WAN optimization peer requirements on page 319.

The peer must have a unique host ID.
Unless authentication groups are used, peers authenticate each other using host ID values. Do not leave the local host ID at its default value.
The peer must know the host IDs and IP addresses of all of the other peers that it can start WAN optimization tunnels with. This does not apply if you use authentication groups that accept all peers.
All peers must have the same local certificate installed on their FortiGate units if the units authenticate by local certificate. Similarly, if the units authenticate by pre-shared key (password), administrators must know the password. The type of authentication is selected in the authentication group. This applies only if you use authentication groups.

Accepting any peers

Strictly speaking, you do not need to add peers. Instead you can configure authentication groups that accept any peer. However, for this to work, both peers must have the same authentication group (with the same name) and both peers must have the same certificate or pre-shared key.

Accepting any peer is useful if you have many peers or if peer IP addresses change. For example, you could have FortiGate units with dynamic external IP addresses (using DHCP or PPPoE). For most other situations, this method is not recommended and is not a best practice as it is less secure than accepting defined peers or a single peer. For more information, see Basic WAN optimization peer requirements on page 319.

How FortiGate units process tunnel requests for peer authentication

When a client-side FortiGate unit attempts to start a WAN optimization tunnel with a peer server-side FortiGate unit, the tunnel request includes the following information:

the client-side local host ID l the name of an authentication group, if included in the rule that initiates the tunnel l if an authentication group is used, the authentication method it specifies: pre-shared key or certificate l the type of tunnel (secure or not).

Configuring peers

For information about configuring the local host ID, peers and authentication groups, see How FortiGate units process tunnel requests for peer authentication on page 319 and How FortiGate units process tunnel requests for peer authentication on page 319.

The authentication group is optional unless the tunnel is a secure tunnel. For more information, see How FortiGate units process tunnel requests for peer authentication on page 319.

If the tunnel request includes an authentication group, the authentication will be based on the settings of this group as follows:

The server-side FortiGate unit searches its own configuration for the name of the authentication group in the tunnel request. If no match is found, the authentication fails.
If a match is found, the server-side FortiGate unit compares the authentication method in the client and server authentication groups. If the methods do not match, the authentication fails.
If the authentication methods match, the server-side FortiGate unit tests the peer acceptance settings in its copy of the authentication group.
If the setting is Accept Any Peer, the authentication is successful.
If the setting is Specify Peer, the server-side FortiGate unit compares the client-side local host ID in the tunnel request with the peer name in the server-side authentication group. If the names match, authentication is successful. If a match is not found, authentication fails.
If the setting is Accept Defined Peers, the server-side FortiGate unit compares the client-side local host ID in the tunnel request with the server-side peer list. If a match is found, authentication is successful. If a match is not found, authentication fails.

If the tunnel request does not include an authentication group, authentication will be based on the client-side local host ID in the tunnel request. The server-side FortiGate unit searches its peer list to match the client-side local host ID in the tunnel request. If a match is found, authentication is successful. If a match is not found, authentication fails.

If the server-side FortiGate unit successfully authenticates the tunnel request, the server-side FortiGate unit sends back a tunnel setup response message. This message includes the server-side local host ID and the authentication group that matches the one in the tunnel request.

The client-side FortiGate unit then performs the same authentication procedure as the server-side FortiGate unit did. If both sides succeed, tunnel setup continues.

Configuring peers

When you configure peers, you first need to add the local host ID that identifies the FortiGate unit for WAN optimization and then add the peer host ID and IP address of each FortiGate unit with which a FortiGate unit can create WAN optimization tunnels.

To configure WAN optimization peers – web-based manager:

Go to WAN Opt. & Cache > Peers.
For Local Host ID, enter the local host ID of this FortiGate unit and select Apply. If you add this FortiGate unit as a peer to another FortiGate unit, use this ID as its peer host ID.

The local or host ID can contain up to 25 characters and can include spaces.

Select Create New to add a new peer.

Configuring authentication groups Peers and authentication groups

For Peer Host ID, enter the peer host ID of the peer FortiGate unit. This is the local host ID added to the peer FortiGate unit.
For IP Address, add the IP address of the peer FortiGate unit. This is the source IP address of tunnel requests sent by the peer, usually the IP address of the FortiGate interface connected to the WAN.
Select OK.

To configure WAN optimization peers – CLI:

In this example, the local host ID is named HQ_Peer and has an IP address of 172.20.120.100. Three peers are added, but you can add any number of peers that are on the WAN.

Enter the following command to set the local host ID to HQ_Peer. config wanopt settings set host-id HQ_peer

end

Enter the following commands to add three peers.

config wanopt peer edit Wan_opt_peer_1 set ip 172.20.120.100

edit Wan_opt_peer_2 set ip 172.30.120.100

edit Wan_opt_peer_3 set ip 172.40.120.100 end

Configuring authentication groups

You need to add authentication groups to support authentication and secure tunneling between WAN optimization peers.

To perform authentication, WAN optimization peers use a certificate or a pre-shared key added to an authentication group so they can identify each other before forming a WAN optimization tunnel. Both peers must have an authentication group with the same name and settings. You add the authentication group to a peer-topeer or active rule on the client-side FortiGate unit. When the server-side FortiGate unit receives a tunnel start request from the client-side FortiGate unit that includes an authentication group, the server-side FortiGate unit finds an authentication group in its configuration with the same name. If both authentication groups have the same certificate or pre-shared key, the peers can authenticate and set up the tunnel.

Authentication groups are also required for secure tunneling.

To add authentication groups, go to WAN Opt. & Cache > Authentication Groups.

To add an authentication group – web-based manager:

Use the following steps to add any kind of authentication group. It is assumed that if you are using a local certificate to authenticate, it is already added to the FortiGate unit

Go to WAN Opt. & Cache > Authentication Groups.
Select Create New.

Configuring authentication groups

Add a Name for the authentication group.

You will select this name when you add the authentication group to a WAN optimization rule.

Select the Authentication Method.

Select Certificate if you want to use a certificate to authenticate and encrypt WAN optimization tunnels. You must select a local certificate that has been added to this FortiGate unit. (To add a local certificate, go to System > Certificates.) Other FortiGate units that participate in WAN optimization tunnels with this FortiGate unit must have an authentication group with the same name and certificate.

Select Pre-shared key if you want to use a pre-shared key or password to authenticate and encrypt WAN optimization tunnels. You must add the Password (or pre-shared key) used by the authentication group. Other FortiGate units that participate in WAN optimization tunnels with this FortiGate unit must have an authentication group with the same name and password. The password must contain at least 6 printable characters and should be known only by network administrators. For optimum protection against currently known attacks, the key should consist of a minimum of 16 randomly chosen alphanumeric characters.

Configure Peer Acceptance for the authentication group.

Select Accept Any Peer if you do not know the peer host IDs or IP addresses of the peers that will use this authentication group. This setting is most often used with FortiGate units that do not have static IP addresses, for example units that use DHCP.

Select Accept Defined Peers if you want to authenticate with peers added to the peer list only.

Select Specify Peer and select one of the peers added to the peer list to authenticate with the selected peer only.

Select OK.
Add the authentication group to a WAN optimization rule to apply the authentication settings in the authentication group to the rule.

To add an authentication group that uses a certificate- CLI:

Enter the following command to add an authentication group that uses a certificate and can authenticate all peers added to the FortiGate unit configuration.

In this example, the authentication group is named auth_grp_1 and uses a certificate named Example_ Cert.

config wanopt auth-group edit auth_grp_1 set auth-method cert set cert Example_Cert set peer-accept defined

end

To add an authentication group that uses a pre-shared key – CLI:

Enter the following command to add an authentication group that uses a pre-shared key and can authenticate only the peer added to the authentication group.

Secure tunneling Peers and authentication groups

In this example, the authentication group is named auth_peer, the peer that the group can authenticate is named Server_net, and the authentication group uses 123456 as the pre-shared key. In practice you should use a more secure pre-shared key.

config wanopt auth-group edit auth_peer set auth-method psk set psk 123456 set peer-accept one set peer Server_net

end

To add an authentication group that accepts WAN optimization connections from any peer – web-based manager

Add an authentication group that accepts any peer for situations where you do not have the Peer Host IDs or IP

Addresses of the peers that you want to perform WAN optimization with. This setting is most often used for WAN optimization with FortiGate units that do not have static IP addresses, for example units that use DHCP. An authentication group that accepts any peer is less secure than an authentication group that accepts defined peers or a single peer.

The example below sets the authentication method to Pre-shared key. You must add the same password to all FortiGate units using this authentication group.

Go to WAN Opt. & Cache > Authentication Groups.
Select Create New to add a new authentication group.
Configure the authentication group:

Name	Specify any name.
Authentication Method	Pre-shared key
Password	Enter a pre-shared key.
Peer Acceptance	Accept Any Peer

To add an authentication group that accepts WAN optimization connections from any peer – CLI:

In this example, the authentication group is named auth_grp_1. It uses a certificate named WAN_Cert and accepts any peer.

config wanopt auth-group edit auth_grp_1 set auth-method cert set cert WAN_Cert set peer-accept any

end

Secure tunneling

You can configure WAN optimization rules to use AES-128bit-CBC SSL to encrypt the traffic in the WAN optimization tunnel. WAN optimization uses FortiASIC acceleration to accelerate SSL decryption and encryption Monitoring WAN optimization peer performance

of the secure tunnel. Peer-to-peer secure tunnels use the same TCP port as non-secure peer-to-peer tunnels (TCP port 7810).

To use secure tunneling, you must select Enable Secure Tunnel in a WAN optimization rule and add an authentication group. The authentication group specifies the certificate or pre-shared key used to set up the secure tunnel. The Peer Acceptance setting of the authentication group does not affect secure tunneling.

The FortiGate units at each end of the secure tunnel must have the same authentication group with the same name and the same configuration, including the same pre-shared key or certificate. To use certificates you must install the same certificate on both FortiGate units.

For active-passive WAN optimization you can select Enable Secure Tunnel only in the active rule. In peer-topeer WAN optimization you select Enable Secure Tunnel in the WAN optimization rule on both FortiGate units. For information about active-passive and peer-to-peer WAN optimization, see Manual (peer-to-peer) and activepassive WAN optimization on page 1

For a secure tunneling configuration example, see Example: Adding secure tunneling to an active-passive WAN optimization configuration on page 1.

Monitoring WAN optimization peer performance

The WAN optimization peer monitor lists all of the WAN optimization peers that a FortiGate unit can perform WAN optimization with. These include peers manually added to the configuration as well as discovered peers.

The monitor lists each peer’s name, IP address, and peer type. The peer type indicates whether the peer was manually added or discovered. To show WAN optimization performance, for each peer the monitor lists the percent of traffic reduced by the peer in client-side WAN optimization configurations and in server-side configurations (also called gateway configurations).

To view the peer monitor, go to WAN Opt. & Cache > Peer Monitor.