FortiOS 6 – Web Filtering

YouTube Education Filter

YouTube for Schools was a way to access educational videos from inside a school network. This YouTube feature gave schools the ability to access a broad set of educational videos on YouTube EDU and to select the specific videos that are accessible from within the school network.

Google decided to stop supporting YouTube for Schools (YTfS) as of July 1, 2016. Consequently, the current YouTube safe search does not work anymore.

Google provides an article entitled “Restrict YouTube content on your network or managed devices” on its support site. At this time, Google offers two options to restrict inappropriate content: DNS and HTTP header.

In FortiOS 5.6 with inspection mode set to proxy-based, in a Web Filter profile under Search Engines you can select Restrict YouTube Access and select either Strict or Moderate.

YouTube Channel Filtering

This Web Filtering feature lets you block or allow matched YouTube channels using one of the following identifiers:

  • <channel-id>
  • youtube.com/channel/<channel-id> l www.youtube.com/user/<user-id> matches channel-id from <meta itemprop=”channelId” content=”UCGzuiiLdQZu9wxDNJHO_JnA”>
  • youtube.com/watch?v=<string> matches channel-id from <meta itemprop=”channelId” content=”UCGzuiiLdQZu9wxDNJHO_JnA”>

Syntax

Note that config youtube-channel-filter is only available when youtube-channel-status is set to either blacklist or whitelist. Also note that, when defining channel-id, both the full URL or just the Channel ID suffix are acceptable, as shown below:

config webfilter profile edit <name> set youtube-channel-status {disable | blacklist | whitelist} config youtube-channel-filter edit <id> set channel-id <url>

next edit <id> set channel-id <channel-id>

next

end end

 

Static URL filter

You can allow or block access to specific URLs by adding them to the Static URL Filter list. The filter allows you to block, allow, or monitor URLs by using patterns containing text, regular expressions, or wildcard characters. The FortiGate unit allows or blocks web pages matching any specified URLs or patterns and displays a replacement message instead.

URL blocking does not block access to other services that users can access with a web browser. For example, URL blocking does not block access to ftp:// ftp.example.com. Instead, use firewall policies to deny ftp connections.

When adding a URL to the URL filter list, follow these rules:

  • Type a top-level URL or IP address to control access to all pages on a web site. For example, example.com or 192.168.144.155 controls access to all pages at this web site.
  • Enter a top-level URL followed by the path and file name to control access to a single page on a web site. For example, example.com/news.html or 192.168.144.155/news.html controls access to the news page on this web site.
  • To control access to all pages with a URL that ends with example.com, add com to the filter list. For example, adding example.com controls access to www.example.com, mail.example.com, www.finance.example.com, and so on.
  • Control access to all URLs that match patterns using text and regular expressions (or wildcard characters). For example, * matches example.com, example.org, example.net and so on.

URLs with an action set to exempt or monitor are not scanned for viruses. If users on the network download files through the FortiGate unit from a trusted web site, add the URL of this web site to the URL filter list with an action to pass it so the FortiGate unit does not virus scan files downloaded from this URL.

URL formats

How URL formats are detected when using HTTPS

Filter HTTPS traffic by entering a top level domain name, for example, www.example.com if:

  • your unit does not support SSL content scanning and inspection
  • you have selected the URL filtering option in web content profile for HTTPS content filtering mode under Protocol Recognition.

HTTPS URL filtering of encrypted sessions works by extracting the CN from the server certificate during the SSL negotiation. Since the CN only contains the domain name of the site being accessed, web filtering of encrypted HTTPS sessions can only filter by domain names.

If your unit supports SSL content scanning and inspection and if you have selected Deep Scan, you can filter HTTPS traffic in the same way as HTTP traffic.

Static URL filter

How URL formats are detected when using HTTP

URLs with an action set to Exempt are not scanned for viruses. If users on the network download files through the unit from trusted web site, add the URL of this web site to the URL filter list with an action set to exempt so the unit does not virus scan files downloaded from this URL.

  • Type a top-level URL or IP address to control access to all pages on a web site. For example, example.com or 192.168.144.155 controls access to all pages at this web site.
  • Enter a top-level URL followed by the path and filename to control access to a single page on a web site. For example, example.com/news.html or 192.168.144.155/news.html controls the news page on this web site.
  • To control access to all pages with a URL that ends with com, add example.com to the filter list. For example, adding example.com controls access to www.example.com, mail.example.com, www.finance.example.com, and so on. l Control access to all URLs that match patterns created using text and regular expressions (or wildcard characters). For example, example.* matches example.com, example.org, example.net and so on. l Fortinet URL filtering supports standard regular expressions.

URL filter actions

You can select one of four actions for how traffic will be treated as it attempts to reach a site in the list.

Block

Attempts to access any URLs matching the URL pattern are denied. The user will be presented with a replacement message.

Allow

Any attempt to access a URL that matches a URL pattern with an allow action is permitted. The traffic is passed to the remaining antivirus proxy operations, including FortiGuard Web Filter, web content filter, web script filters, and antivirus scanning.

Allow is the default action. If a URL does not appear in the URL list, it is permitted.

Monitor

Traffic to, and reply traffic from, sites matching a URL pattern with Monitor action applied will be allowed through in the same way as the Allow action. The difference with the Monitor action is that a log message will be generated each time a matching traffic session is established. The requests will also be subject to all other Security Profiles inspections that would normally be applied to the traffic.

Exempt

Exempt allows trusted traffic to bypass the antivirus and DLP proxy operations by default, but it functions slightly differently. In general, if you’re not certain that you need to use the Exempt action, use Monitor.

HTTP 1.1 connections are persistent unless declared otherwise. This means the connections will remain in place until closed or the connection times out. When a client loads a web page, the client opens a connection to the web server. If the client follows a link to another page on the same site before the connection times out, the same connection is used to request and receive the page data.

When you add a URL pattern to a URL filter list and apply the Exempt action, traffic sent to and replies traffic from sites matching the URL pattern will bypass all antivirus proxy operations. The connection itself inherits the exemption. This means that all subsequent reuse of the existing connection will also bypass all antivirus proxy operations. When the connection times out, the exemption is cancelled.

For example, consider a URL filter list that includes example.com/files configured with the Exempt action. A user opens a web browser and downloads a file from the URL example.com/sample.zip. This URL does not match the URL pattern so it is scanned for viruses. The user then downloads example.com/files/beautiful.exe and since this URL does match the pattern, the connection itself inherits the exempt action. The user then downloads example.com/virus.zip. Although this URL does not match the exempt URL pattern, a previously visited URL did, and since the connection inherited the exempt action and was re-used to download a file, the file is not scanned.

If the user next goes to an entirely different server, like example.org/photos, the connection to the current server cannot be reused. A new connection to example.org is established. This connection is not exempt. Unless the user goes back to example.com before the connection to that server times out, the server will close the connection. If the user returns after the connection is closed, a new connection to example.com is created and it is not exempt until the user visits a URL that matches the URL pattern.

Web servers typically have short time-out periods. A browser will download multiple components of a web page as quickly as possible by opening multiple connections. A web page that includes three photos will load more quickly if the browser opens four connections to the server and downloads the page and the three photos at the same time. A short time-out period on connections will close the connections faster, allowing the server to avoid unnecessarily allocating resources for a long period. The HTTP session time-out is set by the server and will vary with the server software, version, and configuration.

Using the Exempt action can have unintended consequences in certain circumstances. You have a web site at example.com and since you control the site, you trust the contents and configure example.com as exempt. But example.com is hosted on a shared server with a dozen other different sites, each with a unique domain name. Because of the shared hosting, they also share the same IP address. If you visit example.com, your connection your site becomes exempt from any antivirus proxy operations. Visits to any of the 12 other sites on the same server will reuse the same connection and the data you receive is exempt from being scanned.

Use of the Exempt action is not suitable for configuration in which connections through the FortiGate unit use an external proxy. For example, you use proxy.example.net for all outgoing web access. Also, as in the first example, URL filter list that includes a URL pattern of example.com/files configured with the Exempt action. Users are protected by the antivirus protection of the FortiGate unit until a user visits a URL that matches the of example.com/files URL pattern. The pattern is configured with the Exempt action so the connection to the server inherits the exemption. With a proxy however, the connection is from the user to the proxy. Therefore, the user is entirely unprotected until the connection times out, no matter what site he visits.

Ensure you are aware of the network topology involving any URLs to which you apply the Exempt action.

Static URL filter

Status

The Web Site Filter has the option to either enable or disable individual web sites in the list. This allows for the temporary removal of the actions against a site so that it can be later reengaged without having to rewrite the configuration.


Having trouble configuring your Fortinet hardware or have some questions you need answered? Check Out The Fortinet Guru Youtube Channel! Want someone else to deal with it for you? Get some consulting from Fortinet GURU!

3 thoughts on “FortiOS 6 – Web Filtering

    1. Mike Post author

      Backup the config and nit pick through it. Be sure the FortiGates are running the same version of code though!

      Reply

Leave a Reply to Tony Cancel reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.