8.1. Content Filtering

The administrator can use content filtering rules to allow or block certain content transmitted using the HTTP or HTTPS protocols (HTTPS inspection needs to be configured in the latter case). Moreover, UserGate can block HTTPS traffic without content decryption, but only using blocking rules that work at the level of specific UserGate URL Filtering content categories or URL lists containing just the host names. In these cases, UserGate uses SNI (Server Name Indication) or, in the absence of that, the host values from the SSL certificate specified in user host resolution requests.

You can use the following as conditions for a rule:

  • Users and groups.

  • Specific words and phrases (morphology) present on the webpage.

  • Website category.

  • URL.

  • Source zone and IP address.

  • Destination zone and IP address.

  • Content type.

  • Referrer information.

  • Time.

  • Useragent of the user browser.

  • HTTP method.

Note

The Negate checkbox changes the condition to the opposite, which corresponds to a Boolean NOT (negation).

Note

The rules are applied top to bottom in their listing order. Only the first rule in which all conditions are matched is applied. This means that more specific rules must be placed higher in the list than more general ones. To change the order in which the rules will be applied, use the Up/Down and Top/Bottom buttons or drag and drop the rules with the mouse.

Note

If there are no rules created, all content is allowed through.

To create a content filtering rule, go to the Security policies --> Content filtering section, click Add, and provide the desired settings.

Name

Description

Enabled

Enables or disables the rule.

Name

The name of the rule.

Description

A description of the rule.

Action

Deny: blocks the webpage.

Warning: warns the user that visiting this page is undesirable. It is up to the user to decide whether to visit the page or not. If they do proceed to the page, the visit is logged.

Allow: allows visiting the webpage.

Enable logging

If this is enabled, instances of the rule being triggered will be recorded in the corresponding statistics log.

UserGate stream virus check

This is available only for the rules with the Deny action --- that is, if the page contains a virus, the resource will be blocked. If the rule has other conditions (categories, time, etc.), the virus check will be done only when all conditions are matched. If in-stream virus checking is enabled, the content filtering rule will only be triggered when a virus is detected.

Scenario

The scenario that must be active for the rule to be triggered. For more details on how scenarios work, see the section Scenarios.

Important! A scenario is an additional condition. If the scenario was not triggered (one or more scenario triggers did not occur), the rule will not be triggered.

Blocking page

Specifies the block page that will be shown to the user when their access to the resource is blocked. You can specify an external page (by setting the Use custom external URL checkbox) or a UserGate block page. In the latter case, you can select the desired block page template, which can be created in the Response Pages section.

Source

The zone, IP address lists, GeoIP address lists, or URL lists of the traffic source.

The URL list must include only domain names. Every 5 minutes UserGate resolves domain names into IP addresses and stores the result in the internal cache for the DNS record's time-to-live (TTL). When the TTL expires, UserGate automatically updates the IP address value.

Important! There is a limit on the number of GeoIPs that can be specified: the number cannot exceed 15.

Important! Traffic processing performed with the following statements:

  • applying logic OR if several IP lists and/or domain lists are specified;

  • applying logic AND if several GeoIP and lists of IPs and/or domains are specified.

Destination

The zone, IP address lists, GeoIP address lists, or URL lists of the traffic destination.

The URL list must include only domain names. Every 5 minutes UserGate resolves domain names into IP addresses and stores the result in the internal cache for the DNS record's time-to-live (TTL). When the TTL expires, UserGate automatically updates the IP address value.

Important! There is a limit on the number of GeoIPs that can be specified: the number cannot exceed 15.

Important! Traffic processing performed with the following statements:

  • applying logic OR if several IP lists and/or domain lists are specified;

  • applying logic AND if several GeoIP and lists of IPs and/or domains are specified.

Users

The list of users and user groups to which this rule is applied. The Any, Unknown, and Known user types can be used. To apply rules to specific users or Known users, user identification needs to be configured. For more details on user identification, see the chapter Users and Devices.

Categories

UserGate URL Filtering 4.0 category lists. To use URL categories, an appropriate license is required. UserGate URL Filtering 4.0 is a massive database of web resources classified for convenience into 72 categories. The administrator can control access to categories such as pornography, malicious websites, online casinos, gaming and entertainment websites, social networks, and many others.

Important! Starting from UserGate version 5.0.6R6, the administrator can override the category for any website that is, in their opinion, miscategorized or not categorized at all. For more details on overriding a website's category, see the section Requests for white list.

Important! Blocking by URL category can be applied to HTTPS traffic without decrypting the content, but the block page will not be shown.

URL

URL lists. If you have the corresponding license, various URL lists maintained by the UserGate developer team are available to you, such as UserGate's "Black list" and "White list", "RU RKN" (Roskomnadzor's Black List), "Black list of Phishing sites", and "List of search engines without safesearch capability". Administrators can also create custom URL lists. For more details on working with URL lists, see the chapter URL Lists.

Important! Blocking by URL lists can be applied to HTTPS traffic without decrypting the content, provided that the lists contain only host (domain) names, but the block page will not be shown.

Content types

The content type lists. Video, audio, images, executables, and other types of content can be controlled. Administrators can also create custom content type groups. For more details on working with content types, see the chapter Content Types.

Morphology

The list of morphological dictionary databases that will be used to check webpages. If you have the corresponding license, various dictionaries maintained by UserGate are available to you, including "Compliance to RU (Custom 460)" (the list of materials prohibited by the Ministry of Justice of the Russian Federation) and dictionaries on topics such as "Suicide", "Terrorism", "Pornography", "Profanity", "Gambling", "Drugs", and "Compliance to RU FZ436" (Russian child protection legislation). The dictionaries are available in Russian, English, German, Japanese, and Arabic.

Administrators can also create custom dictionaries. For more details on working with morphological dictionaries, see the chapter Morphology.

Time

The time when this rule will be active. The administrator can add the required time period in the Time sets section.

Useragent

The user browser useragents for which this rule will be applied. The administrator can add the desired useragents in the Browser Useragent section.

HTTP method

The method used in HTTP requests, usually POST or GET.

Referrers

The list of referrer URLs for the current page. The rule will be triggered if the referrer URL for the page matches the list. This functionality offers a convenient way to allow access to CDNs (content delivery networks) only when specific websites are visited but not when users try to open CDN content directly.

Usage

The rule triggering statistics: the total number of triggers, the time of the first and last triggers.

Important! If the inspection of data transmitted using the TLS/SSL protocol configured, and the default content filtering rule triggered, only the SSL inspection rule counter will be triggered.

To reset statistics, select rules in the list and click Reset hit counts.

History

The time when the rule was created and last modified, as well as the event log entries related to this rule: adding, updating the rule, changing the position of the rule in the list, etc.