The URL lists page allows you to create URL lists to be used as black and white lists in content filtering rules.
UserGate provides its own updatable URL lists. To use the lists, an appropriate license is required. For more details on product licensing, see the chapter UserGate Licensing.
|
Name |
Description |
|---|---|
|
List of search engines without safesearch capability |
The list of known search engines that do not offer the ability to block search queries related to adult content. It is recommended to block such search engines for parental control purposes. |
|
Compliance with RU URL (Custom 460) |
The list of URLs prohibited by the Ministry of Justice of the Russian Federation. |
|
URL: Compliance with KZ URL (Custom 487) |
The unified registry of domain names, Internet URLs, and network addresses containing information that is prohibited for distribution in the Republic of Kazakhstan. |
|
Educational institutions |
The list of domain names of educational institutions in the Russian Federation. |
|
Phishing sites |
A list of phishing website URLs. |
|
Compliance with RU RKN (URL) |
The unified registry of Internet URLs containing information that is prohibited for distribution in the Russian Federation. This list is available on the website http://eais.rkn.gov.ru. |
|
Compliance with RU RKN (domains) |
The unified registry of domain names containing information that is prohibited for distribution in the Russian Federation. This list is available on the website http://eais.rkn.gov.ru. |
To configure filtering using URL lists, follow these steps:
|
Task |
Description |
|---|---|
|
Step 1. Create a URL list. |
In the URL lists pane, click Add and give a name to the new list. |
|
Step 2. Add the relevant entries to the new list. |
Add URL entries to the new list. You can use wildcards such as "^", "$", and "*": "*": any number of any characters "^": start of a line "$": end of a line. The "?" and "#" characters cannot be used. |
|
Step 3. Create a content filtering rule containing one or more lists. |
See the section Content Filtering. |
If a URL entry starts with "http://", "https://", "ftp://" or contains one or more "/" characters, it is considered a URL and used only for HTTP(S) filtering but not DNS filtering. Otherwise, the string is considered a domain name and used for both DNS and HTTP(S) filtering.
If you want to block an exact address, use the "^" and "$" characters:
To block an exact URL with all child directories, use the "^" character:
Note
Using the special characters is not recommended when creating ad blocking lists.
To block a domain with all possible URLs, use this notation:
domain.com
An example of interpreting URL entries:
|
Example entry |
DNS request processing |
HTTP request processing |
|---|---|---|
|
yahoo.com or *yahoo.com* |
The entire domain and its 3rd level domains are blocked, e.g.: sport.yahoo.com mail.yahoo.com as well as: qweryahoo.com |
The entire domain along with all its URLs and 3rd level domains are blocked, e.g.: |
|
^mail.yahoo.com$ |
Only mail.yahoo.com is blocked. |
Only the following are blocked: |
|
^mail.yahoo.com/$ |
Nothing is blocked. |
Nothing is blocked, since the last forward slash character defines a URL but there is no "https" or "http". |
|
^http://finance.yahoo.com/ personal-finance/$ |
Nothing is blocked. |
Only this address is blocked: |
|
^yahoo.com/12345/ |
Nothing is blocked. |
These are blocked: |
The administrator can create custom lists and distribute them centrally to all computers where UserGate is installed. To create such a list, follow these steps:
|
Task |
Description |
|---|---|
|
Step 1. Generate a file with the relevant URL list. |
Create a file named list.txt with the URL list in the following format: www.site1/url1 www.site2/url2 ... |
|
Step 2. Create an archive containing this file. |
Put the file in a ZIP archive named list.zip. |
|
Step 3. Create a version file for the list. |
Create a file named version.txt and specify the list version number inside it, such as 3. On each update of the list, the version number must be incremented. |
|
Step 4. Upload the files to a web server. |
Upload the list.zip and version.txt files to your website so that they can be downloaded. |
|
Step 5. Create a content type list and specify an update URL for it. |
On each UserGate server, create a URL list. When creating the list, select Updatable as the list type and enter the address for downloading updates. UserGate will check for a new version on your website according to the set update download schedule. The schedule can be configured in the list properties. The available options are:
With the Advanced option, a crontab-like format is used where the date/time string consists of six space-separated fields. The fields specify the time as follows: (minutes: 0‑59) (hours: 0-23) (days of the month: 1-31) (month: 1-12) (days of the week: 0-6, where 0 is Sunday). Each of the first five fields can be defined using:
An asterisk or range spacing: used for spacing out values in ranges. The increment is given after a slash. Examples: "2-10/2" means "2,4,6,8,10" while "*/2" in the "hours" field means "every two hours". |