17.1.8. Data search and filtering

Logs normally contain a huge number of records, and not all fields are available in the basic viewing mode. UserGate offers convenient ways to search and filter the information you need. Administrators can search the contents of the logs in basic and advanced modes.

With a simple search, administrators use a graphic interface to set filters by values of the required log fields, thus filtering out unnecessary information. For example, administrators can specify a time range of interest, a list of users, categories, etc. Setting the search criteria is intuitive and does not require any special knowledge.

You can create more complex filters in the advanced search mode using a special query language. In the advanced search mode, you can build queries using log fields that are not available in the basic mode. To construct a query, use field names and values, keywords, and operators. You can enter field values using single or double quotes, or without quotes, if the values do not contain spaces. To group multiple conditions, use parentheses.

Separate keywords by spaces. You can use the following keywords:

Name

Description

AND/and

Logical AND: all query conditions should be met.

OR/or

Logical OR: at least one condition should be met.

The following operators define filter conditions:

Name

Description

=

Equal to. Field value should exactly match the specified value, for example,

ip=172.16.31.1

displays all log entries where the IP field exactly matches 172.16.31.1.

!=

Not equal to. Field value must not match the specified value, for example,

ip!=172.16.31

displays all log entries where the IP field does not match 172.16.31.1.

<=

Less than or equal to. Field value must be less than or equal to the specified value. This can only apply to fields that support comparisons, such as date, portSource, portDest, statusCode, etc., for example,

date <= '2019-03-28T20:59:59' AND statusCode=303

>=

Greater than or equal to. Field value must be greater than or equal to the specified value. This can only apply to fields that support comparisons, such as date, portSource, portDest, statusCode, etc., for example,

date >= "2019-03-13T21:00:00" AND statusCode=200

<

Less than. Field value must be less than the specified value. This can only apply to fields that support comparisons, such as date, portSource, portDest, statusCode, etc., for example,

date < '2019-03-28T20:59:59' AND statusCode=404

>

Greater than. The field value must be greater than the specified value. This can only apply to fields that support comparisons, such as date, portSource, portDest, statusCode, etc., for example,

(statusCode>200 AND statusCode <300) OR (statusCode=404)

IN

Allows you to specify multiple values for a field in a query. Provide the list of values in parentheses, for example,

category IN (botnets, compromised, 'illegal software', 'phishing and fraud','reputation high risk','unknown category')

NOT IN

Allows you to specify multiple values for a field in a query. Displays records that do not contain the specified values. Provide the list of values in parentheses, for example,

category NOT IN (botnets, compromised, 'illegal software', 'phishing and fraud','reputation high risk','unknown category')

~

Contains. Allows you to specify a substring that the queried field must contain, for example,

browser ~ "Mozilla/5.0"

This operator is applicable only to fields that contain string data.

!~

Does not contain. Allows you to specify a substring that the queried field must not contain, for example,

browser !~ "Mozilla/5.0"

This operator is applicable only to fields that contain string data.

MATCH

To specify the substring that must be found in the specified field using the MATCH statement, use JSON format and single quotes, for example,

details MATCH '\"module\":\"threats\"'

The syntax of queries using this operator is compliant with the RE2 standard. For more details about Google/RE2 syntax, see: https://github.com/google/re2/wiki/Syntax.

NOT MATCH

To specify the substring that must not be found in the specified field using the NOT MATCH statement, use JSON format and single quotes, for example,

details NOT MATCH '\"module\":\"threats\"'

The syntax of queries using this operator is compliant with the RE2 standard. For more details about Google/RE2 syntax, see: https://github.com/google/re2/wiki/Syntax.

When making an advanced query, UserGate shows possible field names, applicable operators, and possible values, making it easier for the system operator to make complex queries. The list of fields and their possible values for each log may be different.

When you switch from basic to advanced search mode, UserGate automatically generates a search query string that matches the filter specified in the basic search mode.