Introduction
This article will explain how to access the desired data simply by using the Apache Lucene language in the search section.
Searching Data with Lucene Usage
In the Logsign architecture, incoming data is parsed into relevant columns and displayed in the search section. The column structure is in the form of MainColumn.SubColumn. For example; All data related to the source in incoming data is assigned to the source main column, and according to the data diversity, it is divided into sub-columns. The representation of the source main column and sub-columns specified as the source in the screenshot below is available.
All columns in Logsign work in this architecture, and even if the data formats are different, a standardization is applied on Logsign and all incoming data is assigned to common columns. For example; Although information about the source is coming in different ways in the formats of data coming from PaloAlto and Fortinet devices, Logsign will use the source main column and its sub-columns, which are common columns, when parsing and dividing this data.
Logical Operators
AND
If multiple conditions are entered while searching data and all of these conditions are required at the same time, the AND operator must be used. While searching, the relevant operator can be used by writing AND or leaving a space between the columns we will search. An example search has been made for the data in which the source IP address is 10.10.5.89 and the destination IP address is 10.10.2.69 in the screenshot below.
OR
If any of the conditions in the query entered while searching are sufficient, the OR operator can be used. An example has been given in the screenshot below to bring the data with the Source IP address of 10.10.9.250 or 10.10.2.65.
NOT
If any condition needs to be excluded from the data to be searched, the NOT operator can be used. An example has been given in the screenshot below, where the source IP address is 10.10.9.250, but the destination port should not be 53.
( ) Parenthesis
The parenthesis operator must be used to separate certain conditions while searching or to enter multiple values in the same column. The example given in the screenshot below shows the search for data with connection addresses of either www.google.com or www.youtube.com.
[ ] Square Brackets
If a numerical value, date or IP address range value needs to be filtered while searching the data, the square bracket must be used. When the square bracket operator is used, the starting and ending values given for value ranges are also included in the search. In the examples given in the screenshots below, a query that can search for a date range (between April 19, 2023, 05:00:00 and April 19, 2023, 07:59:59), a numerical value range (Target port 0 to 9), and a sub-network range (10.10.9.0/24) has been used, respectively.
{ } Curly Brackets
The logic of using the curly bracket operator is exactly the same as the square bracket. The only difference between them is that the curly bracket operator excludes the starting and ending values when used. Same as the example of numerical value above.
* Asteriks
The Asterisk operator is used as a wildcard character. For example, in the given screenshot below, it brings all the connection addresses ending with ".net" regardless of what comes before the ".net" value.
" " Quotation Marks
If the search column value consists of a phrase rather than a single word, the quotation mark operator should be used to prevent spaces between the words from being interpreted as an AND operator. However, if the words are combined with any other symbol instead of a space, it is not mandatory to use quotation marks for those values. IP addresses can be given as an example of this. In the given example in the screenshot below, data with the application name Microsoft Office 365 is filtered, and the value searched is written between quotation marks to be interpreted as a single value.
During the search, all the examples above were searched by entering the main and sub-column information, and the desired filtering was performed. However, it is also possible to search by only entering the value without specifying any column, and all logical operators can be used in this way. In the given examples in the screenshot below, in the first example, the search was performed by only writing "Microsoft" in the search field, and all data containing the word Microsoft was filtered and displayed on the screen regardless of which column it appeared in. In the second example, only the use of logical expressions with the searched value was mentioned.