Logsign Unified SecOps Platform, with more than 200 prepared integrations, collects logs in real-time from hundreds of data sources. Optimization is required on Logsign Unified SecOps Platform as the logs continuously increase. Logsign Unified SecOps Platform DPM (Data Policy Manager) enables you to optimize log management.
- To collect logs from a source or grouped log sources,
- To manage the redundancy durations of the logs,
- To facilitate the management of our storage capacity,
You can create data policies with Logsign Unified SecOps Platform DPM and increase the data collection, storage, and indexing performance.
While adding a source to Logsign Unified SecOps Platform, the configurations included in the Default Policy, as seen on the Data Policy tab, are adjusted as 'select all' and can be altered by the user. It allows a separate Data Policy to be created for each source. You shall deepen the Data Policy issue and see what you can do in general.
Click the Settings > Integrations > Data Policies tab on the Logsign Unified SecOps Platform WEB interface. You will see the Data Policy adjusted as default under the name Default on the page. The data load of the incidents on the relevant source will be relieved if you create a separate Data Policy for each source.
Moreover, you can edit but cannot erase the Default Data Policy rule on this page, the default on the Logsign Unified SecOps Platform. In addition, you can apply the Default Policy rule to all sources.
When you click the + Policy button on the upper right corner of the page, you'll see the screen where you can configure the Data Policy you want to create.
Policy Name: You can determine a name relevant to the source with which it will be correlated. (Example: WindowsDataPolicy).
For Input
In this section, you can include or exclude the incident movements that are about to be placed on Logsign Unified SecOps Platform. (by indicating the Regexp (Regular Expression – a useful software language enabling us to find a specific expression according to a specific pattern) or Key-Value (the section at which you can make screening according to the incident movements by working at the WMI level).)
On the For Input section, you'll be able to select the incident movements you do not want on Logsign Unified SecOps Platform. In other words, you can drop the incident movements you do not wish on Logsign Unified SecOps Platform at the first level.
Include by regexp: After the box next to it is clicked, another row named Add Regexp will be added to the row below. The section is where you can select the incident movements you want to be written on Logsign Unified SecOps Platform from the relevant source. There are two issues to be paid attention to in this section. Firstly, you can enter specific input using Regexp language or type the word on the relevant row without using Regexp language if it is only a word (ex: malware). If your input is an IP, it will not make the relevant query when you type 10.0.0.10 because a command should be added with Regexp logic if the word you think of typing includes symbols (.,-,-$, etc.).
You can use the Regexp examples below:
"192\.168\.1\.1": When you need to enter the IP address
"wareztool. ": It can be matched with expressions such as "Wareztools, wareztoolz, wareztoolx"
"Li[cs]ense" : It checks two expressions - “License or Lisense”
"malware|trojan": It looks for a match with the words "malware or trojan"
"cain (download|indir)": It checks the expression "cain download or cain indir"
"h3808073@mvrht\.com" : It looks for the e-mail address h3808073@mvrht.com
The examples above are the queries written with Regexp logic and are not limited. After typing the input, you want to add, click the Add button to add it to the list. Therefore, you will have the chance to add more than one regexp.
Please note: This section may be more valuable when selected as Exclude by regexp row "*" (without the quotation marks). It will be clarified in the Exclude by regexp section.
Exclude by regexp: After the box next to it is clicked, another row named Add Regexp will be added to the row below. The section is where you can select the incident movements you do not want to be written on Logsign Unified SecOps Platform from the relevant source.
Once the Exclude by regexp field is not selected, you do not need to add the Include by regexp section, as Logsign Unified SecOps Platform includes the incident movement as a whole as default. However,
- if you have a source on your system and you are familiar with the incident movements coming from this source; and
- if you do not want to select the irrelevant incident movements coming in between,
you can place the "*" symbol on the Exclude by regexp section (without the quotation marks) and arrange it as do not select incident movements. Then, you can only add the incident movements you want to select on the Include by regexp row.
Include by Key-Value: Logsign Unified SecOps Platform only works with WMI incident movements on the For Input section. You can select the desired incident movements by giving key-value values.
Exclude by Key-Value: Logsign Unified SecOps Platform only works with WMI incident movements on the For Input section. You can select the undesired and irrelevant incident movements by giving key-value values.
Syslog Redundancy Period: It allows you to select only one of the same incidents occurring at a particular time. Identification is based on seconds.
Field Redundancy
Configurations to be made in this section will affect the JSON (Parsed) Store field. You will see a few rows after clicking the Add button.
Period: You can give values in terms of seconds in this section. For the fields you will choose on the Fields row, we can select only one incident movement if more than one of the same within x seconds.
Fields: It is the section where you can select the areas where you can give the "do not keep on JSON" command if there are more than one of the same incident movements within x seconds,
Collect Fields: If the box next to it is clicked, only the fields you selected on the Collection Fields section will be written on Index.
Collection Fields: The fields selected in this section are written on the Index.
When you click the Save button, your settings are saved. You can create more than one rule by repeating the same process.
For Indexing
In this section, you can determine the incident movements you would like to be written on the Index.
Filter Index Fields: If the box next to it is clicked, the relevant setting will be activated, and the rows below will appear.
Index Fields: It is a helpful feature to make only the selected fields from the relevant source written on the Index. It allows you to not write on Index the unselected areas and not see the irrelevant incident movements.
Select Filter From Fieldset: If the box is clicked, it may be convenient for you to select from the default data sets (fieldset). The items arranged as a dataset (Mail Server, System, WAF, etc.) include various fields. Therefore, you will not need to select areas separately.
Include Logs: If the box next to it is clicked, it will be added as an additional row. You shall see what you can do in this field.
SystemID List: "*" for all: It is the section you can determine which incident movements you'd like to be written on Index from the relevant source. You shall exemplify the definition we call SystemID.
On the image, you see above, accessed the Object Access incident movement by using the Search platform. If you click on the relevant incident movement, a window will be opened where you can see its details. The number opposite is called the SystemID row under the Event column. You can filter on the Include Logs section the desired incident movements you'd like to be written on Index by using SystemID.
On SystemID List, write the incident movement you would like written on Index and add it with the Add button. You can add more than one SystemIDs.
Exclude Logs: It can be used to prevent the undesired and irrelevant incident movements from the relevant source to be written on the Index. After the addition is made with the "*" symbol (do not write any incident movements on Index), you can write on Index only the incident movements you consider to be important on the Include Logs section. For instance, if you have a source with a low traffic level and you are interested in only a part of the incident movements coming from this source, you can write on Index only the incident movements you consider to be important on the Include Logs section after making the setting as "*" on Exclude Logs.
Include by regexp: After the box next to it is clicked, another row named Add Regexp will be added to the row below. It is the section where you can select the incident movements you want to be written on Logsign Unified SecOps Platform from the relevant source.
You can use the Regexp examples below:
"192\.168\.1\.1": When you need to enter the IP address
"wareztool. ": It can be matched with expressions such as "Wareztools, wareztoolz, wareztoolx"
"Li[cs]ense" : It checks two expressions - “License or Lisense”
"malware|trojan": It looks for a match with the word "malware or trojan"
"cain (download|indir)": It checks the expression "cain download or cain indir"
"h3808073@mvrht\.com" : It looks for the e-mail address h3808073@mvrht.com
Exclude by regexp: After the box next to it is clicked, another row named Add Regexp will be added to the row below. The section is where you can select the incident movements you do not want to be written on Logsign Unified SecOps Platform from the relevant source.
Once the Exclude by regexp field is not selected, you do not need to add the Include by regexp section.
Include by Key-Value: It writes the incident movements of the fields you will determine from the relevant source on the Index.
Exclude by Key-Value: It allows you to select field-based incident movements you do not want to be written on Index from the relevant source. Similarly, you can determine the incident movements you consider essential on the Include section after the setting is made as Exclude and "*."
For JSON (Parsed) Store
This section shows the configurations we can make for the extracted incident movements on the Logsign Unified SecOps Platform archive.
Filter Index Fields: If the box next to it is clicked, you'll activate the relevant setting, and the additional row below will appear.
Index Fields: It is a feature you can keep on the archive only the selected fields from the relevant source. You do not store the irrelevant incident movements by not holding the areas you did not choose on the archive.
Include Logs: If the box next to it is clicked, the row below will be added.
SystemID List: "*" for all: It is the section we can determine which incident movements you'd like to be stored on the archive from the relevant source. Exemplify the definition called SystemID.
You see above the image of the incident movement of a process within the Login category on the Search field. Suppose this is an incident you'd like to be kept as JSON, write the SystemID number on the SystemID List row and save it by clicking the Add button. With this process, incident movements related to the SystemID number added will not be written on the JSON field. Similarly, you can add more than one SystemIDs.
Exclude Logs: On this row, the settings made on the Include section are valid. After adding it as the "*" symbol, you can only keep the beneficial incident movements on the JSON field on the Include section.
Include by Key-Value: It keeps on JSON the incident movements of the fields you will determine from the relevant source.
Exclude by Key-Value: It allows you to select field-based incident movements you do not want to be kept on JSON from the relevant source. Similarly, you can determine the incident movements you consider essential on the Include section after the setting is made as Exclude and "*."
For RAW Store
In this field, you will see the configurations you can make on the raw versions (unprocessed, straight from the source) of the incident movements.
Include by regexp: After the box next to it is clicked, another row named Add Regexp will be added to the row below. It is the section where you can select the incident movements you want to be written on the RAW file from the relevant source.
You can use the Regexp examples below:
"192\.168\.1\.1": When you need to enter the IP address
"wareztool. ": It can be matched with expressions such as "Wareztools, wareztoolz, wareztoolx"
"Li[cs]ense" : It checks two expressions - “License or Lisense”
"malware|trojan": It looks for a match with the word "malware or trojan"
"cain (download|indir)": It checks the expression "cain download or cain indir"
"h3808073@mvrht\.com" : It looks for the e-mail address h3808073@mvrht.com
Exclude by regexp: After the box next to it is clicked, another row named Add Regexp will be added to the row below. It is the section where you can select the incident movements you do not want to be written on the RAW file from the relevant source.
Once the Exclude by regexp field is not selected, you do not need to add the Include by regexp section.
For Persist
In this section, you can create a log file peculiar to the source whose rules you configured.
Persist group file: If you click the box next to it, you'll activate the relevant setting, and a row will be added below. On this row, you can create the file on JSON by giving it a definition name that is peculiar to the source of the Data Policy you configured.
For instance, while configuring the Data Policy of your Windows server, if you type Windows on the Persist Group file text, a file with that name will be created on the JSON field and the incident movements will be written on that file.