Creating Event Database Archives

Online v. Offline Storage

Setting Purge and Archive Policies

Archive and Purge Alerts

Online v. Offline Storage

The FortiSIEM event database, eventDB, is for near-to-intermediate term storage and querying of events. As an online database, eventDB has fast query performance, but this performance comes with a limited storage capacity, and is expensive in terms of resource consumption. For these reasons, data needs to be periodically purged from eventDB and moved into offline storage, but still be available for querying for forensic analysis. FortiSIEM checks the capacity of the online EventDB storage every 30 minutes, and when approaches capacity, begins to move event information, in daily increments, into the offline storage location.

The FortiSIEM virtual appliance includes a data archiving function that enables you to define an offline storage location, and a policy for the number of days that events will be kept in online or offline storage. This archiving function also includes the ability for compliance auditors to validate logs to ensure that they haven’t been tampered with in the offline storage. The data is cryptographically signed (SHA256) at the point of entry, and the checksums are stored in the database. The check sums can be re-verified on demand at any point of time, and if the data has been tampered with, then the check sums will not match. The data integrity reports can be exported in PDF format. If the events in offline storage need to be queried at some point in the future, they can be restored to the FortiSIEM virtual appliance.

Setting Purge and Archive Policies

Online data is only moved to the archive location when online storage reaches capacity. When you set the archive policy as described in Managin g Event Data Archive, you are setting the amount of time that archived data will be retained before it is purged. For example, if you set the Data Management Policy for your deployment or an organization to 90 days, then maintenance will run every day to purge data that is over 90 days old. If there is not enough offline storage for 90 days, then archived events will be purged from offline storage to create more capacity. If there is enough storage for the 90 days, then events will only be purged after 90 days. For this reason it is very important that you set an archive location that has sufficient capacity to store the amount of data for the number of days that you specify.

For multi-tenant deployments, you can set archive policies for each organization. If one organization requires 30 days of storage, and another customer requires 90 days of storage, then FortiSIEM will attempt to enforce these policies in relation to the amount of storage available. For the first organization, events will be deleted from the archive storage location on the 31st day to free up capacity for the organization that has longer storage requirements.

As with the online EventDB data, every 30 minutes FortiSIEM will check the capacity of the offline archive storage, and when the remaining storage capacity reaches a 20GB threshold, it will begin to purge data from the archive location, beginning with the oldest data, and purging it in daily increments, until the remaining storage capacity is above 20GB.

Archive and Purge Alerts

There are several system alerts that are related to eventDB capacity and the archiving function:

Alert	Description
Online event database close to full (below 20GB)	When the database reaches a point where the remaining storage capacity is below 20GB, its contents will be purged or archived, depending on whether an archive storage location has been defined
Event Archive started	The archive process has been initiated
Event Archive failed	The archive process has failed, likely due to a lack of capacity in the offline storage location
Event Archive purged because of archive purging policy	The contents of the event archive have been purged from offline storage according to the archive purging policy
Event Archive purged because it is full	The contents of the event archive have been purged from offline storage due to capacity issues

Managing Event Data Archive

Managing Online Event Data

Restoring Archived Data Validating Log Integrity

Importing and Exporting CMDB Report Definitions

Instead of using the user interface to define a report, you can import report definitions, or you can export a definition, modify it, and import it back into your FortiSIEM virtual appliance. Report definitions follow an XML schema.

Importing a Report Definition

1. Log in to your Supervisor node.
2. Go to CMDB > CMDB Reports.
3. Select the folder where you want to import the report definition, or create a new one.
4. Click Import.
5. Copy your report definition into the text field, and then click Import.

Exporting a Report Definition

1. Log in to your Supervisor node.
2. Go to CMDB > CMDB Reports.
3. Select the report you want to export, and then click Export.
4. Click Copy to Clipboard.
5. Paste the report definition into a text editor, modify it, and then follow the instructions for importing it back into your virtual appliance.

XML Schema for Report Definitions

 

Importing a CMDB Report Definition

1. Go to Report listing page and select the CMDB Report folder where the report is to be imported.
2. Click Import and see the report showing up in the correct folder.

Exporting a CMDB Report Definition

 

Creating and Modifying CMDB Reports

There are two ways you can create new CMDB reports: you can create a new report from scratch, or you can clone and modify an existing system or user-defined report.

Creating a New Report

1. Log in to your Supervisor node.
2. Go to CMDB > CMDB Reports.
3. Create a group to add the new report to if you are not adding it to an existing group.
4. Click New.
5. Enter a Name and Description for the report.
6. Select the Conditions for the report.

You can use parentheses to give higher precedence to evaluation conditions.

7. Select the Display Columns.

The Display Column attributes contain an implicit “group by” command. You can change the order of the columns with the Move Row:

Up and Down buttons.

8. Click Save.

Cloning and Modifying a Report

You can modify user-defined reports by selecting the report and clicking Edit. However, you cannot directly edit a system-defined report. Instead, you have to clone it, then save it as a new report and modify it.

1. Log in to your Supervisor node.
2. Go to CMDB > CMDB Reports.
3. Select the system-defined you want to modify, and then click Clone.
4. Enter a name for the new report, and then click Save.

The cloned report will be added to the folder of the original report.

5. Select the new report, and then click Edit.
6. Edit the report, and then click Save.

 

 

Reporting on CMDB Objects

All of the information in the CMDB can be reported on. FortiSIEM includes a number of pre-defined reports that you can run and export to PDF, and you can also create your own reports.

CMDB Report Types

Running, Saving, and Exporting a CMDB Report

Creating and Modifying CMDB Reports

Importing and Exporting CMDB Report Definitions

CMDB Report Types

You can find all system-defined reports in CMDB > CMDB Reports. The reports are organized into folders as shown in this table. Click on a report to view Summary information about it, including the report conditions and the columns included in the report.

Report and Organization Associations for Multi-Tenant Deployments

If you have an FortiSIEM multi-tenant deployment, the Organization column in the CMDB report table will show whether the report is defined for a specific organization. If it is, then that report is available for both the organization and Super/Global users.

CMDB Report Folder	Object to Report On	Report Name
Overall	Device Approval Status	Approved Devices Not Approved Devices
	Users	Discovered Users Externally Authenticated FortiSIEM Users Locally Authenticated FortiSIEM Users Manually Defined Users
	Rules	Active Rules Rules with Exceptions
	Reports	Scheduled Reports
	Performance Monitors	Active Performance Monitors
	Task	All Existing Tasks
	Business Service	Business Service  Membership
Network	Inventory	Network Device Components with Serial Number Network Interface Report Router/Switch Inventory Router/Switch Image Distribution
	Ports	Network Open Ports
	Relationship	WLAN-AP Relationship
Server	Inventory	Server Inventory Server OS Distribution Server Hardware: Processor Server Hardware: Memory and Storage
	Ports	Server Open Ports
	Running Services	Windows Auto Running Services Windows Auto Stopped Services Windows Exchange Running Services Windows IIS Running Services Windows Manual Running Services Windows Manual Stopped Services Windows SNMP Running Services Windows VNC Running Services Windows WMI Running Services

 

	Installed Software / Patches	Windows Installed Software Windows Installed Patches Windows Installed Software Distribution
Virtualization	Relationship	VM-ESX Relationship

 

 

Running, Saving, and Exporting a CMDB Report

1. Log in to your Supervisor node.
2. Go to CMDB > CMDB Reports, and select the report you want to run.
3. Click Run.
4. If you have a multi-tenant deployment, you will be prompted to select the organizations for which you want to run the report.
5. Click Save if you want to save the report.

Reports are only saved for the duration of your login session, and you can view saved reports by clicking Report Results. Each saved report will be listed as a separate tab, and you can delete them by clicking the X that appears when you hover your mouse over the report name in the tab. You can save up to 5 reports per login session

Watch Lists

A Watch List is a smart container of similar items such as host names, IP addresses, or user names, that are of significant interest to an administrator and need to be watched. Examples of watch lists that are already set up in FortiSIEM are

Frequent Account Lockouts – users who are frequently locked out

Host Scanners – IP addresses that scan other devices

Disk space issues – hosts with disks that are running out of capacity

Denied countries – countries with an excessive number of access denials at the firewall

Blacklisted WLAN endpoints – Endpoints that have been blacklisted by Wireless IPS systems

Typically items are added to a watch list dynamically when a rule is triggered, but you can also add items to a watch list manually. When you define a rule, you can also choose a watch list that will be populated with a specific incident attribute, as described in Adding a Watch List to a Rule, and you can use watch lists as conditions when creating reports, as described in Using Watch Lists as Conditions in Rules and Reports. Yo u can also define when an entry leaves a watch list. Typically this is time based. For example, if the rule does not trigger for that attribute for defined time-period, then the entry is removed from the watch list. Watch lists are also multi-tenant aware, with organization IDs tracked in relation to watch list items.

Creating a Watch List

System-Defined Watch Lists

Related Links

Using Watch Lists as Conditions in Rules and Reports

Adding a Watch List to a Rule

Overview of the CMDB User Interface

 

Creating a Watch List

1. Log in to your Supervisor node.
2. Go to CMDB > Watch Lists.
3. Click +.
4. Choose an Organization to associate with the watch list.
5. Enter a Group name and Description for the watch list.
6. Select an object Type for the incident attribute that will be saved to the watch list.
7. Select Case Sensitive if the object type is String and you want to use case sensitivity to compare strings.
8. For Values Expire in, set the time period in which items will expire from the watch if there is no activity for that time.
9. Click OK.

You can now add your new watch list to a rule, so that when the rule is triggered, items will be added to the watch list. You can also use your watch list as a condition in historical search. See Adding a Watch List to a Rule and Using Watch Lists as Conditions in Rules and Reports for more information.

Related Links

Adding a Watch List to a Rule

Using Watch Lists as Conditions in Rules and Reports

 

System-Defined Watch Lists

FortiSIEM includes several pre-defined watch lists that are populated by system-defined rules.

Watch list	Description	Attribute Type	Triggering Rules
Accounts Locked	Domain accounts that are locked out frequently	User (STRING)	Account Locked: Domain

 

 

Application Issues

Applications exhibiting issues

Host Name (STRING)

IIS Virtual Memory Critical SQL Server Low Buffer Cache Hit Ratio SQL Server Low Log Cache Hit Ratio SQL Server Excessive Deadlock SQL Server Excessive Page Read/Write SQL Server Low Free Pages In Buffer Pool SQL Server Excessive Blocking Database Server Disk Latency Critical SQL Server Excessive Full Scan SQL Server scheduled job failed High Oracle Table Scan Usage High Oracle Non-System Table Space Usage Oracle database not backed up for 1 day Exchange Server SMTP Queue High Exchange Server Mailbox Queue High Exchange Server RPC Request High Exchange Server RPC Latency High Oracle DB Low Buffer Cache Hit Ratio Oracle DB Low Library Cache Hit Ratio Oracle DB Low Row Cache Hit Ratio Oracle DB Low Memory Sorts Ratio Oracle DB Alert Log Error Excessively Slow Oracle DB Query Excessively Slow SQL Server DB Query Excessively Slow MySQL DB Query

 

Availability Issues	Servers, networks or storage devices or Applications that are exhibiting availability issues	Host Name (STRING)	Network Device Degraded – Lossy Ping Response Network Device Down – No Ping Response Server Degraded – Lossy Ping Response Server Down – No Ping Response Server Network Interface Staying Down Network Device Interface Flapping Server Network Interface Flapping Important Process Staying Down Important Process Down Auto Service Stopped Critical network Interface Staying Down EC2 Instance Down Storage Port Down Oracle Database Instance Down Oracle Listener Port Down MySQL Database Instance Down SQL Server Instance Down Service Staying Down – Slow Response To STM Service Down – No Response to STM Service Staying Down – No Response to STM
DNS Violators	Sources that send excessive DNS traffic or send traffic to unauthorized DNS gateways	Source IP	Excessive End User DNS Queries to Unauthorized DNS servers Excessive End User DNS Queries Excessive Denied End User DNS Queries Excessive Malware Domain Name Queries Excessive uncommon DNS Queries Excessive Repeated DNS Queries To The Same Domain

 

Denied Countries	Countries that are seeing a high volume of denials on the firewall	Destination Country (STRING)	Excessive Denied Connections From An External Country
Denied Ports	Ports that are seeing a high volume of denies on the firewall	Destination Port (INT)	Excessive Denied Connection To A Port
Environmental Issues	Environmental Devices that are exhibiting issues	Host name (String)	UPS Battery Metrics Critical UPS Battery Status Critical HVAC Temp High HVAC Temp Low HVAC Humidity High HVAC Humidity Low FPC Voltage THD High FPC Voltage THD Low FPC Current THD High FPC ground current high NetBoz Module Door Open NetBotz Camera Motion Detected Warning APC Trap Critical APC Trap
Hardware Issues	Servers, networks or storage devices that are exhibiting hardware issues	Host Name (String)	Network Device Hardware Warning Network Device Hardware Critical Server Hardware Warning Server Hardware Critical Storage Hardware Warning Storage Hardware Critical Warning NetApp Trap Critical Network Trap
Host Scanners	Hosts that scan other hosts	Source IP	Heavy Half-open TCP Host Scan Heavy Half-open TCP Host Scan On Fixed Port Heavy TCP Host Scan Heavy TCP Host Scan On Fixed Port Heavy UDP Host Scan Heavy UDP Host Scan On Fixed Port Heavy ICMP Ping Sweep Multiple IPS Scans From The Same Src

 

Mail Violators	End nodes that send too much mail or send mail to unauthorized gateways		Excessive End User Mail to Unauthorized Gateways Excessive End User Mail
Malware Found	Hosts where malware found by Host IPS /AV based systems and the malware is not remediated	Host Name (String)	Virus found but not remediated Malware found but not remediated Phishing attack found but not remediated Rootkit found Adware process found
Malware Likely	Hosts that are likely to have malware – detected by network devices and the determination is not as certain as host based detection	Source IP or Destination IP	Excessive Denied Connections From Same Src Suspicious BotNet Like End host DNS Behavior Permitted Blacklisted Source Denied Blacklisted Source Permitted Blacklisted Destination Denied Blacklisted Destination Spam/malicious Mail Attachment found but not remediated Spyware found but not remediated DNS Traffic to Malware Domains Traffic to Emerging Threat Shadow server list Traffic to Emerging Threat RBN list Traffic to Emerging Threat Spamhaus list Traffic to Emerging Threat Dshield list Traffic to Zeus Blocked IP list Permitted traffic from Emerging Threat Shadow server list Permitted traffic from Emerging Threat RBN list Permitted traffic from Emerging Threat Spamhaus list Permitted traffic from Emerging Threat Dshield list Permitted traffic from Zeus Blocked IP list

 

 

Port Scanners	Hosts that scan ports on a machine	Source IP	Heavy Half-open TCP Port Scan: Single Destination Heavy Half-open TCP Port Scan: Multiple Destinations Heavy TCP Port Scan: Single Destination Heavy TCP Port Scan: Multiple Destinations Heavy UDP Port Scan: Single Destination Heavy UDP Port Scan: Multiple Destinations
Policy Violators	End nodes exhibiting behavior that is not acceptable in typical Corporate networks	Source IP	P2P Traffic detected IRC Traffic detected P2P Traffic consuming high network bandwidth Tunneled Traffic detected Inappropriate website access Inappropriate website access – multiple categories Inappropriate website access – high volume Inbound clear text password usage Outbound clear text password usage Remote desktop from Internet VNC From Internet Long lasting VPN session High throughput VPN session Outbound Traffic to Public DNS Servers
Resource Issues	Servers, networks or storage devices that are exhibiting resource issues: CPU, memory, disk space, disk I/O, network I/O, virtualization resources – either at the system level or application level	Host Name (STRING)	High Process CPU: Server High Process CPU: Network High Process Memory: Server High Process Memory: Network Server CPU Warning Server CPU Critical Network CPU Warning Network CPU Critical Server Memory Warning Server Memory Critical

Network Memory Warning

Network Memory Critical

Server Swap Memory Critical

Server Disk space Warning

Server Disk space Critical

Server Disk Latency Warning

Server Disk Latency Critical

Server Intf Util Warning

Server Intf Util Critical

Network Intf Util Warning

Network Intf Util Critical

Network IPS Intf Util Warning

Network IPS Intf Util Critical Network Intf Error Warning

Network Intf Error Critical Server Intf Error Warning

Server Intf Error Critical

Virtual Machine CPU Warning

Virtual Machine CPU Critical

Virtual Machine Memory

Swapping Warning

Virtual Machine Memory

Swapping Critical

ESX CPU Warning

ESX CPU Critical

ESX Memory Warning

ESX Memory Critical

ESX Disk I/O Warning

ESX Disk I/O Critical

ESX Network I/O Warning

ESX Network I/O Critical Storage CPU Warning

Storage CPU Critical

NFS Disk space Warning

NFS Disk space Critical

NetApp NFS Read/Write

Latency Warning

NetApp NFS Read/Write Latency Critical

NetApp CIFS Read/Write

Latency Warning

			NetApp CIFS Read/Write Latency Critical NetApp ISCSI Read/Write Latency Warning NetApp ISCSI Read/Write Latency Critical NetApp FCP Read/Write Latency Warning NetApp FCP Read/Write Latency Critical NetApp Volume Read/Write Latency Warning NetApp Volume Read/Write Latency Critical EqualLogic Connection Read/Write Latency Warning EqualLogic Connection Read/Write Latency Critical Isilon Protocol Latency Warning
Routing Issues	Network devices exhibiting routing related issues	Host Name (STRING)	OSPF Neighbor Down EIGRP Neighbor down OSPF Neighbor Down
Scanned Hosts	Hosts that are scanned	Destination IP	Half-open TCP DDOS Attack TCP DDOS Attack Excessive Denied Connections to Same Destination
Vulnerable Systems	Systems that have high severity vulnerabilities from scanners	Host Name (STRING)	Scanner found severe vulnerability
Wireless LAN Issues	Wireless nodes triggering violations	MAC Address (String)	Rogue or Unsecure AP detected Wireless Host Blacklisted Excessive WLAN Exploits Excessive WLAN Exploits: Same Source

 

 

Users

The CMDB Users page contains information about users of your system. For more information about adding users, see Adding a Single User.

User Agents

The CMDB User Agent page lists common and uncommon user agents in HTTP communications. The traditional use case for a user agent is to detect browser types so the server can return an optimized page. However, user agents are often misused by malware, and are used to communicate the identity of the client to the BotNet controller over HTTP(S). FortiSIEM monitors HTTP(S) logs and the system rule Blacklist User Agent Match uses regular expression matching to detect blacklisted user agents.

Adding User Agents

1. Log in to your Supervisor node.
2. Go to CMDB > User Agents.
3. Select the User Agent group where you want to add the new user agent.
4. Click New.
5. Enter the User Agent using regular expression notation.

Protocols

The CMDB Protocols page lists the protocols used by applications and devices to communicate with the FortiSIEM virtual appliance.

Adding a Protocol

1. Log in to your Supervisor node.
2. Go to CMDB > Protocols.
3. Create a new protocol group or select an existing one.
4. Click New.
5. Enter an Name and Description for the protocol.
6. Click + to select a protocol and associate it with a port 7. Select or create an Apps Group to associate with the protocol.
7. Click Save.