In parts 1 and 2 of this three-part series, we answered the question, ‘What is SIEM?’ and covered the detection, response and recovery process following a cyber attack.
» What is SIEM? (Part 1): Cyber Security 101
» What is SIEM? (Part 2): Detection, Response & Recovery
» What is SIEM? (Part 3): How Does SIEM work?
Part 3 will examine how SIEM platforms analyse log data to uncover suspicious activity during a cyber attack. So, let’s discuss how SIEM works.
The SIEM collects and processes data. The log, also known as a log file, is at the core of the data. The log is a data file of a machine or application’s operation. Log file types include FlatFile, SysLog, Microsoft event logs, File Integrity Monitoring (FIM) data, UDLA (Universal Database Log Adaptor), SDEE (Security Device Event Exchange), Checkpoint data, and more.
Logs provide data on a device’s activity, health, user interaction with the device, and application performance and activity.
The SIEM collects log data from various sources, including data related to systems, networks, network devices, and applications. The collected log data could be related to security, unauthorised activities, misuse of company resources, user activity, file integrity and many other common computing activities.
A log provider is a device or application that creates and provides log files. Common devices and applications that give log data are:
Microsoft Windows, UNIX or Linux, and AS400 servers; firewalls; switches and Routers; Intrusion Prevention Systems or Intrusion Detection Systems; and application servers, such as web, database, and email servers.
A log source is a type of log, or a unique log originating from a specific host.
For example, a Windows server would include three log sources by default. One for each event log type:
These devices and applications in your network generate thousands of raw logs every moment. The logs are generated in various formats and represent varying degrees of importance in maintaining and protecting your network.
Logs typically take the form of log messages. Since logs are generated from many different types of systems, there are many log messages. This log data isn’t always easily deciphered when viewed in its raw state!
So, what happens to the log message and the data within?
The log message is typically subject to several processes with the leading’ Next-Gen’ SIEM platforms.
The log message is “time normalised” to ensure timestamps reflect the same time zone. Then, metadata extraction and tagging are performed, actively parsing and breaking down the log message into core components.
Then every identified log message is assigned a classification and a common event to help define the type of activity described in the message. Threat and risk contextualisation may also be performed to evaluate each log and provide a risk-based priority value.
Metadata extraction and tagging actively parse and break down the log message into core components, identifying what kind of data the log message contains, the meaning behind the message, and the relative importance of the action described in the message.
Essentially, a SIEM platform takes the raw log message and parses it, making the data easy to use—the data parsed from the log message is called metadata.
Metadata is stored in databases using common metadata fields, so it can quickly be referenced for more efficient searching, allowing for in-depth reporting and analysis.
By parsing data into common metadata fields, searching for that data in the mass of log messages is easy using consistent terminology or language.
In addition to parsing metadata directly from the log message, a SIEM can derive some metadata from the logs. Derived metadata are fields that use parsed information in log messages to provide additional context about the message. Examples include entity origin, known host impact, and direction.
The leading SIEM platforms will assign every identified log message a Classification and a Common event.
Classifications define the broad range of activity. There are three main types of classification (with sub-classifications beneath):
Common events are not intended to be highly specific but provide a more descriptive context for the activity described in the log message.
By converting many types of log messages from many systems into a common language (or common metadata fields), a SIEM platform can compare like-for-like and correlate data that otherwise might not have been relatable (and therefore useful).
An Event is a more relevant log for operational, security or compliance. It typically includes logs classified as Errors, Failures, or Attacks.
Events are actionable log items only, meaning they are more significant than other logs. Actionable logs include login failures, hacking attempts and the granting of elevated permissions, as well as system-related items such as service or software failures and reboot messages.
Events typically make up 5% or less of the total logs collected. Once events have been identified, they are used for real-time monitoring, report compiling, investigation performance, and alarm definition.
A capable SIEM also aids in the creation and issuance of alarms and reports, often including a real-time alarming system to notify users and systems when certain criteria are met. In addition, the disturbing system should include smart response actions for responding to certain scenarios observed by the SIEM platform. Reports then allow users to generate detailed or summary reports based on log data collected by the platform.
As we’ve seen, SIEM platforms can seem complex. The capabilities and intelligence built into a SIEM are impressive, but this means a skills investment and complexity… for the users, support teams, and the organisation.
While businesses rely more and more on IT teams to deliver core business projects, day-to-day IT operations and maintain security—with limited resources and budgets—it is no wonder that many organisations have realised it is not viable to build their own fully staffed and resourced 24x7 Security Operations Centre (SOC) to secure their critical business information.
» What is SIEM? (Part 1): Cyber Security 101
» What is SIEM? (Part 2): Detection, Response & Recovery
» What is SIEM? (Part 3): How Does SIEM Work?
Managing the complexities of a SIEM platform, keeping pace with the latest security threats, and managing people, processes, and associated technologies is a tall order. This includes factoring in the time and cost to build, train and retain your 24x7 Security Operations Centre (SOC).
Whether fully outsourced Security or working in partnership with internal teams, an outsourced Security Operations Centre will help you quickly scale your security, keep pace with ever-changing threats, and ultimately ensure effective security outcomes at a lower cost than doing it yourself.