Security Operations & Resilience

Logs and SIEM Basics

how scattered records become a story you can search and alert on

2 min readupdated by Syed Khaleel Ahmed

Almost everything a computer does can leave a record: a sign-in, a program starting, a file being changed, a network connection. Individually these logs are dull. Together, in one place and in order, they let you reconstruct what happened.[1]

Useful log sources

  • Identity: directory and single sign-on logs, VPN, cloud sign-ins.
  • Endpoints: Windows event logs, Sysmon, Linux auth and system logs, endpoint security tools.
  • Network: firewall, proxy, DNS, intrusion detection, flow data.
  • Applications: web server, database and application audit logs.
  • Cloud: audit trails of administrative and API activity.
  • Email security and other security tools.

What a SIEM does

A security information and event management (SIEM) system:

  1. Collects logs from many sources (agents, syslog, APIs).
  2. Normalizes and parses them into common fields (user, source IP, action, time), so a failed login looks the same whichever system reported it.
  3. Stores and indexes them for searching, with retention to meet investigation and compliance needs.
  4. Correlates events across sources to find patterns no single log shows.
  5. Alerts analysts and supports investigation with search, dashboards and case tracking.

Examples of detections

  • Many failed sign-ins followed by a success from the same address (password spraying or brute force).
  • A user signing in from two distant countries within minutes ("impossible travel").
  • A new administrator account created outside a change window.
  • An office application launching a command shell.
  • A server suddenly sending large volumes of data to an unknown address.

What makes it work

  • Log what matters. Collect enough to answer investigative questions without drowning in noise or cost. Decide use cases first, then sources.
  • Accurate time. Keep clocks synchronized (NTP) or timelines will not line up.
  • Protect the logs. Restrict who can alter or delete them, and send copies off the machine, since attackers often try to erase their traces.
  • Quality control. Watch for sources that quietly stop sending data.
  • Tune and test detections, and document what each one means and how to respond.[2]
  • Retention. Keep logs long enough: attackers can be inside for weeks or months.

Pitfalls

Collecting everything and detecting nothing, alerts nobody owns, parsing errors that hide key fields, and treating the SIEM as a set-and-forget product.

Getting started

Even a modest setup is valuable: central sign-in, firewall and endpoint logs, a handful of well-understood detections, and a routine of reviewing them.

Sources

  1. NIST SP 800-92: Guide to Computer Security Log Management
  2. CISA: Best Practices for Event Logging and Threat Detection

Test yourself

My notes

Private — saved only on this device, never sent anywhere.