Logs and SIEM Basics
how scattered records become a story you can search and alert on
Almost everything a computer does can leave a record: a sign-in, a program starting, a file being changed, a network connection. Individually these logs are dull. Together, in one place and in order, they let you reconstruct what happened.[1]
Useful log sources
- Identity: directory and single sign-on logs, VPN, cloud sign-ins.
- Endpoints: Windows event logs, Sysmon, Linux auth and system logs, endpoint security tools.
- Network: firewall, proxy, DNS, intrusion detection, flow data.
- Applications: web server, database and application audit logs.
- Cloud: audit trails of administrative and API activity.
- Email security and other security tools.
What a SIEM does
A security information and event management (SIEM) system:
- Collects logs from many sources (agents, syslog, APIs).
- Normalizes and parses them into common fields (user, source IP, action, time), so a failed login looks the same whichever system reported it.
- Stores and indexes them for searching, with retention to meet investigation and compliance needs.
- Correlates events across sources to find patterns no single log shows.
- Alerts analysts and supports investigation with search, dashboards and case tracking.
Examples of detections
- Many failed sign-ins followed by a success from the same address (password spraying or brute force).
- A user signing in from two distant countries within minutes ("impossible travel").
- A new administrator account created outside a change window.
- An office application launching a command shell.
- A server suddenly sending large volumes of data to an unknown address.
What makes it work
- Log what matters. Collect enough to answer investigative questions without drowning in noise or cost. Decide use cases first, then sources.
- Accurate time. Keep clocks synchronized (NTP) or timelines will not line up.
- Protect the logs. Restrict who can alter or delete them, and send copies off the machine, since attackers often try to erase their traces.
- Quality control. Watch for sources that quietly stop sending data.
- Tune and test detections, and document what each one means and how to respond.[2]
- Retention. Keep logs long enough: attackers can be inside for weeks or months.
Pitfalls
Collecting everything and detecting nothing, alerts nobody owns, parsing errors that hide key fields, and treating the SIEM as a set-and-forget product.
Getting started
Even a modest setup is valuable: central sign-in, firewall and endpoint logs, a handful of well-understood detections, and a routine of reviewing them.
Sources
Test yourself
My notes
Private — saved only on this device, never sent anywhere.