Security - Threat Detection

How Threat Detection Works: The Main Components

Threat detection isn't a single tool; it's a layered strategy involving several key components and techniques.

1. Data Collection (The Eyes and Ears)

You can't detect what you can't see. The first step is to collect vast amounts of data (telemetry) from every part of your software environment. This includes:

• Log Files: Application logs, web server logs (e.g., Nginx, Apache), database logs, operating system logs (Windows Event Logs, Linux syslog).
• Network Traffic: Packet captures, flow logs (like VPC Flow Logs in the cloud), DNS queries, firewall logs.
• Endpoint Data: Information from servers and employee laptops, such as running processes, file modifications, and system calls (collected by an EDR agent).
• Cloud Provider Logs: AWS CloudTrail, GCP Cloud Audit Logs, Azure Activity Log. These record every API call made in your cloud environment.
• Identity Provider Logs: Logs from services like Okta or Azure AD, showing who logged in, from where, and when.

2. Analysis Techniques (The Brain)

Once you have the data, you need to analyze it to find malicious patterns. There are several common techniques:

• Signature-Based Detection:

  • Analogy: Like antivirus software looking for a known virus signature.
  • How it works: It looks for known "bad" patterns, such as a specific malware file hash, a known malicious IP address, or a specific string in a network packet that matches a known exploit.
  • Pros: Very accurate for known threats. Low false positives.
  • Cons: Completely ineffective against new, unknown attacks (zero-day attacks).

• Behavioral-Based Detection / Anomaly Detection:

  • Analogy: Like noticing your quiet, predictable coworker suddenly starts trying to access the CEO's office at 3 AM.
  • How it works: First, the system establishes a "baseline" of what normal activity looks like. Then, it monitors for deviations from that baseline.
  • Examples of anomalies:
    • A user who normally logs in from New York suddenly logs in from North Korea.
    • A web server that normally only serves web traffic suddenly starts trying to connect to other servers on the network (lateral movement).
    • A developer's account suddenly starts trying to download massive amounts of data from a database it has never accessed before.
  • Pros: Can detect new and unknown (zero-day) attacks.
  • Cons: Can have a higher rate of false positives, as "unusual" doesn't always mean "malicious." Requires tuning.

• Threat Intelligence Correlation:

  • Analogy: Checking your security camera footage against a police bulletin of known burglars in the area.
  • How it works: The system takes data from your environment (like an IP address that connected to your server) and compares it against external feeds of known malicious indicators (Indicators of Compromise, or IoCs). These feeds are provided by cybersecurity firms and information-sharing communities.

3. Alerting and Triage (The Alarm Bell)

When a detection system finds something suspicious, it generates an alert. This is where the human element comes in.

• Alerting: The alert is sent to a central system and flagged for a security analyst.
• Triage: The analyst's job is to investigate the alert. Is it a true positive (a real attack) or a false positive (a benign activity that accidentally triggered a rule)?
• Prioritization: Alerts are prioritized based on their severity. A "potential ransomware activity" alert is much more critical than a "user logged in from a new location" alert.

4. Response (Putting out the Fire)

This is technically part of "Incident Response," but it's the direct result of detection. Once a threat is confirmed, the response could involve:

• Isolating the compromised machine from the network.
• Forcing a password reset for a compromised user account.
• Blocking a malicious IP address at the firewall.

Common Threat Detection Tools in Software

• SIEM (Security Information and Event Management): The central brain. Tools like Splunk, QRadar, or Chronicle collect and correlate logs from all over the environment to find threats.
• IDS/IPS (Intrusion Detection/Prevention System): Monitors network traffic for known attack patterns.
• EDR (Endpoint Detection and Response): An agent that runs on servers and laptops to monitor for malicious behavior at the process and file level. (e.g., CrowdStrike, SentinelOne).
• Cloud-Native Tools:
  • AWS GuardDuty: A managed threat detection service that analyzes AWS logs (CloudTrail, VPC Flow Logs, DNS logs) to find threats.
  • Azure Sentinel: Microsoft's cloud-native SIEM.
  • Google Security Command Center: Detects misconfigurations and threats in GCP.

Use YARA Rules for Threat Detection

Check YARA page.

Use eBPF for Threat Detection

Using eBPF (extended Berkeley Packet Filter) for threat detection is one of the most powerful and modern approaches in cybersecurity. It represents a fundamental shift from older, slower methods to real-time, in-kernel security monitoring.

Instead of just collecting logs and analyzing them later (the old way), eBPF allows you to run a tiny, secure, sandboxed program directly inside the Linux kernel whenever a specific event happens (like a file is opened or a network connection is made). This gives you unprecedented visibility and speed.

The workflow involves creating, loading, and listening to eBPF programs. This is typically done using high-level frameworks, as writing raw eBPF bytecode is extremely difficult.

Common Frameworks:

• BCC (BPF Compiler Collection): A Python-based toolkit for creating eBPF programs. Great for scripting and rapid prototyping.
• bpftrace: A high-level tracing language inspired by DTrace/Awk. Excellent for one-liners and ad-hoc analysis.
• Libbpf (with C): The modern, production-grade way to write eBPF programs. Tools often use a C skeleton for the kernel-space part and a user-space agent in Go, Rust, or C++ to manage it.

Here are concrete examples of threats you can detect using eBPF, categorized by the type of event you would "hook" into.

1. Detecting Malicious Process Execution

• The Goal: Detect when a suspicious program is run.
• eBPF Hook Point: The execve or execveat system calls. These are called every time a new program is executed.
• Detection Logic:
  1. Your eBPF program attaches to the execve syscall.
  2. Whenever any process is executed, your program gets triggered.
  3. It inspects the filename, the arguments, and the user who ran it.
  4. It can then check this information against a threat intelligence feed or a set of rules:
     • Rule: Is the filename ncat, mimikatz.exe, or other known hacking tools? -> Alert!
     • Rule: Is a web server process (like nginx or apache2) suddenly trying to execute /bin/bash? (This is a classic sign of a web shell.) -> Alert!
     • Rule: Is a process running from a suspicious directory like /tmp or /dev/shm? -> Alert!

2. Detecting Rootkit Activity

• The Goal: Detect attempts to load malicious kernel modules or hide processes.
• eBPF Hook Point: The init_module syscall (for loading kernel modules) or kernel-level functions (kprobes) that are involved in listing processes.
• Detection Logic:
  1. Attach an eBPF program to init_module.
  2. If this syscall is ever triggered on a production server, it's highly suspicious. -> Alert!
  3. A more advanced technique (used by tools like Tetragon) is to hook into both the user-space ps command's syscalls and the kernel's internal process list functions. If the lists don't match, it's a strong sign that a kernel rootkit is hiding a process. -> Alert!

3. Detecting Suspicious Network Activity

• The Goal: Detect command-and-control (C2) callbacks, lateral movement, or data exfiltration.
• eBPF Hook Point: The connect, accept, and bind system calls for TCP connections, or kernel-level network functions for packet processing.
• Detection Logic:
  1. Attach an eBPF program to the connect syscall.
  2. When any process makes an outbound network connection, your program inspects the destination IP address and port.
  3. Rule: Is the destination IP on a known list of malicious C2 servers? -> Alert!
  4. Rule: Is the destination a Tor exit node or a known cryptomining pool? -> Alert!
  5. Rule: Is a database server, which should never make outbound connections, suddenly trying to connect to a random IP on the internet? -> Alert!

4. Detecting Ransomware Behavior

• The Goal: Detect the rapid, suspicious file access patterns typical of ransomware.
• eBPF Hook Point: The openat, renameat, and unlink system calls.
• Detection Logic:
  1. Your eBPF program monitors file operations for a single process.
  2. It keeps a short-term count of how many files a process is accessing.
  3. Rule: Does a single process open, read, write, and then rename a large number of files with a new extension (e.g., .encrypted, .locked) in a very short amount of time? This is a classic ransomware heuristic. -> Alert and potentially terminate the process!

Real-World eBPF Security Tools

You don't have to build all of this from scratch. Several open-source and commercial security tools are built on top of eBPF to do exactly this:

• Tetragon (from Isovalent/Cilium): An open-source, eBPF-based Security Observability and Runtime Enforcement platform. It's one of the most powerful examples of this technology in action.
• Tracee (from Aqua Security): Another open-source runtime security and forensics tool using eBPF.
• Falco (from the CNCF): While originally built on kernel modules, Falco now has an eBPF driver and is a powerful runtime security engine.
• Commercial EDRs: Leading Endpoint Detection and Response vendors like CrowdStrike and SentinelOne are increasingly incorporating eBPF into their Linux agents because of its superior performance and visibility.

What is APT?

An advanced persistent threat (APT) is a stealthy threat actor, typically a state or state-sponsored group, which gains unauthorized access to a computer network and remains undetected for an extended period. (i.e. "government backed actors")