Zeek

This skill covers Zeek (formerly Bro), the open-source network analysis framework. It has deep knowledge of Zeek's event-driven scripting language, structured log formats, protocol analysis capabilities, cluster deployment, and integration with security analytics platforms.

Important distinction: Zeek is NOT a traditional IDS. It does not use alert rules like Suricata/Snort. It is a passive network analysis framework that generates rich structured metadata about all network activity. Pair Zeek with Suricata for both metadata and signature-based detection.

How to Approach Tasks

When you receive a request:

Classify the request:
- Log analysis -- Apply structured log knowledge; reference log schema
- Script writing -- Apply Zeek scripting language guidance
- Cluster deployment -- Load references/architecture.md
- Threat hunting -- Apply log correlation methodology
- SIEM integration -- Apply integration patterns for Elastic/Splunk/Kafka
- Protocol parsing -- Reference Spicy and built-in protocol support
Clarify use case -- Zeek is used for forensics/hunting, not real-time blocking. If the user needs active prevention, recommend adding Suricata alongside Zeek.
Gather context -- Throughput, deployment mode (standalone vs. cluster), SIEM platform, log retention requirements, programming experience with Zeek scripts.
Recommend -- Provide specific Zeek script code, log analysis queries, or configuration snippets.

Core Expertise

Zeek Log Reference

Zeek generates structured tab-separated logs (or JSON) for each protocol and event type.

Core logs:

conn.log -- The foundation log. Every network connection generates one record at connection close:

ts          uid                id.orig_h    id.orig_p  id.resp_h    id.resp_p  proto  service  duration  orig_bytes  resp_bytes  conn_state  history
1705330425  CXfbCm1ABCDEF123   10.1.2.3     54321      192.168.1.1  80         tcp    http      0.234     1234        5678        SF          ShADadfFRR

Key fields: uid (unique connection ID, links all logs for same session), conn_state (SF=normal close, S0=no response, REJ=rejected, RSTO=RST by originator, RSTR=RST by responder), history (packet-level state machine trace)

dns.log -- All DNS queries and responses:

ts          uid       id.orig_h  id.orig_p  id.resp_h  id.resp_p  proto  query                 qtype  qtype_name  rcode  answers
1705330425  CXFBCM1A  10.1.2.3   54321      8.8.8.8    53         udp    evil.example.com      1      A           0      1.2.3.4

http.log -- All HTTP transactions:

ts          uid       id.orig_h  id.orig_p  id.resp_h    id.resp_p  method  host             uri              referrer  user_agent         status_code  resp_body_len
1705330425  CXFBCM1A  10.1.2.3   54321      192.168.1.1  80         GET     evil.example.com /payload/mal.exe  -         python-requests/2  200          102400

ssl.log -- TLS/SSL handshakes:

ts          uid       id.orig_h  id.orig_p  id.resp_h    id.resp_p  version  cipher               curve    server_name      established  cert_chain_fuids  subject              issuer               validation_status
1705330425  CXFBCM1A  10.1.2.3   54321      1.2.3.4      443        TLSv12   TLS_AES_256_GCM_SHA  x25519   evil.example.com  T            FuniqID1         CN=evil.example.com  CN=Let's Encrypt     ok

files.log -- Files observed traversing the network:

ts          fuid      tx_hosts   rx_hosts  source  depth  analyzers  mime_type                  filename       duration  md5                               sha1
1705330425  FuniqID1  1.2.3.4   10.1.2.3  HTTP    0      MD5,SHA1   application/x-dosexec      malware.exe    0.1       d41d8cd98f00b204e9800998ecf8427e

x509.log -- TLS certificate details:

ts          id        certificate.subject      certificate.issuer  certificate.not_valid_before  certificate.not_valid_after  san.dns
1705330425  FcertID   CN=evil.example.com      CN=Let's Encrypt    2024-01-01                   2024-04-01                   evil.example.com,*.evil.example.com

notice.log -- Zeek notice framework output (from scripts that generate notices):

ts          uid       note              msg                                  src         dst         sub
1705330425  CXFBCM1A  HTTP::SQL_Injection  SQL injection attempt detected   10.1.2.3    1.2.3.4     /search?q=1'%20OR%201=1

Other important logs:

Log	Content
`weird.log`	Protocol anomalies and unexpected behavior
`smb_files.log`	SMB file operations
`smb_mapping.log`	SMB share mapping events
`ntlm.log`	NTLM authentication exchanges
`kerberos.log`	Kerberos authentication events
`ftp.log`	FTP sessions
`ssh.log`	SSH connections (includes hassh fingerprints)
`smtp.log`	Email transactions
`tunnel.log`	Tunneled traffic detection
`dpd.log`	Dynamic Protocol Detection failures
`capture_loss.log`	Packet loss indicators
`reporter.log`	Zeek internal messages
`pe.log`	Portable Executable (Windows binary) metadata

Log Correlation Using uid

The uid field is critical -- it uniquely identifies each connection and links across all log types:

# Find all log entries for a specific connection
UID="CXfbCm1ABCDEF123"

# In all logs simultaneously
grep $UID conn.log http.log ssl.log files.log dns.log

# With zcat for compressed logs
zcat /path/to/logs/2024-01-15/*.log.gz | grep $UID

# Using zeek-cut for structured output
cat conn.log | zeek-cut uid id.orig_h id.resp_h service duration | grep $UID

zeek-cut Utility

zeek-cut extracts specific fields from Zeek tab-separated logs:

# Extract specific fields from conn.log
cat conn.log | zeek-cut ts id.orig_h id.resp_h id.resp_p proto service duration

# Filter with awk
cat conn.log | zeek-cut id.orig_h id.resp_h id.resp_p bytes | awk '$4 > 1000000'

# Count connections by destination
cat conn.log | zeek-cut id.resp_h id.resp_p | sort | uniq -c | sort -rn | head 20

# Convert epoch timestamps to human-readable
cat conn.log | zeek-cut -d ts id.orig_h id.resp_h

Zeek Scripting Language

Zeek's scripting language is event-driven. Scripts subscribe to events generated by the protocol analyzers and network framework.

Script structure:

##! Script description using ##! for doc comments

module MyModule;

# Global variables and state
global suspicious_hosts: set[addr] = {};

# Export definitions (accessible from other scripts)
export {
    ## Redef logging stream to add custom fields
    redef enum Log::ID += { LOG };
    
    type Info: record {
        ts:   time    &log;
        host: addr    &log;
        note: string  &log;
    };
}

# Event handler: called when Zeek sees an HTTP request
event http_request(c: connection, method: string, original_URI: string,
                   unescaped_URI: string, version: string)
{
    if ( /\.exe$/ in unescaped_URI ) {
        # Generate a notice
        NOTICE([$note=Notice::Policy,
                $msg=fmt("EXE download: %s", unescaped_URI),
                $conn=c,
                $identifier=cat(c$id$orig_h, unescaped_URI)]);
    }
}

# Event handler: called at script initialization
event zeek_init()
{
    Log::create_stream(MyModule::LOG, [$columns=Info, $path="my_custom"]);
    print "MyModule initialized";
}

# Event handler: called on connection close
event connection_state_remove(c: connection)
{
    if ( c$id$resp_h in suspicious_hosts ) {
        Log::write(MyModule::LOG, [$ts=network_time(),
                                   $host=c$id$resp_h,
                                   $note="Connection to suspicious host"]);
    }
}

Common events to handle:

Event	When
`zeek_init()`	Script loaded
`zeek_done()`	Zeek shutting down
`new_connection(c)`	New connection started
`connection_state_remove(c)`	Connection closed
`http_request(c, method, uri, ...)`	HTTP request observed
`http_reply(c, version, code, reason)`	HTTP response observed
`dns_request(c, msg, query)`	DNS query
`dns_A_reply(c, msg, ans, TTL)`	DNS A record response
`ssl_client_hello(c, version, ...)`	TLS client hello
`ssl_server_hello(c, version, ...)`	TLS server hello
`file_new(f)`	New file observed
`file_hash(f, kind, hash)`	File hash computed
`smb1_tree_connect_andx_request(c, ...)`	SMB tree connect
`ssh_auth_result(c, result, ...)`	SSH auth result

Data types:

# Basic types
local s: string = "hello";
local i: int = 42;
local c: count = 0;
local r: double = 3.14;
local b: bool = T;
local t: time = current_time();
local a: addr = 10.0.0.1;
local p: port = 80/tcp;
local sub: subnet = 10.0.0.0/8;

# Compound types
local s_set: set[addr] = {};
local s_vec: vector of string = vector();
local s_table: table[addr] of count = {};

# Connection record (the most important type)
local conn: connection;  # Has c$id, c$orig, c$resp, c$service, c$uid, etc.

Pattern matching:

# Check if string matches regex
if ( /malware/ in some_string ) { ... }

# Check subnet membership
if ( some_addr in 10.0.0.0/8 ) { ... }

# Check set membership
if ( some_addr in suspicious_hosts ) { ... }

Intelligence Framework

The Zeek Intelligence Framework allows feeding IOC feeds into Zeek for real-time matching:

Intel feed format (tab-separated):

#fields indicator	indicator_type	meta.source	meta.desc	meta.url
1.2.3.4	Intel::ADDR	my_feed	Known C2 IP	https://source.example.com
evil.example.com	Intel::DOMAIN	my_feed	Malware domain	https://source.example.com
d41d8cd98f00b204e9800998ecf8427e	Intel::FILE_HASH	my_feed	Malware MD5

Loading intel:

# /etc/zeek/site/intel.zeek
@load frameworks/intel/seen
@load frameworks/intel/do_notice

redef Intel::read_files += {
    "/etc/zeek/intel/malicious-ips.intel",
    "/etc/zeek/intel/malicious-domains.intel",
    "/etc/zeek/intel/malware-hashes.intel"
};

Intel types:

Intel::ADDR -- IP address
Intel::SUBNET -- IP subnet
Intel::URL -- URL
Intel::SOFTWARE -- Software version string
Intel::EMAIL -- Email address
Intel::DOMAIN -- Domain name
Intel::USER_NAME -- Username
Intel::FILE_HASH -- File hash (MD5/SHA1/SHA256)
Intel::FILE_NAME -- File name
Intel::CERT_HASH -- Certificate hash

Zeek Packages (ZKG)

The Zeek Package Manager (zkg) installs community packages:

# Install package manager (usually bundled with Zeek)
pip3 install zkg

# Refresh package list
zkg refresh

# Search for packages
zkg search zeek-long-connections
zkg search dns-tunneling

# Install packages
zkg install zeek-long-connections
zkg install corelight/bro-sysmon
zkg install zeek/packages/ja3

# List installed
zkg list installed

# Enable installed packages (add to site/packages.zeek)
echo "@load packages" >> /etc/zeek/site/local.zeek

Essential packages:

zeek/zeek-ja3 -- JA3 TLS fingerprinting (adds ja3 hash to ssl.log)
corelight/zeek-community-id -- Community ID flow hashing (links Zeek, Suricata, Elastic)
zeek/zeek-long-connections -- Detects unusually long connections (beaconing)
corelight/bro-simple-scan -- Port scan detection
zeek/zeek-EternalSafety -- EternalBlue/MS17-010 detection
mitre-attack/bzar -- MITRE ATT&CK-based detection scripts

Threat Hunting with Zeek Logs

Common Hunt Queries

Beaconing detection (regular interval connections):

# Find hosts making many small connections to the same destination
cat conn.log | zeek-cut id.orig_h id.resp_h id.resp_p duration orig_bytes \
  | awk '$4 < 5 && $5 < 1000' \
  | awk '{print $1, $2, $3}' \
  | sort | uniq -c | sort -rn \
  | awk '$1 > 100'  # More than 100 short connections = potential beacon

DNS with high NX domain rate (potential DGA):

cat dns.log | zeek-cut id.orig_h query rcode \
  | awk '$3 == 3' \  # NXDOMAIN = rcode 3
  | awk '{print $1}' \
  | sort | uniq -c | sort -rn \
  | awk '$1 > 50'    # More than 50 NX domains from single host

Large outbound file transfers:

cat conn.log | zeek-cut ts id.orig_h id.resp_h service orig_bytes resp_bytes \
  | awk '$5 > 10000000' \  # Originator sent > 10MB (potential exfil)
  | awk '!($4 == "http" || $4 == "ssl")' \ # Exclude normal web traffic
  | sort -k5 -rn

New external connections from internal hosts:

# Connections to IPs not seen in previous 7 days (requires log comparison)
# Practical approach: look for connections to IPs with low history
cat conn.log | zeek-cut ts id.orig_h id.resp_h id.resp_p service \
  | awk '!/^(10\.|172\.(1[6-9]|2[0-9]|3[01])\.|192\.168\.)/' \
  | awk '{print $3}' | sort -u  # Unique external IPs contacted

SMB lateral movement:

cat smb_mapping.log | zeek-cut ts id.orig_h id.resp_h share path \
  | grep -E "\\\\(C|ADMIN|IPC)\$" \  # Admin share access
  | sort

Kerberoasting detection:

cat kerberos.log | zeek-cut ts id.orig_h request_type service cipher error_msg \
  | awk '$2 == "TGS" && $4 == "rc4-hmac"' \  # RC4 encryption for service tickets = Kerberoasting
  | sort | uniq -c | sort -rn

Common Pitfalls

Expecting Zeek to block traffic -- Zeek is passive/read-only. To block, you need Suricata IPS, firewall, or NAC integration.
Not understanding uid correlation -- Each connection has a unique uid. All logs for the same connection share this uid. Always use uid to pivot between log types during investigation.
Ignoring conn_state -- S0 (connection attempt, no response) and REJ (rejected) are as forensically important as SF (successful). A mass of S0 events = port scan.
Missing the weird.log -- weird.log contains protocol anomalies that often indicate exploitation or evasion. It is underused but high signal.
Writing Zeek scripts that aren't event-driven -- Don't try to write procedural Zeek scripts. Structure everything as event handlers. State persistence happens through global tables and sets.
Not using zeek-cut for field extraction -- Parsing Zeek TSV logs with raw awk/cut is error-prone because field positions change when fields are added. Use zeek-cut which handles headers.

Reference Files

Load these when you need deep knowledge for a specific area:

references/architecture.md -- Zeek internals (event engine, script execution, cluster deployment with manager/logger/proxy/workers, log rotation, Spicy custom parsers, performance tuning). Read for architecture, cluster design, and advanced scripting questions.

zeek

SKILL.mdplugins/security/skills/zeek/SKILL.md