Two Levels of Correlation

▶ Watch (1:41)

Correlation means two things. First, group similar events from the same session—for example, 50 net flows, 400 sysmons, and 1,000 Windows event logs all describing a 2-minute website visit. Second, assemble those groups into an attack story that shows the progression from initial access to exfiltration. Audience members Manolo and Bill defined both objectives. The problem is that correlation rules handle neither level well.

Why Correlation Rules Fail with Context

▶ Watch (13:09)

Rules look at only a few fields—same IP, same port, within 5 minutes. Tahoun showed logs where the same IP repeats 500 times, yet those duplicates describe valid sessions. Manolo pointed out that grouping requires considering every field: different ports, different times, different users. A rule cannot capture all combinations. The solution is to treat grouping as a recommendation problem, like YouTube suggesting related videos by analyzing all features, not just a few tags.

Group Everything with Clustering

▶ Watch (15:25)

Tahoun explained that every human activity generates multiple logs across network, endpoint, and proxy sources. Clustering models like K-means or DBSCAN place each event in a multidimensional space based on every field. Events close together form a group. This approach captures context that no rule set can—events five hours apart, different IPs, but same user behavior. After grouping, duplicates vanish because each cluster contains one representative entry.

Chain Events into Attack Stories

▶ Watch (22:06)

Once groups exist, the second level of correlation uses sequencing models (Markovian) to link groups whose tactics appear sequentially—for example, initial access followed by execution, then exfiltration. Tahoun showed the Singapore Health attack story as an example. Groups that carry tactics in kill-chain order become an attack story. False positives rarely align into a sequential chain, so the number of real alerts drops to a handful per day.

Real-World Savings: 93% Volume Reduction

▶ Watch (40:27)

Tahoun demonstrated a 4-terabyte-per-day Splunk environment. After grouping and deduplication, the volume fell to 0.25 terabytes. At $4 million license cost, the reduction saved $2.7 million. Even a poor configuration saved 40%. The same method turns millions of false positive alerts into one group. Tahoun released the code on GitHub; it runs on-premises inside Logstash or any pipeline. No data leaves the organization.

Q&A

What algorithms does the tool use? Clustering (K-means, DBSCAN) for grouping, small language models for enrichment, Markovian models for sequencing. ▶ 46:21

How does it incorporate business context? Lookup tables from Active Directory feed entity enrichment—department, manager, location. These features influence clustering distances. ▶ 50:48

Is the model pre-trained and supervised? No. The approach uses semi-supervised expert ML. The only trained component is a small language model for mapping log text to MITRE techniques, built from 3 years of hand-labeled data. ▶ 58:51

Notable Quotes

all of the data is false positive. All of our data is noisy. All of our data is garbage. Ezz Tahoun · ▶ 44:30

I think AI is the biggest data transfer scam in the world. I think we’re all sending our data so that the telco companies make a lot more money Ezz Tahoun · ▶ 58:17

false positives don’t stack together into a story. And if they do, it’s like a negligible amount that people would not really complain about. Ezz Tahoun · ▶ 43:37

The smaller the model you see it’s missing an L. It’s just LM. … big models … they call them LLM large language models we use small language models they don’t speak English they speak cyber security Ezz Tahoun · ▶ 46:52

Key Takeaways

  • Replace correlation rules with clustering to group all events by full context.
  • Sequence models turn grouped events into attack stories using kill-chain tactics.
  • On-prem open-source code can reduce SIEM log volume by 93% and cut false positives.