False Positives and Machine Learning, Part 1

29 May 2019

A false positive (FP) is when a security system treats legitimate activity as an attack. It sounds like a minor annoyance, but the costs add up:

• E-commerce sites lose real customers.
• Legitimate web crawlers get blocked, which hurts SEO.
• Training data gets contaminated, so the errors compound.
• Resources get pulled away from real threats.

A common suggestion is to find an "optimal ratio" between false positives and false negatives (FNs), based on what each one costs in money. I don't think this approach works. In Part 2 I'll describe what I measure instead.

Machine learning for security

ML algorithms used in security fall roughly into two groups.

Shallow learning (SL):

• Supervised: shallow neural networks, HMM, KNN, random forest
• Unsupervised: clustering, association
• Requires a lot of feature engineering

Deep learning (DL):

• Supervised: FNN, RNN, CNN
• Unsupervised: SAE, DBN
• Less need for manual feature selection

To compare approaches, we use the standard metrics:

False Positive Rate = FP / (FP + TN)
Accuracy = (TP + TN) / (FP + TP + FN + TN)

Existing methods

The research literature has several approaches to reducing FPs:

• parameter tuning in intrusion detection systems (IDS),
• rule-based classification for specific attack types,
• neural network models such as GHSOM,
• two-stage correlation systems.

From research to production

Many of these methods look good in papers and then run into problems in production:

• They work offline, but production needs real-time analysis.
• Accuracy doesn't carry over from one organization's traffic to another's.
• Threats change, so models need to adapt continuously.
• Someone has to decide what gets automated and what stays with a human.

In Part 2 I'll go into how we handle these in practice.