BsidesLVPresso2016_JZeditsv6

No Silver Bullet
Multi contextual threat detection via
Machine Learning.
By @rodsoto @jozephzadeh

$:Whoami..
• Rod Soto
– Researcher at Splunk UBA, former AKAMAI,
Prolexic PLXSert. Like to break things, p0wn
botnets and play CTFs.
• Joseph Zadeh
– Data Scientist at Splunk UBA, building behavioral
intrusion detection technologies at scale. Enjoy
working on defense projects that combine
security, artificial intelligence and distributed
systems.

Agenda
• Introduction: Big Data and Machine Learning
• Machine Learning in security workflows and
how it can help and limitations
• Describe central nervous system approach to
behavioral security: Lambda Defense

Challenges in Current Threat Indicator
Technologies
• Many devices generating logs and alerts
• Data distributed in too many places slows analysis, preventing
analysts from effectively analyzing all alerts
• SIEM makes life somewhat easier, giving analysts one place to
collect data but still deal with needle in haystack issues

The Big Data Challenge
• " It costs organizations an average of $1.27
million annually in time wasted responding to
erroneous or inaccurate malware alerts.
According to respondents, an average of 395
hours is wasted each week detecting and
containing malware because of false positives
and/or false negatives. The extrapolated average
value of lost time is estimated at approximately
$25,000 per week or $1.27 million each year for
participating organizations.” Ponemon Institute

The Big Data Challenge
• SOCs are challenged and limited in the scope of
detection, analysis and action.
• Constant required training, updates and turnover
of SOCs present a challenge for organizations.
• As of now People vs People model has proven to
be more effective as current threat
detecting/prevention technologies do not seem
sufficient nor effective against malicious actors.
The numbers speak for themselves.

Big Data challenge, presents a new
opportunity as well. Enter Machine
Learning
• Machine learning is a subfield of computer science[1] that
evolved from the study of pattern recognition and
computational learning theory in artificial intelligence.[1]
Machine learning explores the study and construction of
algorithms that can learn from and make predictions on
data.[2] Such algorithms operate by building a model from
example inputs in order to make data-driven predictions or
decisions,[3]:2 rather than following strictly static program
instructions.
*Wikipedia

Machine Learning & Big Data
Technologies
• The ability to process very large sets of data
through distributed computing plus the ability
to apply algorithms that can learn based on
these large datasets, will provide analysts with
more meaningful detection and actionable
items.

Learning Algorithms
• “a process or set of rules to be followed in
calculations or other problem-solving operations,
especially by a computer.” *Wikipedia
• These learners can be designed and develop to
scale against all these sources of data and
produce meaningful detection of anomalies.
• By applying these learners we can build models
that can approach threats from a multi
contextual, dynamic perspective, thus going
beyond the concept of static signature based
security technologies.

Sequencing the Security DNA
• The next gen paradigm:
– 1:1 Correspondence between users data footprint
and Compute Resources
• Commoditization of compute means for
300,000 User Accounts means assign 300,000
individual threads + memory + disk to run
learning algorithms per individual log
footprint simultaneously

Adversarial Drift
• Current status quo, is driven by adversaries
developing and introducing changes in their
TTPs, bypassing all current detection
technologies.

Advesarial Models
• Machine Learning
Looses
Effectiveness the
more complex the
adversary

Advesarial Models
Automatable
Actions: Good for
ML
Non-Automatable
Actions: Hybrid
Human/Computer
Analysis

Learning Algorithms in Security

Advantages of using ML
• Using ML allows us to put together very large
and distinct sources of data into a platform for
analysis, interpretation and prediction.
• ML allows us to go beyond of static signature
based technologies.
• ML creates an scenario where detection of
threats based on dynamic and multi
contextual indicators is possible.

18
Automating the Forensic Workflow
• Incident Response Is Hard Work! What
can we automate?
A security analyst is an oracle whose
input is evidence and whose output is
True Positive, False Positive, True
Negative or False Negative
– The list of possible questions is large but
typically the flow is a type of decision tree
for example

19
ML as a tool to make your job easier
Security Oracle Workflow
Example 1:
Evidence => Periodic Communication
=> LAN to WAN Data =>WAN URL has
Bad Reputation => Correlate with VT
=> True Positive
Example 2:
Evidence => Potential C2 Domain =>
LAN to WAN Data => WAN URL is new
Google IP => False Positive

Learning = Compression?
• There is a duality between learning and compression
Input Data Total
Size = 1 GB
Learned output is a
set of “coefficients”
Total Output Size =
1K
Primary Key
Tim
e
UserI
D
Count
Row 1 … … …
Row 2 … … …
Row 3 … … …
… … … …
Row N … … …
C
1
C
2
C
3
C4 C
5

• Example of Linear Regression in R

• Train a model to predict mpg as a function of car
weight, number of cylinders and displacement

• The overall input data is reduced in a “compressed
form” to use in future predictions

• This process is extremely brittle in terms of modeling a changing
signal or an adversary that changes patterns over time

• The simple linear model gives us output that separates the Signal
from the Noise (this is not always possible with a model)

• Real example of random forest trained on C2 traffic

• We really “learn” a function we can call in batch or real
time

ML Challenges
• Over fitting/Under fitting
• Technology still in early stages
• “Operationalization”
• Advesarial drift and changing TTP’s means
models have to change over time (retraining)

Decomposing Behaviors for Intrusion
Detection

Behaviors: Sequential + “Unordered”
• Sequential Behaviors
– Exploit Chains
– Timing Analysis
(Periodicity)
– Active Directory
Sequence
– Authentication Graph
• Non Sequential
Behaviors
– Fingerprinting
– Grouping Behaviors
– Application Counts
– Rare file extension
counts for Webshell
detection

Mapping Behaviors to Code
• Easy to Parallelize
– Count()
– Average()
– Time series()
– Local state
computations
• Per user/IP/account/…
• Hard to Parallelize (NC
Complete Complexity)
– Rank()
– Median
– …
– Anything that keeps
track of global state

Lambda Security
• Lambda architecture provides a design paradigm
for a “Scalable Central Nervous System” for the
SOC whose components include
– Machine learning based ETL(Extract/Transform/Load)
– Distributed crawlers
– Automated identity/session resolution and fingerprinting
– Formal evidence collection protocol for automated
labeling of incident response data
– Analytics Metrics and establishing benchmarks for
heterogeneous data

Batch Features + Real Time Features
• Keep in mind all work is done on a cluster
(distributed system)
– Concepts: groupBy (User,Domain, “arbitrary field”)
• Batch Example
– Data driven domain popularity
• Real time example
– Exploit chain content types
• Lambda => Immutable/Functional data structures
– Spark RDD’s (abstraction for a distributed
computation as opposed to result of a distrubted
computation)

Lambda + Central Nervous System
• Augment “in memory” lightweight signal from
the point with large scale processing platforms
that can “sequence the security DNA”
– Classical IDS/FW/Point solutions have significant
limitations in terms of sharing state and being able
to correlate across nodes

37
Lambda Architecture
• Architecture is described by three simple equations:
batch view = function(all data)
realtime view = function(realtime view, new data)
query = function(batch view, realtime view)

Model Life Cycle Implementation

Lambda Security
DHCP
IMS/IPAM
FW
Proxy
VPN
AD
Data
Ingest

Lambda Security
DHCP
IMS/IPAM
FW
Proxy
VPN
AD
Real Time Identity Resolution
Distributed
ETL
Username = select
coallesce(user_name,
hostname, IP) from
Active_ID_Table
where IP =
‘10.10.100.23)
IP DHCP.MAC DHCP_Lasteventtime AD_FQDN
10.100.1.23 58:5c:35:c3:6e:a4 2014-03-11T14:00:00 joe.eng.acme.com
10.13.11.221 12:3a:74:b2:6a:22 2014-03-12T14:30:00 ad.hr.acme.com
Sequential
Models and
IOC’s
Data
Ingest
Real Time Layer

Lambda Security
44
DHCP
IMS/IPAM
FW
Proxy
VPN
AD
Distributed
ETL
Username = select
hostname, IP) from
Active_ID_Table
where IP =
‘10.10.100.23)
Sequential
Models and
IOC’s
Data
Ingest
Large Scale Models and
Non-Sequential IOC’s
Real Time Layer
Batch
Layer

Lambda Security
45
DHCP
IMS/IPAM
FW
Proxy
VPN
AD
Distributed
ETL
Username = select
hostname, IP) from
Active_ID_Table
where IP =
‘10.10.100.23)
Sequential
Models and
IOC’s
Data
Ingest
Real Time Layer
Batch
Layer
Hybrid View
(Batch + Real
Time)

46
DHCP
IMS/IPAM
FW
Proxy
VPN
AD
Distributed
ETL
Username = select
hostname, IP) from
Active_ID_Table
where IP =
‘10.10.100.23)
Sequential
Models and
IOC’s
Data
Ingest
Hybrid View
(Batch + Real
Time)

47
DHCP
IMS/IPAM
FW
Proxy
VPN
AD
Distributed
ETL
Username = select
hostname, IP) from
Active_ID_Table
where IP =
‘10.10.100.23)
Sequential
Models and
IOC’s
Data
Ingest
Automated process to
accelerate workflows like
Splunk Query to retrieve PCAP
for further analysis combined
with automatic VT/heuristic
correlations
Hybrid View
(Batch + Real
Time)

ML + Sequencing the Security DNA
• We parallelize across many nodes (JVMs) and use
both real time and batch computations
JVM 1
JVM 2
JVM 3
1. GET http://forbes.com/gels-contrariness-domain-
punchable/"
2. GET http://portcullisesposturen.europartsplus.org/
3. POST http://dpckd2ftmf7lelsa.jjeyd2u37an30.com/
1. GET http://youtube.com/
2. GET http://avazudsp.net/
3. GET http://betradar.com/
4. GET http://displaymarketplace.com/
1. GET http:/clickable.net/
2. GET http://vuiviet.vn/
3. GET http://homedepotemail.com/
4. GET http://css-tricks.com/

ML applied to Malware Research
Dridex, Zeus
• Malware uses covert command and control
techniques to evade detection
• Malware communication leaves footprints of
anomalous behaviors
– Domain Generation Algorithms
– SSL command and control
– Twitter/Facebook/Gmail based steganography
– RFC Compliant DNS backdoor

Adaptive Filter
(Crowd sourced
Popularity
Metrics)
External Domain/IP Profile
Data In
Global
Evidence
Collection
C2 Model
Timing
Features
Lexical
Analysis
Communic
ation Stats
Example:
Variance of Inter-
arrival Times
Example:
N-Gram
Score
Ratio of Bytes
In/Bytes Out
Domain
Communication
Score
Timing Score Layer 7 Score NLP Score
Analyst
Recommendation
www.evil.com High Risk Moderate Risk Moderate Risk No Risk
Critical Prioirty:
Communication is
active and going
unlbocked
www.khhjdkshj33ejj.com 0 Moderate Risk 0 High Risk
Low Priority: Traffic
is blocked by
firewall
www.google.com No Risk No Risk No Risk No Risk No Action Needed
Classification Algorithm
Human Feedback Loop

Key to ML: Label Your Analysis
Domain Name TotalCnt RiskFactor
AGD
SessionTime RefEntropy NullUa
europartsplus.org 144 6.05 1 1 0 0
jjeyd2u37an30.com 6192 5.05 0 1 0 0
cdn4s.steelhousemedia.com 107 3 0 0 0 0
log.tagcade.com 111 2 0 1 0 0
go.vidprocess.com 170 2 0 0 0 0
statse.webtrendslive.com 310 2 0 1 0 0
cdn4s.steelhousemedia.com 107 1 0 0 0 0
log.tagcade.com 111 1 0 1 0 0
• Label output of every investigation in a
consistent manner!!!

Key to ML: Label Your Analysis
Domain Name TotalCnt RiskFactor
AGD
SessionTime RefEntropy NullUa Outcome
yyfaimjmocdu.com 144 6.05 1 1 0 0 Malicious
jjeyd2u37an30.com 6192 5.05 0 1 0 0 Malicious
cdn4s.steelhousemedia.com 107 3 0 0 0 0 Benign
log.tagcade.com 111 2 0 1 0 0 Benign
go.vidprocess.com 170 2 0 0 0 0 Benign
statse.webtrendslive.com 310 2 0 1 0 0 Benign
cdn4s.steelhousemedia.com 107 1 0 0 0 0 Benign
log.tagcade.com 111 1 0 1 0 0 Benign
• This is how the algorithms will “learn” from
human expertise and help support a common
security workflow
Human Expertise is manually encoded into a format
computers understand: Sometimes this process is
called Labeling or “Truth-ing” the data

Sequential Behaviors: Exploit Chain
1. Initial Redirect From Poisoned Domain: [29/Apr/2015:16:52:23 -0700] "Nico Rosberg" 192.168.122.177 69.162.78.253
1500 200 TCP_HIT "GET http://forbes.com/gels-contrariness-domain-punchable/1.html/548828415920276748 HTTP/1.1"
"Internet Services" "low risk" "text/html" 604 142 "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; WOW64;
Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)"
"http://forbes.com/gels-contrariness-domain-punchable/1.html" "-" "0" "" "-”
Sequencing data by account name is a great way to
catch certain attacks over http data that are
otherwise very expensive to compute downstream

1500 200 TCP_HIT ”GET http://forbes.com/gels-contrariness-domain-punchable/1.html/548828415920276748 HTTP/1.1"
2. Flash Exploit: [29/Apr/2015:16:52:26 -0700] "Nico Rosberg" 192.168.122.177 69.162.78.253 1500 200 TCP_HIT "GET
http://portcullisesposturen.europartsplus.org/IMvOBBZKDLqAJYIDe02t5hMMNyzBLN_q4kafJkVNqJVTnTmd HTTP/1.1"
"Internet Services" "low risk" "application/x-shockwave-flash" 518 821 "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1;
WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)"
"http:///forbes.com/gels-contrariness-domain-punchable/1.html/548828415920276748" "-" "0" "" "-”

3. Payload: [29/Apr/2015:16:52:27 -0700] "Nico Rosberg" 192.168.122.177 69.162.78.253 1500 200 TCP_HIT "GET
http://portcullisesposturen.europartsplus.org/UX7n1YkbNn8FUV6QVtEZLj-p-gLvRKlWEWmz3r7Ug8suRiY_ HTTP/1.1"
"Internet Services" "low risk" "application/octet-stream" 136 915 "" "" "-" "0" "" "-”

4. Command and Control: [29/Apr/2015:16:52:33 -0700] "Nico Rosberg" 192.168.122.177 104.28.28.165 1500 200 TCP_HIT
"GET
http://dpckd2ftmf7lelsa.jjeyd2u37an30.com/tsdfewr2.php?U3ViamVj49MCZpc182ND0xJmlwPTIxMy4yMjkuODcuMjgmZXhl
X3R5cGU9MQ== HTTP/1.1" "Internet Services" "low risk" "text/html; charset=UTF-8" 566 5 "Mozilla/4.0 (compatible; MSIE
8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" "" "-" "0" "" "-”

"Internet Services" "low risk" "application/x-shockwave-flash" 518 821 "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT
6.1; WOW64; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0)"
4. Command and Control: [29/Apr/2015:16:52:33 -0700] "Nico Rosberg" 192.168.122.177 104.28.28.165 1500 200 TCP_HIT
"GET
http://dpckd2ftmf7lelsa.jjeyd2u37an30.com/tsdfewr2.php?U3ViamVj49MCZpc182ND0xJmlwPTIxMy4yMjkuODcuMjgmZ
XhlX3R5cGU9MQ== HTTP/1.1" "Internet Services" "low risk" "text/html; charset=UTF-8" 566 5 "Mozilla/4.0 (compatible;
MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)" "" "-" "0" "" "-”

Conclusion
- ML can potentially become a milestone
technology in Cybersecurity
- Upcoming advances in hardware and
distributed computing will accelerate
development in ML: Lambda Security
- Need to industry standard to share behavioral
indicators and labels
- NO SKYNET in the foreseeable future 

Thank you
- Rod Soto
rsoto@splunk.com @rodsoto
- Joseph Zadeh
jzadeh@splunk.com @josephzadeh

Cybersecurity Analytics: ROIv1

Lambda Firewalls?!
Manage the paths accordingly start building lambda
workflows into Everything!!!
• Lambda firewall
– Statistical whitelist computation aspect (fuzzy ACL’s)
– Path for signatures and sequential behaviors that is more expressive
than PCRE
• Central nervous system approach to blending signals
– Defense should scale up and down the size of organization: a properly
engineered central nervous system should be able to protect SMB
market as well as large scale deployments
• Difference between a classical firewall and a lambda firewall

Parallel Sequencing of Behaviors
Number
1. http://forbes.com/gels-contrariness-domain-
punchable/1.html"
2. http://portcullisesposturen.europartsplus.org/I
MvOBBZKDLqAJYIDe02t5hMMNyzBLN_q4kafJk
VNqJVTnTmd
3. http://portcullisesposturen.europartsplus.org/
UX7n1YkbNn8FUV6QVtEZLj-p-
gLvRKlWEWmz3r7Ug8suRiY_
4. http://dpckd2ftmf7lelsa.jjeyd2u37an30.com/ts
dfewr2.php?U3ViamVj49MCZpc182ND0xJmlwP
TIxMy4yMjkuODcuMjgmZXhlX3R5cGU9MQ==
GroupBy(“User”)

BsidesLVPresso2016_JZeditsv6

More Related Content

What's hot

Similar to BsidesLVPresso2016_JZeditsv6

More from Rod Soto

BsidesLVPresso2016_JZeditsv6

Editor's Notes