Detection, Tracking and Recognition
in Video Sequences
Project By:
Sonia Kan
Ora Gendler
Supervised By:
Dr. Ofer Hadar
Mr. Uri Perets
Ben-Gurion University of the Negev
Department of Communication Systems Engineering
Outline
Motivation
Our System
Phase 1: Detection
Phase 2: Tracking
Phase 3: Recognition
Result Video
Motivation
National security is a matter of high priority.
Main method of security cameras.
Disadvantage requirement for man power.
A better method is required.
Outline
Motivation
Our System
Phase 1: Detection
Phase 2: Tracking
Phase 3: Recognition
Result Video
Our System
An automatic security system.
Stationary camera.
Goal detect, track and recognize a moving
human object in a video sequence.
Each phase is implemented as a separate algorithm.
Simulation in Matlab.
Hall Monitor
A General View on the System
State machine :
Detect
Recognize
Track
No
Tracking
successful?
Yes
Outline
Motivation
Our System
Phase 1: Detection
Phase 2: Tracking
Phase 3: Recognition
Result Video
Object Detection
Use Canny edge detector to
find all edge maps.
Use four images:
Background edge map Eb.
Difference edge map DEn.
Current edge map En.
Previous algorithm output MEn-1.
Previous gray
level image
Current gray
level image
I n-1
In
Find edge map of difference
| I n-1 - I n |
image
Eb
DEn
En
Find moving object edge
MEn for frame n
MEn-1
Delay
MEn
Object Detection (Cont.)
Background
frame
Extracted background
edge map
Object Detection (Cont.)
Use four images:
Previous gray
level image
Current gray
level image
I n-1
In
Background edge map Eb.
Difference edge map DEn.
Current edge map En.
Previous algorithm output
Men-1.
Difference image contains
the moving parts of the
frame.
Find edge map of difference
| I n-1 - I n |
image
Eb
DEn
En
Find moving object edge
MEn for frame n
MEn-1
Delay
MEn
Object Detection (Cont.)
Edges
extracted from
the difference
image
Edges
extracted from
the original
frame
Current frame
Object Detection (Cont.)
Use four images:
Background edge map Eb.
Difference edge map DEn.
Current edge map En.
Previous algorithm output MEn-1.
Find pixels that belong to the moving parts of the object
MEnchange .
Find pixels that belong to the still parts of the object MEnstill .
Combine the two components MEn MEnchange MEnstill .
Detection Result
Final edge map
Outline
Motivation
Our System
Phase 1: Detection
Phase 2: Tracking
Phase 3: Recognition
Result Video
Object Tracking
Input:
Current edge map
Previous edge map
Goal - estimate objects location in the next
frame.
The estimation is performed for several frames,
until tracking fails. In that case, the object is redetected.
Object Tracking (Cont.)
Divide the previous edge map into square
blocks.
Use thresholding to determine which blocks
contain parts of objects.
Empty block
Block containing object
Object Tracking (Cont.)
Locate the blocks containing objects from the
previous edge map in the current edge map:
To match the blocks, we use correlation.
previous
Matched
Current
Correlation Statistics
Average correlation per frame
0.8
Correlation %
0.75
0.7
0.65
0.6
0.55
30
40
50
frame#
60
70
80
Matching Results
Previous
edge map
with blocks
Previous edge map
Current edge
map with
matched
blocks
Object Tracking (Cont.)
For each matched block, calculate the amount of pixels it
moved in each axis.
Calculate the average E and standard deviation , and
divide into five ranges.
E
Object Tracking (Cont.)
Estimate the location of each block in the
next edge map according to the
corresponding speed.
Verify true location of each block using
correlation.
Tracking Statistics
percentage of detected blocks per frame
1.1
1
detected blocks %
0.9
0.8
Threshold
0.5
0.7
0.6
0.4
30
40
50
frame#
60
70
80
Tracking Results
Current, next and following frames,
double estimation.
Outline
Motivation
Our System
Phase 1: Detection
Phase 2: Tracking
Phase 3: Recognition
Result Video
Scenario Analysis
Static
Environment
Dynamic
Actors
Size
Motion
Interaction
Equipment
Static
Dynamic
Interaction
Recognition
Monitor amount of objects
Recognize behavior of human object
Size based on detected edges
Size indicates motion inwards or outwards
Motion based on tracked blocks
Recognize sudden changes in speed as suspicious
Interaction with environment based on skeleton
Recognize suspicious postures.
Monitoring Amount of Objects
Skeleton Construction
Locate center of mass
Using:
1 Nb
xc
xi ;
Nb i 1
( xc , yc )
1 Nb
yc
yi
Nb i 1
Find end points of limbs
Mark spine and limbs location
Calculate angles between spine
and limbs to recognize the
posture
Outline
Motivation
Our System
Phase 1: Detection
Phase 2: Tracking
Phase 3: Recognition
Result Video
Separated Objects Demo
You!
ClickThank
to edit Master
title style
Click to edit Master subtitle style
32
f*
Freq. Domain Correlation
If there is a match between an image f and an object
h, the correlation will be maximal at the location of
h in f.
Correlation is given by:
1
f ( x, y ) h ( x , y )
M N
M 1 N 1
f * ( m, n ) h ( x m , y n )
m 0 n 0
When: f ( x, y) h( x, y) F * (u, v) H (u, v)
*
f denotes the complex conjugate of f.
Mathematical Background
Difference edge map:
DEn (| I n 1 I n |)
Pixels that belong to the moving parts of the object :
MEnchange {e En | min || e x || Tchange }
xDEn
Pixels that belong to the still parts of the object :
MEnstill {e En | e Eb , min || e x || Tstill }
Result:
xMEn 1
MEn MEnchange MEnstill
