COMP/ECE/ENGI: COMPUTER

VISION

Lecture 2: Overview of Computer Vision

COMPUTER VISION www.mun.ca

LABS – DATES FIXED

Labs 10% 5% -
Lab 1 2.5% 1.25% February 6 (due approximately one
week later)
Lab 2 2.5% 1.25% February 27 (due approximately one
week later)
Lab 3 2.5% 1.25% March 13 ((due approximately one
week later)
Lab 4 2.5% 1.25% March 27 (due approximately one
week later)

CS3300 INTERACTIVE TECHNOLOGIES www.mun.ca

QUESTIONS?
Today
• Intros
• Computer vision overview
• Course overview

Readings for today

• Book: Richard Szeliski, Computer Vision: Algorithms and Applications
● Chapter 1
What is computer vision?
automatically identifying objects in images or video
Extracting latent information from visual data
technology that interprets light stimuli
computers seeing/learning things that the programmers who made them
didn't tell them
Mimicking the human perception of sight with computational algorithms
train computers to understand the visual world
The study of understanding the world through visual perception

a simulation of eyes
Converting images to more understandable things like distance, edges,
directions etc.
Computer getting information out of images/video
Giving the computer "eyes" to see and identify as humans would.
Teach computer to interpret and understand our world through images.
What is computer vision?

Terminator 2
What is computer vision?

Terminator 2
Every picture tells a story

Goal of computer vision is to write computer programs

that can interpret images
What do computers see?
243 239 240 225 206 185 188 218 211 206 216 225

242 239 218 110 67 31 34 152 213 206 208 221

243 242 123 58 94 82 132 77 108 208 208 215

235 217 115 212 243 236 247 139 91 209 208 211

233 208 131 222 219 226 196 114 74 208 213 214

232 217 131 116 77 150 69 56 52 201 228 223

232 232 182 186 184 179 159 123 93 232 235 235

232 236 201 154 216 133 129 81 175 252 241 240

235 238 230 128 172 138 65 63 234 249 241 245

237 236 247 143 59 78 10 94 255 248 247 251

234 237 245 193 55 33 115 144 213 255 253 251

248 245 161 128 149 109 138 65 47 156 239 255

190 107 39 102 94 73 114 58 17 7 51 137

23 32 33 148 168 203 179 43 27 17 12 8

17 26 12 160 255 255 109 22 26 19 35 24

slide by Larry Zitnick

Can computers match (or beat) human vision?

Yes and no (but mostly no!)

• humans are much better at “hard” things
• computers can be better at “easy” things
Human perception has its shortcomings…

Sinha and Poggio, Nature, 1996

Copyright A.Kitaoka
2003
Current state of the art
The next slides show some examples of what
current vision systems can do
3D Maps

Apple Maps
2D Maps

Google Maps
Computational photography

Portrait mode
simulating wider aperture
Even wider aperture...

How scientists captured the first image of a black hole, 2019

3D photos

3D Photos on Facebook
Estimate depth from photo to create animation
https://ai.facebook.com/blog/-powered-by-ai-turning-any-2d
-photo-into-3d-using-convolutional-neural-nets/
Face recognition

Who is she?
Vision-based biometrics

“How the Afghan Girl was Identified by Her Iris Patterns” Read the story
Object recognition
 Special effects: shape capture

The Matrix movies, ESC Entertainment, XYZRGB, NRC

Sports

Sportvision first down line

Nice explanation on www.howstuffworks.com
Games

Microsoft’s XBox Kinect

2
5
Virtual Reality

Oculus Quest, Beat Saber

Augmented Reality

Microsoft Hololens 2
HoloLens2 Sensors
4 head-tracking
8Mpix RGB cameras
camera (stereo + periphery)
+
IMU

1Mpix depth camera

(near and far range)

IR eye cameras + IR
LEDs

5 microphone
array
Augmented Reality
Phone-based AR

https://www.youtube.com/watch?v=0Pj-jzy6ESE
Body Tracking
Robotics

NASA’s Mars Curiosity Rover (self portrait)

https://en.wikipedia.org/wiki/Curiosity_(rover)
Smart cars Slide content courtesy of Amnon Shashua

Mobileye
• Vision systems currently in high-end BMW, GM, Volvo models
Self-driving cars

https://waymo.com/tech/
Drones

https://www.skydio.com/
Research: Timelapse
Research: Neural Rendering
Research: Yolo
Research: StyleGan
Current state of the art
You just saw examples of current systems.
• Many of these are less than 5 years old

This is a very active research area, and rapidly changing

• Many new apps in the next 5 years
Perception
Images
Filtering
 Machine Learning for Computer Vision

Predictor
Low-level features
Mid-level High-level Process features
Lines, oriented
features features and predict output
edges
Combine edges: Combine shapes:
curves, shapes objects, scenes
Object Detection

https://heartbeat.fritz.ai/introduction-to-basic-object-detection-algorithms-b77295a95a63
Segmentation

https://gts.ai/how-do-we-solve-the-challenges-faced-due-to-semantic-segmentation/
Features
Optical Flow

https://www.commonlounge.com/discussion/1c2eaa85265f47a3a0a8ff1ac5fbce51
Stereo and Depth
3D Mapping (SLAM and SfM)
3D Shape and Appearance
Computational Photography

https://ai.googleblog.com/2017/10/portrait-mode-on-pixel-2-and-pixel-2-xl.html