Hai Tao at AI Frontiers: Deep Learning For Embedded Vision System
6 likes2,460 views
Vion Technologies, founded in 2005, specializes in computer vision solutions for intelligent transportation, smart surveillance, and business intelligence, employing over 200 staff. The company's innovations include high-resolution cameras and advanced algorithms, targeting markets such as public security and smart traffic management, with a focus on low-power, cost-effective processing technology. Their research and development team is led by experts with strong academic backgrounds in electrical engineering and computer science, emphasizing deep learning and embedded vision hardware.
Hai Tao at AI Frontiers: Deep Learning For Embedded Vision System
- 1. Deep Learning for Embedded Vision System Vion Technologies Co., Ltd. Jan. 11th, 2017 Hai Tao, Dr. Credits to all my colleagures who make this presntation possible
- 2. Essentials Founded in 2005 2014.12 Series A Funding 2016 NEEQ/ Series B Funding Vion Technologies: A Leader in the Field of Computer Vision • Vion Technologies Co. Ltd., founded in 2005, currently employs 200+ talented staffs. The company is developing CV HW/SW total solutions for intelligent transportation systems (ITS), smart video surveillance systems and business intelligence systems. • Huge potential for CV products in ToB markets • Every year more than 40 million surveillance cameras are sold globally (IDC data analysis) • High resolution (720p, 1080p, even 4K resolution) IP cameras are replacing the D1 resolution analog cameras • Better algorithms enable more applications in ToB applications • High performance, low power consumption, low cost processors are available
- 3. • IOT+Computer Vision, Where Are the Applications ? • Embedded CV Hardware • GPU, VPU, and FPGA
- 4. Intersection violation capture & smart plate number recognition & light control Transit & emergency vehicle lane use capture Parking Violation Capture Smart Traffic Smart parking management Industry Applications
- 5. Malls Retail Stores Theatre People CountingSubway People Counting Cultural Attraction Guest Traffic Transit People Counting Smart People Counting Industry Applications
- 6. Security & Counterterrorism: Fighting Security & Counterterrorism: Chasing Rail: Driver Fatigue Intruder Alerts Banking: ATM Protection Mining: Production Safety Public Security, City Management, Banking, Rail, Border Control and Many More ... Industry Applications
- 7. • IOT+Computer Vision, Where Are the Applications ? • Embedded CV Hardware • GPU, VPU, and FPGA
- 8. • Integrated image sensing and analysis • Wifi probe & iBeacon • POE powered • Patented exterior design, screw free installation • H.264 real-time video output • 2-year data storage • Integrated image sensing and analysis • RS485, GPIO • Patented exterior design specially for transit • H.264real-time video output • 2-year data storage • IP65, sealed against dust & water • Sensor rich (multi-axis/temp) • 3/6/8MP 25fps • High performance platform • 3G/4G/WIFI • Smart traffic industry Sensor Rich ITS Camera Smart Traffic Camera Bus People Counting Smart Cameras
- 9. Spec: 4K resolution 4/3' CCD, Ambarella processor, Xilinx FPGA module Applications: ePolice at road intersections, covering 4 lanes. The first 4K@25fps ePolice in the world Release data: 2016 Q3 Smart Cameras - 4K ePolice Camera
- 10. Spec: ARM processor, compact form format Applications: People counting for Shopping malls and retail stores. Release Date: 2016 Q3 Smart Cameras - People Counting
- 11. Tarsier I Module - A Step to Smart Edge Device Low Power <1.5W Video & Audio Interface camera, other processors Low Cost <$15 Multi-Modal IC Technology 28nm low power High Performance Deep Learning（CNN， Recurrent DNN） > 40GFLOPS Front-End AI Module - Tarsier I Quick Time-to-Market 16'Q3
- 12. Smart Cameras Design- Bus Traffic Counting
- 13. • GPU platform, 300 GFLOPS • Analog/IP Video Input • 2.5 inch hardrive & EMMC • USB3.0, dual gigabyte LAN • High performance， 300 GFLOPS • 4 3.5” hard drives • USB 3.0 • Dual gigabyte network ports Front End Control Terminal • Dual GPU, 600 GFLOPS • 8 analog video & audio input • Hard drive & EMMC storage • 4 alarm in, 2 out Smart Video & Audio Analysis Terminal • 40 nVidia GPUs • 80ch 1080P H.264 decoding • Processing up to 160ch@D1 or 80ch@1080p High Density GPU Cluster Server Back-End GPU Processing Units CBox - Single GPU Unit
- 14. Spec: 40 nVidia GPUs, <600W, analyze up to 160ch@D1 or 80ch@1080p in real time Applications: ITS, crowd management, IVS in various industries Release date: Q3,2016 Back-End GPU Processing Units - StarNet I
- 15. • IOT+Computer Vision, Where Are the Applications ? • Embedded CV Hardware • GPU, VPU, and FPGA
- 16. DNN Speed on TK1, MA2450 • Nvidia TK1: 120ms/frame <12W • Movidius MA2450: 140ms/frame <1.5W
- 17. Nvidia Tegra K1: CNN Implementation • GPU for detection (relatively low frequency) and CPU for tracking • Memory footprint is optimized via buffer sharing and TK1's unified mem mechanism • Maximize CPU & GPU utilization via nvidia asynchronous ops and streams. • cuDnn library for general layers • Non-standard layers are implemented based on fine-tuned kernels • 1x1 convolution, Balance between MACs & accuracy • Balance between depth & width, depth for more representative power
- 18. Movidius Implementation • fp16 is used with no accuracy loss • Net architecture is tuned based on depth, width, kernel size • Convolution/bias/relu/pooling -> combined layer • All combined layer operations run in the on-chip CMX memory • DDR and CMX exchange data when a combined layer is completed • Implement 2D convolution in assembly kernel • Bias, relu and pooling are done via processor intrinsics • Make full use of the underlying “SIMD” shave architecture
- 19. Movidius Implementation: Inter-shave Task Parallelism • Output feature map oriented strategy Put each shave in charge of several output feature maps, with load balanced among all shaves Input feature map oriented strategy each shave processor could take charge of “a band” of input feature maps, and compute all output channels of that spatial “band” The above strategies are employed according to each layer's specific configurations, to minimize the amount of data transferred.
- 20. FPGA and DNN - Pottwal Project
- 21. FPGA and DNN - Pottwal Project The detection of neural network (Faster_RCNN)The detection of neural network (Faster_RCNN) Softmax, NMS, Coordinate inversion and so on Most of the computation : Image(RGB) CONV Layers ROI projection Detection result Region Proposal Network (RPN) FAST
- 22. FPGA and DNN - Pottwal Project CONV Layers ROI projection Region Proposal Network (RPN) FAST Interface External memory interconnect • Global pipeline • Ping-Pong • Reduced data interaction • SIMD • Int8 Design Features
- 23. FPGA and DNN - Pottwal Project Performance • Up to 8 channels of 1080p@30 detection • Effective performance ：1.2T ops • PE computational efficiency ：87.2% • Latency：11.5ms Platform Performance(Effective ) Power Performance per Watt Our FPGA Platform 1.2T ops 7W 171.4G ops/W NVIDIA TX1 220G ops 10W 22G ops/W NVIDIA TK1 55G ops 10W 5.5G ops/W Movidius MA2450 40G ops 1.5W 27G ops/W
- 24. Vision without Limits! Vion Technologies Co., Ltd.
- 25. Vion Core Team Hai Tao, Dr., Founder&CEO Tsinghua Univ. BS'91, MS'93; UIUC PdD'99; Sarnoff 99-01; UCSC Assoc.&Tenured Prof. 01-10. US NSF 2004 Young Career Award. Pulished 150+ papers in CV, 10+ US patents. Jun Song, CTO Tsinghua Univ. Math, BS'01, MS'04; Responsible for all R&D work. Leads the smart traffic product core development & hardware system design. Xiang Zheng, Director, ITS Tsinghua, CS, BS'01, MS'04; CV algorithm expert; data department manager; Rich vision product experience. Fan Yang, Director, Smart Counting Tsinghua Univ. EE, BS'03, MS'06; Manager: business intelligence group; Manager: smart counting product line. Yu Lin, Director, Vision System Tsinghua Univ. AE, BS'03, MS'06; Manager: smart city product line; Manager: face recognition and intelligent video analysis group. Tianshu Wang, Product Director Xian Jiaotong Univ. BS'93,PhD'03; Microsoft Research 97-03, IBM Research 03-10; Lenovo Research 10-16, joined Vion in 2016.
- 26. Embedded CNN Structure • decrease the model size, less than 1 million params • limit the complextity to 1.5GMAC, < 2% of VGG
- 27. Embedded CNN Performance • Detection Rate >89% (FDDB) • 5% lower than VGG ( 0.2FP/frame) • Face detection scale from 20 pixels to 400 pixels • Detection Rate >83% for real unconstrained local scenarios (illumination, expression, occlusion, pose)